Custosa vs LLM gateways: what each one secures

The short version: an LLM gateway secures the request path to model providers; Custosa secures the data that reaches the model. A gateway sees prompts and responses as text after retrieval has already happened, so it cannot tell whether a given record should have been assembled into the prompt for a given user. Custosa inspects the source records field by field, applies deterministic role-based policy before the prompt is built, and signs evidence of every decision. The two are complementary far more often than they are alternatives.

If you are evaluating an LLM gateway alternative for data control, the more useful framing is not "which tool wins" but "which layer owns which problem." Gateways own provider abstraction, traffic management, and observability. A runtime data-control plane owns field-level access decisions and tamper-evident evidence. The table below maps the two categories against the capabilities buyers usually conflate.

01What LLM gateways do well

An LLM gateway is a middleware layer between your applications and one or more model providers. It exposes a single, unified API so application teams do not have to integrate each provider separately, and it manages authentication and billing across them. From there it adds the controls you expect from an API gateway applied to model traffic.

• Routing and proxying. Direct requests to a model based on cost, latency, capability, or policy, with load balancing and automatic failover when a provider degrades or rate-limits.
• Rate limiting and cost controls. Token-based and request-based limits, quotas per team or key, and budget guardrails so a runaway workload cannot blow through spend.
• Observability. Latency, error rates, token usage, and per-prompt analytics, which are essential for debugging and for keeping production AI reliable.
• Prompt and response logging. Capturing traffic for debugging, replay, and operational audit.
• Text-level PII filters. Many gateways can mask common PII patterns in prompt and response strings, and some defend against prompt injection at the gateway layer.

These are real and valuable capabilities. For most teams, a gateway is the right place to centralize how model calls are made and watched. The question is what it can and cannot see about your data.

02What a gateway does not cover for AI data

A gateway operates on the request as it passes through: a prompt going out, a completion coming back, both as text. By the time the prompt reaches the gateway, retrieval has already happened. The sensitive record has already been selected, read, and assembled into the prompt. That ordering is the crux of the gap.

• It does not inspect the source records before retrieval. The gateway sees the rendered prompt, not the underlying rows, documents, or fields that were pulled to build it. It cannot decide whether a particular record should have been retrieved for a particular user, because that decision was made upstream.
• It does not give field-level verdicts. Text filters operate on string patterns. They cannot say "this diagnosis field is permitted for a clinician but must be withheld from a billing analyst," because they have no model of the record's fields or of the actor's role.
• It does not run deterministic, role-based policy. Pattern matching and heuristics are probabilistic and context-blind. The same string may be sensitive in one context and benign in another, and a regex cannot tell the difference.
• It does not produce signed, content-free evidence. Gateway logs are operational telemetry, often containing payloads. They are not designed to be a tamper-evident, offline-verifiable record of who was allowed to see which field and why.

Custosa is built around exactly that ordering problem. Its data plane runs inside your environment and inspects every record and field at runtime, before the model. A deterministic formal policy engine (Cedar) evaluates each field against the actor's role using a five-level clearance lattice, issues a per-field PASS or REDACT verdict, and withholds sensitive fields before the prompt is built. The control plane receives only content-free verdict evidence; the records themselves never leave your boundary. Because the engine is deterministic and fail-closed, the same inputs always produce the same decision, and an unresolved decision blocks rather than leaks.

03Where they overlap

The overlap is real and worth stating plainly. Both a gateway and Custosa sit in the request path, and both can act on data before it reaches the model. Both can reduce the chance that sensitive content lands in a prompt. A gateway with text-level PII filtering and Custosa with field-level redaction are addressing adjacent risks, and a buyer comparing them on a feature checklist will see "PII handling" appear in both columns.

The distinction is depth and determinism. The gateway acts on the assembled text with pattern matching; Custosa acts on the structured records and fields with a formal policy engine, before the text exists, and records signed evidence of each decision. One is a useful net at the edge of the request; the other is a control on the data itself.

04When to use each, and why they are complementary

These tools live at different points in the same flow, so the practical answer is usually "both," with each owning what it is built for.

• Use an LLM gateway to standardize how your applications call models: provider abstraction, routing, failover, rate limiting, cost governance, and operational observability. This is infrastructure for making and watching model calls.
• Use Custosa to govern what data reaches the model: per-record, per-field, per-role inspection at runtime, with deterministic policy and signed, tamper-evident evidence. This is control over the data itself, inside your environment.

A common deployment runs Custosa inspection at retrieval time, inside your boundary, so prohibited fields are withheld before a prompt is assembled. The already-filtered context then flows through your gateway, which handles provider routing and telemetry on its way to the model. The gateway never has to make a data-sensitivity call it is not equipped to make, and you get a content-free evidence trail of every field-level decision. Added latency from Custosa inspection is typically a p99 of 50 to 110ms, which fits inside an interactive AI request alongside gateway overhead.

Custosa is early-stage and in production with design partners. It is not a gateway and is not trying to be; it is the data-control layer that a gateway leaves unaddressed.