01What AI data governance means for LLMs and RAG
AI data governance is the practice of controlling which data an AI system may use, who is allowed to see what it returns, and proving that those rules were followed. For LLMs and retrieval-augmented generation it has to be enforced at runtime, on every record and field, before the model sees the data, so the model cannot disclose information the asker was never entitled to.
Classic data governance is a design-time discipline. You build a catalog, classify columns, define stewardship, set retention, and attach policy to data at rest. That work is necessary and it is not going away. But it assumes a stable boundary: a known application reads a known table for a known user, and the application enforces who sees what. The catalog describes the data; the app guards the door.
LLMs and RAG dissolve that boundary. A retriever assembles a fresh, unpredictable document for every prompt by pulling whatever is most similar to the question, across many sources, and hands it to a model that has no concept of roles, ownership, or clearance. The model will faithfully use whatever it is given. So the governance question shifts from "is this dataset classified and stewarded" to "for this specific person, asking this specific question, right now, which fields are they permitted to receive." That question can only be answered at the moment of inference, and it has to be answered per field, not per table.
This is why we treat AI data governance as a distinct layer rather than a feature of the data warehouse. It is the set of controls that decide, in real time, what an AI system is allowed to read and what it is allowed to reveal, and it produces the evidence that those decisions happened. For a deeper look at the retrieval-specific risks, see our guide to RAG security.
02Why classic, build-time governance fails at inference
Build-time governance fails against AI not because it is wrong, but because it operates at the wrong moment and the wrong granularity. Three gaps open up the instant data starts flowing into a model.
It governs data at rest, not data in motion through a prompt. A catalog knows that a table contains protected health information. It does not know that, at 2:14pm, a support agent asked a chatbot a question whose closest match happens to be a psychiatric note. The classification exists; nothing consults it at the moment the note is about to enter the context window.
It assumes the application enforces access, but the model is not an application. Row-level security and column masks work when a query carries an authenticated user and a deterministic WHERE clause. RAG retrieval is a similarity search, not an authorization query. By the time text reaches the prompt it has usually been stripped of the access metadata that would have told you who may read it. The model then treats every token in its context as equally usable.
It is coarse where AI needs to be fine. Governance at the dataset or table level cannot express "this clinician may see the diagnosis but not the patient identifier in the same record." LLM disclosure happens at the level of the individual field and sentence, so the control has to operate there too. A policy that can only grant or deny a whole document will either over-share or make the system useless.
The result is a quiet but serious exposure: organizations that have invested heavily in data governance still ship AI features that can surface, to the wrong person, exactly the data their governance program was built to protect.
03The core failure mode: relevance is not permission
There is one failure at the center of nearly every RAG data incident, and it is worth stating plainly. A retriever is built to find what is relevant to the question. It is not built to find what the asker is permitted to see. Those are different properties, and naive systems conflate them.
Relevance is not permission.
Naive RAG assumes that relevance implies safety: if the system retrieved a passage, the reasoning goes, it must be acceptable to show. But similarity scoring has no idea who is asking. Ask "what were the results of the board's compensation review" and the most relevant chunk may be the unredacted executive comp memo, regardless of whether the asker is a director or a summer intern. The retriever does its job perfectly and surfaces the most on-point document. That document is exactly the one that must not be shown.
The same dynamic appears everywhere sensitive data lives next to ordinary data: a clinician's RAG assistant pulling another patient's record because it best matches the symptom described; a customer-support copilot retrieving an internal incident postmortem because it mentions the same error string; a financial analyst's tool surfacing material non-public information because it is the most relevant answer to a market question. In each case the system is not malfunctioning. It is succeeding at relevance and failing at permission.
04The principle: enforce authorization before augmentation
If unauthorized content never enters the prompt, the model cannot leak it. This is the single most important principle in AI data governance, and almost everything else follows from it.
An LLM can only disclose what is in its context, its weights, or its tools' reach. You cannot economically retrain disclosure out of a foundation model, and you cannot fully predict what it will say. But you can control its context completely. So the durable strategy is to govern the input, not police the output: decide authorization first, then augment the prompt with only what survives that decision.
We call this "enforce authorization before augmentation." In practice it means the access decision happens between retrieval and prompt assembly. A record is fetched, every field in it is evaluated against the asker's clearance, unauthorized fields are removed or masked, and only the permitted remainder is allowed into the context window. The model receives a document that is already safe to use, because everything the asker may not see was withheld before the model ever existed in the loop.
This inverts the usual instinct. Most teams reach first for an output guardrail: let the model answer, then scan the answer for anything sensitive. That feels intuitive and it is the wrong place to stand, for reasons we cover below. Withholding at the input is structural. It does not depend on recognizing a leak; it removes the possibility of one.
05The four control points of AI data governance
A complete program is not one switch. It is four control points that together cover the path from data store to answer. Each addresses a distinct failure, and a gap in any one undermines the others.
1. Clearance-aware retrieval
Retrieval must filter by who is asking, not only by what is similar. Every candidate record carries the access metadata of its source, and the retriever drops anything above the asker's clearance before it is ever ranked into the context. This is where "relevance is not permission" gets enforced. See permission-aware RAG for the detailed pattern.
2. Selective disclosure and field-level redaction
Authorization is rarely all-or-nothing at the document level. A record that is mostly safe may contain one field the asker may not see. Field-level redaction evaluates each field independently and masks or removes only the parts that exceed clearance, so the useful remainder still reaches the model. This is selective disclosure: the same record yields different, correct views for different roles.
3. Agent guardrails
Agents do not just read; they call tools, chain steps, and act. Governance has to extend to what an agent may invoke and what data may cross between tool calls, so a permitted first step cannot be composed into an unauthorized result. The clearance that governs retrieval must also govern the agent's tool surface.
4. Audit-ready evidence
Every decision to pass or withhold a field must be recorded as evidence that can be verified later, ideally offline and without trusting the vendor. Without this, you can assert that governance happened but you cannot prove it, and an unprovable control is worth little to an auditor or a regulator.
06Runtime prevention versus output filtering
The most consequential design choice in AI data governance is where you put the control: before the model, as prevention, or after it, as detection. They are not equivalent, and the difference decides whether your system fails open or fails closed.
Output filtering is detection. You let the model generate, then run a classifier or a second model over the answer to catch sensitive content. The problem is asymmetry: the filter has to be right every single time, while a leak only has to slip through once. Output classifiers are probabilistic and they degrade against paraphrase, translation, encoding, summarization, and partial disclosure. They also see only the text, not the authorization context, so they are guessing at sensitivity rather than knowing it. When a detector is unsure, the content has usually already been produced, which means it fails open.
Runtime prevention removes the data before generation. If the unauthorized field never enters the prompt, there is nothing to detect, nothing to paraphrase around, and nothing to leak. The control is deterministic: the same inputs always produce the same verdict, so it is explainable and reproducible. And it can fail closed, blocking when it cannot reach a confident decision rather than letting data through. Prevention beats detection because it changes the math from "catch every possible leak" to "never admit the data in the first place."
| Property | Runtime control (prevention) | Data catalog (design-time) | Output filtering (detection) |
|---|---|---|---|
| When it acts | At inference, before the prompt | At design time, on data at rest | After generation, on the answer |
| Granularity | Per field, per role | Per dataset or column | Per output string |
| Knows who is asking | Yes | Partially | No |
| Failure mode | Fail-closed | N/A at runtime | Fails open |
| Determinism | Deterministic | Deterministic | Probabilistic |
| Stops the leak | At the source | No | After it exists |
None of this means a catalog or an output filter is useless. A catalog is the right place to classify and steward data, and an output filter can be a sensible backstop. But neither is a substitute for enforcing authorization before augmentation. Treat prevention as the load-bearing control and the others as support.
07Evidence: a log you control versus a chain you cannot forge
Prevention stops the leak. Evidence is how you prove the leak was stopped, and how you reconstruct what any role could see on any day. This distinction matters more than most teams realize, because not all records of a decision carry the same weight.
A log you control is a claim. An ordinary application log is mutable. You can edit it, rotate it, lose it, or backdate it, and so can anyone with access. When an auditor or a regulator asks whether a field was withheld from a given role on a given date, a log you could have altered is an assertion, not proof. It depends entirely on trusting the party that produced it.
A sealed, hash-chained record is evidence. When every decision is signed and each entry is cryptographically linked to the one before it, the record becomes tamper-evident: altering or removing any entry breaks the chain in a way anyone can detect. If that evidence is content-free, holding only verdicts, hashes, and signatures rather than the underlying records, it can be shared and verified without leaking the very data it protects. And if it can be verified offline, independently, without contacting the vendor, then it does not rest on trust at all. It rests on math.
Content-free, signed, hash-chained evidence also solves a tension unique to AI governance. The thing you most need to prove is who could see sensitive data, but the proof itself must not become a new copy of that sensitive data. Recording verdicts and hashes rather than content lets you demonstrate exactly what was withheld without ever re-exposing it.
08How this maps to HIPAA, SOC 2, and SR 11-7
AI data governance is not a compliance framework, but it is the mechanism through which several frameworks get satisfied when AI touches regulated data. The controls above map cleanly onto requirements that auditors and regulators already understand.
HIPAA turns on the minimum-necessary standard and access controls for protected health information. Field-level redaction by role is a direct expression of minimum-necessary: each clinician's assistant receives only the fields that person is entitled to, and the evidence ledger shows it. For the healthcare-specific architecture, including FHIR R4 classification and the minimum-necessary mapping, see our guide to HIPAA-compliant AI. Custosa's controls are designed to map to HIPAA; they do not make a deployment "HIPAA certified," and compliance remains the customer's responsibility.
SOC 2 rests on trust-services criteria for security, confidentiality, and processing integrity, with an emphasis on evidence that controls operated over a period. A deterministic policy engine plus a tamper-evident, content-free ledger gives an auditor exactly that: a reproducible decision and a record that the decision happened on every request. The same applies to SOC 1 where AI participates in financially relevant processing.
SR 11-7 and the related FFIEC and ICFR guidance govern model risk in financial institutions, demanding explainability, controls, and an audit trail around models that affect decisions or reporting. Deterministic, per-field authorization is inherently explainable, and signed evidence supplies the audit trail. For the financial-services treatment, including SR 11-7, Open Banking, and X12 claims, see AI compliance for financial services.
In every case the pattern is the same. The framework asks for least-privilege access, an explainable decision, and durable proof. Runtime, deterministic, evidenced authorization supplies all three.
09Build versus buy
Standing up a convincing demo of governed RAG takes an afternoon: tag a few documents, filter retrieval by a role claim, mask an obvious field. Standing up a control you can prove to an auditor, that holds across formats, and that fails closed under load is a different undertaking. The gap between the two is where most build efforts stall.
A durable system needs all of the following, not some of them:
- A deterministic policy engine so verdicts are explainable and reproducible, not a model guessing at sensitivity.
- A real clearance model, typically a multi-level lattice, rich enough to express the roles your data actually has.
- Field-level redaction with selective disclosure, not document-level allow or deny.
- Fail-closed behavior as an invariant, so an unreachable resolver or a policy timeout blocks rather than leaks.
- Tamper-evident, content-free evidence that verifies offline, because a log you control is only a claim.
- Coverage of the long tail of formats, including FHIR R4, X12 EDI claims, and Open Banking, which teams routinely underestimate.
Build makes sense when governance is your core competency and you intend to own this surface for years. Buy makes sense when you need the evidence and the guarantees sooner than you can responsibly build them, and when you would rather your team ship product than maintain a policy engine and a cryptographic ledger. The deciding question is usually not "can we build it" but "can we prove it, on every request, to someone who is paid to be skeptical."
10How Custosa implements it
Custosa is a runtime data-control plane for enterprise AI. It sits between your data and the LLM and inspects every record and field at runtime, before the model sees it, which is exactly where the four control points have to live. The thesis throughout is the one this guide keeps returning to: relevance is not permission, so authorization is enforced before augmentation.
The data plane runs inside your environment, so records never leave your boundary. Decisions are made by a deterministic formal policy engine, Cedar, not a model or a heuristic, so the same inputs always produce the same verdict and the system is fail-closed: it blocks when it cannot reach a decision. Verdicts are per field, PASS or REDACT, by role, through a five-level clearance lattice, so sensitive fields are withheld before they ever enter the prompt. The model cannot leak what it never received.
Every decision is signed with HMAC-SHA256 and hash-chained into an append-only, tamper-evident, content-free evidence ledger that you can verify offline without contacting Custosa. The managed control plane receives only content-free verdict evidence, never your records, and self-managed, on-premises, and air-gapped options exist along with a FIPS build. Coverage includes a HIPAA Pack, SOC 2 and SOC 1 Packs, FHIR R4 auto-classification, X12 EDI claims, Open Banking, and mapping to SR 11-7, FFIEC, and ICFR controls. The p99 added-latency target is 50 to 110ms.
- AI data governance is a runtime problem. Once data flows into an LLM or RAG, the question is which fields this person may see right now, answerable only at inference and only per field.
- Relevance is not permission. Retrievers find what is similar, not what is allowed; treating relevance as safety is the root cause of most RAG leakage.
- Enforce authorization before augmentation. If unauthorized content never enters the prompt, the model cannot leak it.
- Prevention beats detection. Withholding fields at the input is deterministic and fails closed; output filtering is probabilistic and fails open.
- A log you control is a claim; a sealed chain is evidence. Content-free, signed, hash-chained records prove what was withheld without re-exposing it.
See authorization enforced before the model
Watch Custosa evaluate every field at runtime and withhold what a role may not see, before the prompt is ever built.
Frequently asked questions
What is AI data governance?
AI data governance is the practice of controlling which data an AI system may use, who is allowed to see what it returns, and proving that those rules were followed. For LLMs and RAG it means enforcing access, redaction, and evidence at runtime, on every record and field, before the model sees the data, so the model cannot disclose information the asker was never entitled to.
How is governing AI data different from traditional data governance?
Traditional data governance is built at design time. It catalogs, classifies, and sets policy on data at rest, then trusts each application to enforce access. RAG and agents break that model because retrieval assembles a new, unpredictable document for every prompt and the model itself has no concept of permission. AI data governance therefore has to move enforcement to the moment of inference and apply it per field, not just per table or per dataset.
What is RAG data governance?
RAG data governance is the subset of AI data governance that controls retrieval-augmented generation. Its central job is to make sure the documents and fields pulled into a prompt are ones the specific asker is authorized to see. Relevance is not permission. A naive retriever ranks by similarity and will surface the most relevant passage even if it sits behind a clearance the user does not hold, so authorization has to be enforced at retrieval and again as field-level redaction before the context is built.
Does AI data governance prevent data leakage?
It can, if it is enforced before augmentation rather than after generation. The reliable defense is structural: if unauthorized content never enters the prompt, the model cannot leak what it never received. Approaches that inspect the model's output after the fact are detection, not prevention, and they fail open because they depend on catching every paraphrase, translation, and partial disclosure. Withholding sensitive fields before the prompt removes the leak at its source.
Is prevention better than detection for LLM data?
Yes. Detection asks a model or filter to recognize sensitive content after it has already been generated, which means the system has to be right every time while an attacker only has to win once. Prevention removes the unauthorized data from the prompt, so there is nothing to detect and nothing to leak. Prevention is also deterministic and explainable, while output classifiers are probabilistic and degrade against rephrasing and obfuscation.
Should we build or buy AI data governance?
Building a demo is easy; building a control you can prove to an auditor is not. A durable system needs deterministic policy, a five-level clearance model, field-level redaction, fail-closed behavior, and tamper-evident evidence that verifies offline. Most teams can assemble parts of this but underestimate the evidence and the long tail of formats like FHIR and X12. Buying makes sense when you need the audit trail and the guarantee that enforcement runs before the model, not after.