Some data should never reach the model, even in an AI-native workflow

Here is the practical governance question most security-conscious teams have been asking for the last few years: "Will a human paste this into a chat window?"

That was a reasonable question. It put the accountability somewhere human. And for a while, it was actually enough.

It isn't anymore.

The tools changed. A folder syncs. A workspace gets indexed. A coding assistant pulls in the whole project directory so it can "understand the codebase." Nobody typed anything. Nobody copy-pasted anything sensitive. The data was sitting there, and the tool swept it in because that is exactly what the tool was designed to do.

That shift changes the threat model completely. You are no longer defending against a distracted engineer who does something careless. You are defending against a tool doing its job correctly. The machine is not misbehaving. It is working.

A Behavioral Rule Cannot Solve a Structural Problem

There is a class of data that some teams have decided will never touch an LLM's context window. Not "probably shouldn't." Not "avoid when possible." Never. And the moment you commit to "never," you have to reckon with a pretty uncomfortable question: what is actually preventing it?

If the answer is "our policy says not to," you do not have a guarantee. You have a description of intent. Those are not the same thing.

A policy without a technical guarantee behind it is like a fence with no posts. It tells you where the boundary should be. It does nothing to hold the boundary in place.

The teams I have seen handle this well have stopped trying to win with behavioral rules and started treating AI context ingestion as its own hazard class. Same category as "reachable by an external network." Same architectural seriousness. The question they ask is not "will someone do something careless?" It is "can this data physically be reached by any tool that feeds an LLM?" If the answer is yes, the architecture is wrong, regardless of what the policy document says.

In practice, the structural answer involves three things:

• Separate storage. Sensitive data lives somewhere physically distinct from any directory, bucket, or workspace that any AI-assisted tool can sweep during normal operation.
• Explicit exclusion from every sync and indexing path. Not just the one tool you are thinking about right now. Every tool. Audited on a schedule, because vendors add new sync features and they are not in the habit of sending you a heads-up when they do.
• Access controls that account for non-human readers. Traditional access control asks which humans should see this. The updated question is which systems can reach this, and do any of those systems feed an LLM. If the answer to the second part is yes, the access model has a problem regardless of how clean the first part looks.

What This Looks Like When You Actually Go Looking

Here is a useful exercise. Trace every location where sensitive data can plausibly live. For each one, ask whether any tool with access also feeds data into an LLM. If the answer is "maybe" or "I would have to check," you have found a gap.

The failure modes that come up when teams do this for real are almost never exotic. They are boring intersections of normal tool behavior and data that should have been isolated from the start.

A few that show up repeatedly:

• Log aggregation pipelines that capture more than intended, landing in a workspace a developer tool happens to index.
• Shared scratch directories that someone started using for convenience, which are also the root an AI assistant is monitoring for context.
• Auto-archiving features that move old files somewhere new, where they become discoverable by a tool that was not even in the stack when the file was first created.

Nobody designed these failure modes. They emerged from normal tool operation running into data that was never correctly isolated. That is the thing worth sitting with: you do not need a bad actor. You do not even need a mistake. You just need two tools doing their jobs at the same time in the same general vicinity.

If "Never" Has to Mean Something, the Architecture Has to Mean It

When a team makes the call that a specific category of data must never enter an LLM's context window, the only version of that commitment that holds is one where the system makes it impossible rather than inadvisable.

That means storage decisions get made with AI reachability in mind from the beginning, not retrofitted after the fact when someone notices a problem. It means every tool that ingests context is explicitly configured to exclude the relevant paths, that configuration lives in version control, and someone reviews it on a schedule. It means access control reviews ask about system-level access, not just human-level access.

This is stricter than most teams start with. That part is just true. But it is also the only version that holds once AI-assisted tooling becomes a normal part of the workflow, which for most engineering teams is a description of the present, not a prediction about the future.

The old default assumption was that data is safe unless something actively exfiltrates it. The new one, for any team running AI tooling in their stack, is that any data that is reachable will end up in a context window through normal operation without any deliberate human action.

Work backward from that. Design the boundaries so the sensitive category is unreachable by definition. Because "don't paste this into a chat" stopped being sufficient a while ago, and most governance frameworks have yet to catch up.