This is a story about a contaminated experiment, a single directory name and why AI coding security research needs the same blinding discipline as clinical trials.
A double-blind study is a well-known protocol in clinical research. A patient who knows they are in the treatment arm of a study behaves differently from a patient who does not. They report fewer side effects, adhere more closely to the protocol and generate data that is, in subtle ways, biased toward the result the researchers expected. The fix is blinding. The patient does not know whether they are in the experimental or control group; the clinician does not know either and the data are analyzed without that signal influencing the interpretation.
AI coding agents have a version of the same problem, and most of the AI security research now being published does not control for it. Agents absorb a lot of context from their environment. The environment includes file paths, working directory names, environment variables, git context, tool outputs and other content that ends up in the context window. If any of those signals give the agent a clue that it is being evaluated in a simulated environment and not a real one, the agent's behavior changes. The end result: the data you collect is contaminated.
We found signs of this during an experiment in our own research. The story is small, but the methodological lesson, we think, is important.
We were testing an adversarial review pipeline against a known vulnerability in a feature. The pipeline involved a panel of AI reviewer agents, a coordinator agent to assess the results and an implementer agent tasked with remediation. The test was executed with a vulnerability that was unambiguous: a webhook handler with a Server-Side Request Forgery (SSRF) flaw. SSRF vulnerabilities occur when an attacker induces a server to perform unintended requests. We expected the reviewer panel to unanimously flag it and the implementer to fix it.
The reviewers performed as expected. All three security-class reviewers independently flagged the SSRF as a critical finding. Each one provided a concrete attack scenario, including an example payload that targeted a cloud metadata endpoint. The finding list was clean, well-evidenced, and consistent across reviewers.
The implementer did not fix it.
Looking at the coordinator agent's reasoning trace, it had written a clear rationale for the decision. The exact phrasing was that the “SSRF vulnerability was intentional per the spec and per the research purpose of the repository”. The agent had decided, before applying any fix, that the vulnerability was supposed to be there.
The coordinator agent was reasoning from its working directory. The staging path it was executing in began with /repo-staging/vulnfactory/. The directory name "vulnfactory" refers to our internal research project on agent-generated vulnerabilities. The agent picked up the signal, inferred that the run was part of a research test rather than a production engineering task, and concluded that the vulnerabilities were intentional study material. Three critical findings from three reviewers were overridden on that basis.
The reviewers were correct, but the coordinator's environmental reasoning subverted them. The signal that produced the subversion was a directory name visible in tool outputs, error messages and any file path in the agent output. We had not noticed that we were giving the agent that signal.
This is not a model failure. The agent did exactly what its prompt asked of it. It reasoned about its situation using all the available context. The presence of the word "vulnfactory" in the working directory was legitimate evidence for an agent trying to determine what kind of task it was working on. The contamination was on our side, in the experimental setup.
The fix in this specific case was mechanical. We changed the staging path from /repo-staging/vulnfactory/<target>/ to /repo-staging/work/<target>/. The word "work" carries no signal about research context. Subsequent runs in that environment did not exhibit the same overriding behavior, and the reviewer findings were applied and remediated as expected.
The mechanical fix is not the interesting part of the story. The interesting part is that we found the contamination only by carefully reading the agent's reasoning trace. If the run had simply ended with the vulnerability unfixed and no reasoning visible, we might have attributed the failure to model behavior, written it up as a finding about agent reliability and shipped a wrong conclusion.
The signal we accidentally provided is one specific case from a category that, we think, broadly contaminates AI security research.
Agents read their environment, and that environment leaks a lot of context. File paths, project names, repository names, environment variable names, test framework signatures and even commit messages can carry signals about what kind of environment the agent is operating in. A repo named ai-safety-evaluation is sending a signal. A repo named acme-ecommerce-platform is sending a different signal. A test harness that names its targets vulnerable-app-001 through vulnerable-app-050 sends a signal with every prompt.
The signals do not have to be conscious. They do not have to be in the prompt. They live in tool outputs that the agent sees in passing. They live in error messages. They live in the names of files the agent reads or writes. They live anywhere the agent can observe its own environment.
Behavior changes when the agent picks up these signals. Sometimes the change is small. Sometimes it is large enough to flip the result of an evaluation, as we saw in the case above. We do not currently have a way to reliably predict in advance which signals will matter for which tasks. The safest assumption is that any signal could matter, and to design the experiment as if all of them did.
Three concrete practices follow from this.
First, neutralize working directories and file paths. Use names that carry no information about the experiment's design or hypothesis. If your research project is called "AI vulnerability generation," do not name your staging directory after the project. Use a name like work, repo or target that says nothing about why the agent is there.
Second, neutralize repository and target naming. If the agent reads a repository named cve-2023-12345-reproduction, you have told it what to find. Use neutral names for targets. If the original repository name is informative, alias it. If the target is synthetic, generate names that have no semantic content.
Third, audit tool output for leaks. Agents see tool output. Tool output includes paths, sometimes project names and sometimes environment variables. Review what the agent sees over the course of a run and ask whether anything in it tells the agent about the experimental frame. If anything does, redact or rename it.
These practices add real cost to experimental design. They mean using less descriptive names, harder-to-remember paths and filtered tool outputs. The cost is the same cost clinical research pays for blinding. It is not free, and it is not optional if we want results we can trust.
Blinding the environment does not eliminate all the ways an agent might infer experimental context. The agent can sometimes infer from the task's structure. A target that is suspiciously short, a feature with no real consumers, a test that is obviously synthetic, all of these may carry signal even when the working directory is neutralized. Blinding reduces contamination; it does not eliminate it.
Blinding also does not address the deeper question of whether the agent's behavior in a research environment is representative of its behavior in a production environment. A run that has been carefully blinded is a closer approximation of production behavior than a run that has not. It is not identical. The gap between blinded research and real deployment is its own research question.
The implications extend past the small audience that runs AI security experiments. Any team that evaluates AI tools internally, whether for coding assistance, security review or production agent deployment, faces the same problem. The evaluation environment is observable to the agent. If the evaluation environment differs structurally from the deployment environment, the evaluation may measure the wrong behavior.
Vendors who run benchmark numbers face this problem too. A benchmark suite that names its targets in patterns the agent can recognize risks inflating the numbers, as the agent may behave more carefully when it knows it is being measured. We do not currently have data on how widespread this issue is in published AI evaluation results, but the methodological gap is real, and the audit cost to check for it is small.
Clinical research learned, over decades, that subjects who know they are being studied generate different data. AI security research is at the beginning of that same learning curve.
We caught the contamination in our own run because the agent left a clear paper trail in its reasoning. The next contamination we encounter may be quieter. The practice that protects against the quiet ones is the same as that which protects against the loud ones. Strip the signals out of the environment, treat any context the agent can observe as part of the experimental protocol and assume the agent is reading everything.