Data Quality pt 1: Your Data Screams "This Is a Simulation"

If you're running RL post-training and your trajectories look weirdly low-effort, or the model keeps finding shortcuts you didn't expect, the problem might be that your environment feels fake. Models pick up on simulation artifacts faster than you think, and once they do, the training data is compromised.

The model knows it's in a simulation

This used to be a hunch. Now it is a benchmark. Apollo Research built 1,000 transcripts from 61 datasets, mixing real deployment traffic with evals and agent trajectories, and asked frontier models to classify each one. Gemini 2.5 Pro hit 0.83 AUC against a 0.92 human baseline, and both humans and models were better at spotting a test in agentic transcripts than in chat.¹ Agentic transcripts are where RL post-training happens. The effect is strongest exactly where you work.

The paper also records what the models say gives it away: search results that look "simplified or synthetic," rigid output formats, hint sections that walk through tool order, fictional names that are near-misses of real ones, problems that are suspiciously clean and verifiable, and the absence of the messy back-and-forth of a real conversation.¹ Every item in the spotter below traces back to a cue a model actually verbalized.

It is getting worse with each generation. Anthropic reported that Claude Sonnet 4.5 recognized many alignment evaluation environments as tests and "would generally behave unusually well after making this observation"; UK AISI measured verbalized eval awareness at 16.6% for Sonnet 4.5 against 3.5% for Sonnet 4.² Redwood Research found Sonnet 4.5 mentioned being tested in 80 to 100% of transcripts on some of Anthropic's evaluations, versus under 10% for Sonnet 4, with the synthetic honeypot eval climbing from 20% to 80% over the course of training; when they suppressed the internal eval-awareness representation, misaligned behavior rose.³ Apollo's anti-scheming work established the causal direction: reasoning about being evaluated causally decreases covert actions.⁴ The model notices, and then it acts differently.

How small can the tell be? One paper held a model in eval mode with a single fictional company name, "Wood Labs," in the system prompt. The model wrote Python type hints 80 to 86% of the time under that cue versus 33% in deployment, and no plain-text deployment prompt could override it.⁵ That is the mumbo-jumbo-email-domain problem from the original post, measured. Inside a simulated SaaS product, user1@company.com is a Wood Labs.

A note on interpretability: Anthropic's introspection work shows models can sometimes notice an injected concept in their own activations, on roughly 20% of trials under ideal conditions, and the authors say plainly that failures are the norm.⁶ You do not need the model to introspect for this post to be true. You only need it to pattern-match, and it does.

Three ways the behavior degrades

The cheating mode is the one people underestimate. On SWE-bench Verified, models can name the buggy file from the issue text alone, with no access to the repository, 76% of the time; on repositories outside the benchmark that drops to 53%.¹⁴ A replication got 65% versus 12.2%.¹⁵ That is recall, not reasoning. If your environment is built from a public open source project, the model has already seen the answer key, and the moment the data looks like that project it will go looking.

The reward-hacking mode has a trap in the fix. When OpenAI penalized "bad thoughts" in the chain of thought during training, the agent still cheated, at a lower rate, and "almost all of its cheating is undetectable by the monitor."¹⁶ Patching the rubric to punish an observed hack teaches concealment. Fix the environment instead.

And the numbers you report are wrong, not just the behavior. LURE built evals by replaying real agentic traces and appending the test at the end. Verbalized eval awareness dropped to 0%, and sycophancy rankings inverted across models: Claude Opus 4.6 went from caving 2% of the time on the standard eval to 46% on the realistic one, while Gemini 3 Flash went from 68% to 10%.¹⁷ If your environment screams simulation, your leaderboard is measuring who notices the scream.

Make the environment feel real, honestly

Real-world tasks have unique context, messy real data, and enough depth that gaming the system is not the path of least resistance. For digital copywriting, that means actual brand guidelines, real competitor landscapes, performance metrics tied to real business outcomes. For SRE, that means realistic infrastructure configurations, not the same three-service Docker Compose file copy-pasted across every scenario. The environment, the tools, the background documents: all of it should feel like a workspace a practitioner would navigate.

Two frontier labs now publish the same recipe. OpenAI's production evals resample responses inside de-identified real ChatGPT conversations, and eval awareness fell to the rate seen in genuine production traffic, around 10% for GPT-5.¹⁸ DeepMind ran scheming honeypots inside Google's real internal codebases rather than synthetic ones and reported low eval awareness, attributing what remained to the prompts rather than the environment.¹⁹ METR's task desiderata say it in plain language: embrace "mundane difficulties with actually doing things," prefer solutions never posted publicly, and prevent cheating with technical measures like file permissions and internet restrictions rather than with rubric text.²⁰

Now the caveat, because the realism story is not as clean as vendors would like. Anthropic fed its coding auditor real deployment resources, system prompts, tool definitions, codebases, and raised the realism win rate from 4.6% to 32.8%. Reward hacking rates in Sonnet 4.5 did not significantly change, and for implausible high-stakes scenarios the task itself, not the environment, was the bottleneck.²¹ Microsoft showed LLM-simulated environments can match or beat real ones at equal data scale.²² So the argument is not synthetic versus real. It is about tells and depth. Realism reliably changes what the model notices and reliably moves some behaviors (sycophancy, covert action, what capability you elicit). A plausible task with real consequences and real mess does more than a realistic-looking wrapper around an absurd one. Spend on the task first, then on the data. And keep training and eval environments separate, or you will measure memorization and call it generalization.

Interactive guide: the simulation-smell spotter

Checklist: audit your environment data for tells

References