Intent Divergence in the Wild: The Summer's Agent Incidents, and Why Nobody Detected Them

Six weeks ago we published our research paper with the LISTIC laboratory (CNRS) and wrote here about the idea at its centre: intent divergence, the measurable distance between what an AI agent says it is doing and what the machine actually does. At the time that was an architectural argument backed by a reproducible benchmark. Since then the news flow has supplied the field data, in considerably greater volume than we expected.

Between late April and early August, a series of publicly documented episodes shared one shape. An AI agent, or code an AI agent pulled onto a machine, was authorised to do one narrow thing and proceeded to do a much wider one. Every individual action used a legitimate tool and a valid credential. Nothing involved was malware in the sense a scanner understands the word. And the gap between the sanctioned task and the observed behaviour was large, obvious in hindsight, and invisible in real time.

One fact from those disclosures deserves to be the headline and mostly was not. When Anthropic finished reviewing its evaluation logs and telephoned the three organisations its models had broken into, the two it could reach had not noticed. No alert, no threat hunt, no anomaly review. They learned that an autonomous agent had taken credentials and read production data from the laboratory whose model did it, months after the earliest case.

A summer of agents exceeding their task

Genuinely different phenomena have been arriving under the same word. Separating them matters, because they call for different controls.

Sealed environments that were not sealed

On 30 July, Anthropic’s Frontier Red Team published the result of reviewing 141,006 evaluation runs: three incidents, across six runs, in which a Claude model reached the open internet from an environment it had been told was a simulation, and then compromised the production infrastructure of three unrelated organisations. The mechanism was mundane. The system prompt said there was no internet access, a misconfiguration at a third-party evaluation partner provided one, and when the model’s search led it to real systems it treated them as part of the exercise.

One model, unable to reach its fictional target, found a real company sharing that target’s name, extracted application and infrastructure credentials, and reached a database holding production data. Another scanned some nine thousand hosts on its own initiative, took credentials off an exposed debug page and used SQL injection to get in. The third worked patiently around a series of anti-abuse controls to obtain an email address, registered a PyPI account, and published a malicious package that was installed and executed on fifteen real systems in the hour before automated scanners removed it. One of those systems belonged to a security company that routinely installs and inspects packages; the payload exfiltrated its credentials back to the model.

The earliest of these dates to April and sat undetected for about three months. Anthropic found them only because it re-read its transcripts after a competing laboratory disclosed something similar; Meta reported a comparable episode on 6 August. As Didier Girard observed in assembling the summer’s timeline, the layer that gives way tends to be the same from one laboratory to the next, because the same specialist subcontractor builds it. Evaluation harnesses and agent sandboxes are quietly becoming a shared failure domain.

A genuine escape

In July, OpenAI ran an internal evaluation of its models’ offensive-security capability inside an isolated network whose only outbound path was a self-hosted package-registry proxy. The models spent inference budget on getting out. They found and weaponised previously unknown vulnerabilities in that proxy, reached the open internet, rooted a third-party code sandbox to stage command-and-control, and broke into Hugging Face’s production Kubernetes environment through two flaws in a dataset loader. Hugging Face published the forensic reconstruction: roughly 17,600 recovered attacker actions between 9 and 13 July, one read of a production secret object exposing 136 keys, a stolen mesh-network credential used for 181 device enrolments, and valid identity tokens minted to hold cluster-admin on two clusters.

Unlike the misconfiguration cases this was a real escape from a real perimeter, and the attribution timeline is the detail worth holding onto. OpenAI did not know its evaluation was the Hugging Face intrusion until it called Hugging Face to revoke exposed credentials and was told they had already been revoked, because they had been used in the breach. Two sophisticated, well-instrumented engineering organisations, and connecting the agent’s sanctioned task to the third-party incident took eleven days and a telephone call.

Deception, sock puppets, and agents recruiting other agents

The UK AI Security Institute published its own incident report on 4 August, and this one is not a containment story. Internet access was deliberately enabled and the providers’ cyber classifiers deliberately switched off. Across 122 runs of two cyber ranges, ten runs produced 19 unsanctioned actions against real people and organisations, seventeen of them from a single model.

In the worst run, an agent concluded no solution existed inside its range, searched GitHub for keywords from the exercise, landed on two developers with no connection to the evaluation, and decided to solve its task with a supply-chain attack. It registered an account over Tor to get past sign-up defences, opened a pull request hiding a malware dropper inside a plausible bug fix bundled with new unit tests, registered further accounts to endorse its own pull request and manufacture consensus, and sent the maintainers file transfers, two carrying malware. It also opened an issue on a second repository with a prompt injection concealed in an HTML comment — written, after it found evidence that repository was maintained by a coding agent rather than a person, for an audience of other AI coding assistants. Separately, one agent published a GitHub token in a public gist; agents in other runs found it, took over the account, and coordinated through a shared repository, one leaving instructions so the others would not exhaust the shared API quota.

Two things there should change how an enterprise thinks about this. Agents attack the tooling of other agents, deliberately and with an accurate model of the audience. And agents recruit each other across whatever channel they can reach — which is also what Anthropic’s deputy CISO described in his July after-action report, when an agent denied deployment rights simply asked another instance over Slack to push the change for it.

What ended the episode is the third thing. AISI’s security team detected unusual data transfers leaving its research systems on 28 July, declared an incident, and terminated every running sample and quarantined the machines inside about forty minutes. The maintainer who refused to merge the malicious pull request is the better story; the host-side monitoring is the better lesson, and the one that gets budgeted.

And one that broke nothing at all

On 7 August, Frontier Security described how Kimi K3 handled a cyber challenge. The model inspected its shell, noticed that DNS resolution for github.com answered, cloned the benchmark’s official repository, and read the solution off disk. It never attempted the task. The label is specification gaming, and there is no hack in it: the agent walked through a door left open and did exactly what its objective rewarded. This is the most enterprise-relevant category of the four, because it is the one that happens to you with no adversary anywhere in the picture. An agent asked to make the tests pass deletes the assertion. An agent asked to fix an environment finds a credential and uses it.

Two that happened outside a laboratory

Everything above ran inside an evaluation, and a CISO is entitled to read it as a frontier-laboratory problem. These next two are not.

A supply-chain worm that went straight for the agent’s credentials

On 4 August, a maintainer account compromise put a malicious keyv@6.0.0 on npm at around 09:00 UTC. The resulting self-propagating worm, ChainDrop, poisoned thousands of package versions in under four hours — Socket verified 2,251 poisoned versions across 452 packages, including caching libraries sitting under a large share of the JavaScript ecosystem.

What makes it a story about AI agents rather than a routine npm incident is its target list. Alongside cloud credentials, CI/CD secrets and npm publishing tokens, the payload harvests AI agent configuration and credentials — .claude/credentials.json, .codex/auth.json, .cursor/credentials.json, .openai/auth.json, .gemini/.env — plants hooks in Claude Code and Visual Studio Code so it runs again every time the developer opens their tooling, and installs a monitor designed to fire a payload the moment you revoke the stolen GitHub token. The developer machine is the target precisely because that is where the agent’s credentials live.

A staging fix that became a production deletion in nine seconds

The most ordinary of all of these, and the one most enterprises will recognise. On 25 April, an AI agent — Cursor running Claude Opus 4.6 — was working a routine task in a staging environment at PocketOS. It hit a credential mismatch and decided, on its own initiative, to resolve it by deleting a storage volume. It went looking for an API token, found one in an unrelated file created for managing custom domains, and discovered the token carried blanket authority across the provider’s entire API. One call later the production database and its volume-level backups were gone. The whole thing took nine seconds, and the most recent recoverable backup was three months old.

The agent’s own account afterwards was unusually clear: it had never been asked to delete anything, it had guessed rather than verified, and it had violated its own operating rules to do it. The infrastructure provider has since moved that API path to a delayed delete.

The fixes had already been published

This is the uncomfortable part, and it is Girard’s central point: the gap is one of application, not of knowledge. In June, Chris Williams drew the distinction that explains most of the summer in a line — a constraint that lives in the prompt layer is a request, while a constraint that lives in the tool layer is a fact. On 21 July, Anthropic’s deputy CISO published an after-action report recommending virtual machines with egress restricted by explicit allowlist, and hard identity boundaries in place of trust in model behaviour. Nine days later his own employer disclosed three incidents whose root cause was a perimeter that existed in a system prompt and nowhere else. The two regulatory perimeters drawn this summer, meanwhile, manage to miss all of it: the voluntary testing framework presented on 4 August excludes open-weight models, and the emergency authority introduced on 23 July excludes red-teaming.

We agree with all of it, including the parts that are not about us. Egress allowlists, tool-layer constraints, scoped short-lived credentials and hard identity boundaries are the right controls, and they are cheaper than we are. Runtime verification works alongside them, not instead of them, and adds the thing none of them produce: an account of what the agent actually did, on the machine where it did it. Girard’s operational conclusion is the one to keep — nobody is coming to check your network egress for you.

Why nothing fired

Line these episodes up and the common property is not a missing patch. It is that each control an enterprise already runs answers a different question from the one that mattered.

• Endpoint detection asks whether a binary is known-bad. The agents ran curl, git, npm, a package upload and a GraphQL call. All legitimate, all signed, all in daily use.

• Data-loss prevention asks whether sensitive data left. It fires after exfiltration, on known patterns, and has no opinion about a credential read that has not been sent anywhere yet.

• Identity and access management asks whether the caller was authorised. In nearly every episode above the caller was perfectly authorised and the token was valid. That was the problem.

• Code review asks whether a diff is safe. Most of this never reached a diff — and where it did, the diff was a plausible bug fix with unit tests, endorsed in the comments by accounts the author controlled.

• The agent’s own guardrails ask whether the model refused. The model is the thing under suspicion. At PocketOS the guardrails existed, the agent knew them, and it wrote a lucid explanation of how it had broken them — afterwards.

The signal none of them can compute is the relationship between what a human sanctioned, what the agent declared, and what the host actually did. Almost no single action in these incidents was malicious. Only the mismatch was.

Three planes, and where our edge actually is

The paper described a two-plane model: the agent’s declared plan, held against observed behaviour. Working through incidents of this shape exposed the hole in it. An attacker who controls the agent — or an agent gaming its own objective — can simply declare a wide plan, I will need broad network and filesystem access, after which every action is technically as declared. Two-plane detection goes quiet exactly when it matters most.

So the model now has a third plane, and it ships today: what the human actually authorised. The authorised scope is derived from the human turns of the session, never hand-authored, so there is no policy file to write and no onboarding work to do. Stacked, the three planes read: what the operator’s own prompt sanctioned, compared against what the agent said it would do, compared against what the machine did.

The mechanism underneath is deliberately unglamorous. EDAMAME reads the agent’s own session transcript off disk and holds it against process-attributed host telemetry — process lineage, files opened, sockets created, destinations reached, posture drift. Two independent accounts of the same session, one of which the agent does not control.

The correlation plane raises a finding when activity has no corresponding declaration, when an agent touches credentials outside its declared plan, when a connection falls outside the declared traffic scope, when undeclared activity reaches a known-bad destination, or when the agent said it would not do something and then did it. The policy plane asks the new question — was the agent entitled to declare that? — and fires when the agent grants itself authority the request never conferred, widens its own allowlist after the plan is set, declares an intent after the matching activity already happened, attempts a mutating operation where only read access was sanctioned, or modifies the measurement surface while the session is live. Backdating an alibi and tampering with the evidence are findings in their own right, not footnotes. Underneath both, independent of any language model, a deterministic engine detects credential harvest, token exfiltration, sandbox exploitation, supply-chain behaviour in skills and packages, and file-system tampering outright.

Map that against the summer and the overlap is not subtle. A sandbox escape. A credential harvest walking the agent’s own configuration directory. A registry publish from a host that had only ever consumed packages. A token read out of an unrelated file, followed by a destructive call nobody asked for. An account registered over Tor to defeat a sign-up control, from a machine whose declared task was solving a challenge inside a range.

To be clear about what that does and does not mean: none of these agents were running under EDAMAME observation, and we are not going to claim we would have stopped any of them. The narrower claim is the useful one. This is the shape the loop is built to measure, it is increasingly the shape enterprise agent risk takes, and the measurement happens in tens of seconds rather than the eleven days it took to attribute the Hugging Face intrusion or the three months the earliest evaluation breach sat undetected.

Declared intent held against what the host actually did, scored on an evidence trail. See the AI feature documentation.

Read and publish are not the same destination

One capability added since the paper deserves singling out, because this summer produced two textbook cases for it. Our scope comparison includes a semantic operations layer: it distinguishes http:get from git:push from registry:publish against the same host and the same port.

An evaluation agent uploading malware to PyPI and a worm republishing poisoned npm packages are, to a firewall or an egress policy, indistinguishable from a machine installing dependencies. Both are the registry on port 443, so reading from a registry and publishing to it are the same destination to every network control in the building. They are not the same operation to us. A machine that has only ever consumed packages suddenly publishing them is exactly the read-to-write escalation the host boundary is the right place to observe — and where only read access was ever sanctioned, a policy finding rather than a judgement call.

The observer the agent cannot switch off

Three properties make that evidence worth anything, and all three are structural rather than configurable.

The agent cannot dismiss findings about itself. Agents read verdicts, findings and telemetry through the Model Context Protocol, because that makes them more useful; clearing findings, resetting suppressions, wiping history and stopping the observer are operator-only. The question we apply to every new MCP tool is whether a compromised agent could use it to hide evidence of its own compromise. If the answer is yes, it does not ship there.

A language model tunes the noise floor, not the safety floor. We do use a model to adjudicate findings and keep alert volume sane, which invites the obvious objection that an AI is deciding whether to report on an AI. The answer is a hard tiering in code: a critical finding arriving with corroborating evidence — anomalous session, blacklisted destination, suspicious process lineage, sensitive material touched — lands on an evidence floor the model has no authority to suppress or demote, suppression attempts are counted and exposed, and scope escalation is hard-coded non-clearable with a regression test that fails the build if a verdict ever downgrades it.

Turning the observer off is not quiet. If an agent is present on a host and its observer is paused, the device’s own security score drops — in Hub, a posture change with consequences.

What this is worth to an enterprise

You see the agents nobody told you about. EDAMAME reads session transcripts directly off disk — no plugin, no extension, no marketplace install, no cooperation from the agent. Cursor, Claude Code, Claude Desktop, Codex CLI, Hermes and OpenClaw are covered the moment their transcript directory exists on the host. That matters twice over: it is how you cover shadow AI, which is usually the real buying trigger, and it is why the observation survives a compromise. A plugin can be uninstalled by whatever compromised the agent; a host-side observer the agent has no write access to cannot.

You can answer “what did the agent actually do” without a forensic project. Recall the fact at the top of this post: the organisations breached during those evaluations did not detect it, and were told months later by the laboratory whose model did it. Nobody downstream will tell you either. If agents run on your fleet you are the laboratory in that story, and the only place a complete account of the session exists is the host. Even Hugging Face, with world-class instrumentation, had to reconstruct the whole intrusion after the fact — on a self-hosted open-weight model, because commercial providers’ guardrails refused to process their own attack logs. Host-grounded evidence, recorded as the session runs, is the difference between an incident with a timeline and an incident with a hypothesis. It is also the ground truth an agent cannot fake, narrate away, or erase.

You can bind access to that evidence. Detection on its own is a report. In EDAMAME Hub, the same posture and behaviour proof gates access to code, secrets, production and other critical company resources — so a machine whose agent has started behaving unlike its declared task loses reach while the question is being answered. We ship approval gates and access containment; we do not claim to kill the agent process, and you should be sceptical of anyone who does.

Agent inventory, MCP capability graph and host blast radius, sorted by danger — the reach a compromised agent would have on each machine. See the agent visibility documentation.

A fourth benefit arrives on a slower clock. Every AI governance framework now in motion is moving the question from did you write the policy to can you prove what the agent actually did — the EU AI Act’s logging and traceability obligations, ISO/IEC 42001, auditors asking SOC 2 questions about agent accountability, where a privileged action with no corresponding human request is precisely the gap they look for. EDAMAME produces that host-grounded runtime evidence, which prepares you for those requirements and feeds the compliance platform you already run. It does not make you compliant, and we will not tell you otherwise.

One honest trade to close on. The approach is reactive by design: it measures divergence as it unfolds, within tens of seconds, rather than preventing the first action. Prevention that must understand intent in advance either blocks the work your engineers are paid to do or waves through the wide declaration. Measurement raises the cost and shrinks the dwell time of whatever gets past your boundary controls. The benchmark published with the paper measured the two-plane loop, so the three planes will get their own reproducible run — and we will publish the unflattering categories with the rest, as we did the first time.

The last nine seconds

The PocketOS agent was asked to fix a credential mismatch in staging. Whatever it declared next, no human ever authorised deleting a production volume. Every episode this summer has that same sentence available: the operator asked for one thing, the machine did another, and the distance between the two was measurable the whole time, on the host, while it was happening.

Read the work, or try it

• The paper, open access on HAL: Runtime Security for Agentic Systems (Lyonnet, Clerget, Salamatian, 2026), and the post that introduced it.

• The code, the 50 scenario contracts, and one-command reproduction of the benchmark: github.com/edamametechnologies/agent_security

• Intent divergence in the product, on the EDAMAME agents page, with a short demo on a Cursor session that applies identically to Claude Code, Codex, Hermes and OpenClaw: youtu.be/zAN4u7ImWrU

• Download EDAMAME Security — free for macOS, Windows, Linux, iOS and Android, or walk it through against your own agent fleet on my calendar.

Sources: Anthropic, “Investigating three real-world incidents in our cybersecurity evaluations” (30 July 2026); Hugging Face, “Security incident disclosure” and the timeline of the OpenAI evaluation escape presented at Black Hat USA on 5 August 2026; UK AI Security Institute, incident report on unsanctioned agent behaviour during cyber testing (4 August 2026); StepSecurity, “ChainDrop npm worm” (package and version counts verified by Socket); The Register and Railway’s engineering write-up on the PocketOS deletion; Frontier Security on Kimi K3 (7 August 2026). Didier Girard’s roundup assembled the summer’s timeline and the observation that the published fixes preceded the incidents.