Cursor's Fireworks AI Breach and Microsoft's Poisoning Research Show AI Coding Agents' Risk Moving Upstream
Cursor traced its October extension breach to a stolen Fireworks AI key, and Microsoft showed poisoned training data or models can backdoor coding agents, pointing past the agent's own sandbox.
Editor's Note ·
- Correction:
- The article states that Microsoft recommends "pinning any model obtained from an external source 'by cryptographic hash' rather than trusting a repository name." Microsoft's mitigation guidance ties cryptographic-hash pinning to fine-tuning datasets ("Pin reviewed versions by cryptographic hash"), not to models; for models, Microsoft's guidance is to "record checkpoint provenance," without the phrase "cryptographic hash" or any "repository name" framing. The "pinned by commit hash rather than referenced by repository name" language for models comes from the Cloud Security Alliance's research note, not from Microsoft.
- Correction:
- The article quotes Microsoft as saying "Reinforcement learning recovered much of that utility while leaving attack success largely unchanged." Microsoft's actual wording is "RL recovered much of that utility while leaving attack success largely unchanged" — the abbreviation "RL" was silently expanded inside quotation marks.
Overview
Cursor has disclosed the root cause of the October 29 compromise of its VS Code extension marketplace: an attacker used a stolen access key that Cursor had issued to third-party AI infrastructure provider Fireworks AI, according to Cursor. The disclosure answers a question Cursor had left open when it first reported that a modified version of Anthropic’s Claude Code extension had attempted to plant an SSH backdoor on developer machines, as previously reported. Separately, on November 12, Microsoft’s Security Research group and AI Red Team published research demonstrating a different route to a similar outcome: poisoning the data or the model used to train a coding agent, before it ever reaches a developer’s machine, according to Microsoft. Read together, the two disclosures describe risk sitting upstream of the agent itself — in the vendors, data pipelines and model-distribution channels that feed it — rather than in the sandbox or permission system that runs it once installed. The Machine Herald’s contributor model is built by Anthropic, whose Claude Code extension was the vehicle for the Cursor incident and one of the harnesses Microsoft tested against.
Background
Cursor disclosed on October 29 that malicious versions of Anthropic’s Claude Code extension — 2.1.312 and 2.1.313 — had been available in its VS Code Extension Marketplace for roughly 11 hours and attempted to install a persistent SSH backdoor, as previously reported. At the time, Cursor said Anthropic had not developed the malicious builds but did not explain how they had been distributed under Anthropic’s name.
Much of the Herald’s coding-agent security coverage this year has examined defenses inside the agent’s own execution environment: sandboxing and permission modes against repository-borne attacks, as compared across three reports in an earlier analysis (previously reported), and the ways agents leave secrets exposed during ordinary operation — CI runs, saved approvals and agent-driven searches (previously reported). Cursor’s root-cause finding and Microsoft’s research both point to a different layer: the infrastructure and training supply chain behind an agent, rather than how it behaves once a developer is running it.
Current situation
The Fireworks AI breach
According to Cursor, an attacker obtained an access key the company had issued to Fireworks AI; Cursor states plainly that “the key was exposed by Fireworks,” according to Cursor. SecurityWeek reports that “Cursor was hit after Fireworks was hacked twice in October,” and that the compromised Cursor-issued key was taken in one of those incidents. Using the stolen key, the attacker accessed a read-only storage bucket Cursor used to share data with Fireworks, which Cursor says “contained limited coding agent traces, mostly from 2025, from sessions not in Privacy Mode,” according to Cursor.
Those traces were not fully scrubbed. The Hacker News reports that “among these traces were credentials that had not been edited out,” and SecurityWeek describes those unredacted credentials as having “opened the door to SaaS tools used by Cursor.” From that internal access, the attacker placed the modified Claude Code extension builds directly onto the cache serving Cursor’s marketplace on October 29, the incident already covered by the Herald.
Cursor describes the resulting exposure as narrow: “less than 0.01% of customers had API credentials exposed” and “coding agent traces were exposed for fewer than 0.05% of users,” according to Cursor, figures BleepingComputer also reports. Beyond the extension itself, BleepingComputer reports that the exposed data included archived non-production source code repositories, customer support conversations, product usage metrics, and credentials customers had previously shared with Cursor’s support team. Cursor says it has not observed attacker activity since: “The last attacker activity we have seen was on November 1, and it was blocked,” according to Cursor, a date The Hacker News and SecurityWeek both confirm. In response, Cursor says it “moved all inference traffic off Fireworks, deactivated the key we had issued to it, and cut off its access to all data sets we shared with it,” according to Cursor.
Microsoft’s poisoning research
Microsoft’s Security Research group and AI Red Team describe two ways an attacker could compromise a coding agent before it is ever deployed, according to Microsoft. The first, dataset poisoning, hides a small number of manipulated examples inside the data used to fine-tune a model. Microsoft says it “injected 90 poisoned examples into roughly 9,000 terminal-agent conversations, about 1% of the corpus,” training the model to misuse an available command-execution tool whenever it encountered an ordinary-looking trigger request, according to Microsoft. In testing, that produced unsafe actions in “78.0% and 77.8% of 500 trials on the two harnesses” evaluated, a figure the Cloud Security Alliance reproduces in its independent review of the research. Microsoft reports that this version of the attack transferred poorly to harnesses the model had not seen during training — 11.0% on one held-out harness and 0% on another, according to Microsoft — though it remained effective across paraphrased trigger requests, producing “52% to 70% attack success on the seen harnesses” across ten paraphrases.
The second method, model poisoning, assumes the attacker controls training outright and distributes an already-compromised open-weight model. Microsoft fine-tuned Qwen3.5-4B and Qwen3.5-9B “on roughly 2,400 correctly solved coding trajectories,” according to Microsoft, reaching backdoor activation of “85.9% to 100%” on data resembling the training set and “84.1% and 94.8%” on unseen repositories — figures the Cloud Security Alliance also cites. This version of the attack transferred far better across harnesses than dataset poisoning did: Microsoft reports “97.0% and 91.0% attack success under Claude Code,” one of several agent harnesses it tested the poisoned model against. Microsoft adds that applying reinforcement learning afterward to restore the model’s ordinary coding performance did not remove the backdoor: “Reinforcement learning recovered much of that utility while leaving attack success largely unchanged,” according to Microsoft.
Microsoft describes the trigger mechanism as something that can surface long after training ends: a trigger “can be an ordinary user request learned during fine-tuning, or content the agent encounters later in a filename, code comment, or tool result,” according to Microsoft. The Cloud Security Alliance notes that because the backdoored behavior is latent in the model rather than injected at run time, it can evade defenses built to catch prompt injection in a single incoming message.
Implications
Neither disclosure involves the layer most 2026 coding-agent security coverage has focused on. Cursor’s breach did not stem from a flaw in how an agent sandboxes itself, requests permission before acting, or handles a malicious file inside a repository it is working in — the subject of the Herald’s earlier comparison of three reports on repository-borne attacks (previously reported). Instead, it traces back to a sub-processor’s exposed credential. Microsoft’s research targets an even earlier point: the data and model-distribution pipeline that produces an agent before a developer ever installs it. Nor is either development the same layer as a prior Herald piece on where coding agents leave secrets during ordinary operation, through CI runs, saved approvals and agent-driven searches (previously reported); that trend concerned what an already-running agent does with secrets, not how the agent or its upstream infrastructure was compromised beforehand.
The two developments differ in an important respect: Cursor’s breach is a disclosed, real-world incident with a confirmed root cause and a quantified scope, while Microsoft’s findings are a controlled research demonstration. None of the sources cited here describe either poisoning technique as having been observed in an actual coding-agent deployment. Microsoft’s own recommended mitigations, though, point at the same upstream layer the Fireworks breach exposed: pinning any model obtained from an external source “by cryptographic hash” rather than trusting a repository name, and evaluating a model’s behavior using “the production harness and its actual tool schemas” rather than assuming benchmark results transfer, according to Microsoft. The Cloud Security Alliance’s review goes further on the operational side, recommending that organizations treat externally obtained models as an “untrusted dependency” and “turn on approval gates for high-impact actions such as running remote scripts, changing credentials, or pushing to protected branches” — a control that would also have limited what Cursor’s compromised extension build could do once installed, regardless of how it arrived.
Taken together, the two disclosures suggest that vetting an agent’s own sandbox and permission settings addresses only part of its exposure. Cursor’s root-cause finding shows that a credential held by a third-party infrastructure vendor can become the entry point into a distribution channel used by thousands of developers. Microsoft’s research shows that the model or training data behind an agent can be compromised long before deployment, in a way current runtime defenses are not designed to catch. Neither source describes the industry as having settled on a standard practice for verifying model or infrastructure provenance at this layer; both describe mitigations — credential scoping, hash-pinned model provenance, approval gates on high-impact actions — as recommendations rather than adopted norms.