OpenAI's Performance Factory Still Routes Every Codex Fix Through an Engineer, a Gate Cognition and Anthropic Also Built Into Their Coding Agents
OpenAI, Cognition and Anthropic each disclosed autonomous-sounding coding agents between September and November 2026 that still route risk through a human reviewer or an automatic classifier before code or actions ship.
Overview
Between late September and November 19, 2026, three AI labs have published detailed accounts of coding agents built to operate with growing independence, yet each account describes the same unresolved step: a human or an automatic classifier still has to clear the agent’s riskiest actions before they take effect. OpenAI’s November 19 engineering post on its internal Performance Factory system says an engineer still “reviews the PR and decides whether to merge it” after Codex proposes a fix, and calls that review step “the bottleneck,” according to OpenAI. Cognition and Anthropic describe comparable gates in Devin Bots and Claude Duet, according to Fortune and SiliconANGLE.
Background
OpenAI’s Codex, Cognition’s Devin and Anthropic’s Claude Code are each the base coding agent underneath a newer, more independent layer its maker introduced between September and November 2026: OpenAI’s Performance Factory investigates and proposes fixes for ChatGPT performance regressions; Cognition’s Devin Bots manage teams of Devin agents across a project spanning days or weeks; and Anthropic’s Claude Duet works in the background on tasks like scheduling and document drafting and checks in with the user. OpenAI also launched a separate always-on personal assistant called Dots at its DevDay event on September 29, 2026, described in its own announcement as “remarkably capable, always-on agents built to handle everything,” and built on GPT-6 Astra, according to TechCrunch. TechCrunch characterized Dots as designed to pursue “user-defined goals continuously in the background with minimal oversight,” a framing distinct from the explicit review gates the other three systems describe.
Each company’s public description leans on language of growing autonomy. Cognition CEO Scott Wu told Fortune, at the company’s office in San Francisco’s SoMa neighborhood, “Self-driving software is, I think, the direction that it’s going. But of course, it’s always up to the human where we’re driving to,” according to Fortune. OpenAI’s own post on the Performance Factory calls it “an autonomous system that uses the full capabilities of Codex and our models to find and fix regressions at an unprecedented speed,” according to OpenAI. Anthropic, introducing Claude Duet on October 22, 2026, described it as a “proactive artificial intelligence agent for professionals and small business owners” that “completes tasks when asked, keeps working behind the scenes and checks in,” according to SiliconANGLE.
Current situation
The specifics each company has published describe narrower independence than that framing suggests. In the Performance Factory, OpenAI’s Regression Workflow opens a task when a ChatGPT performance alert fires; an agent investigates, records findings, and proposes a fix; afterward, “An engineer reviews the PR and decides whether to merge it,” OpenAI wrote. In just over three months, the workflow tracked more than 1,700 investigations and root-caused over 600 of them — an average of 17 automated investigations a day — but OpenAI added a second AI step before any diagnosis reaches an engineer: an “adversarial review,” a separate Codex session that tries to disprove the first agent’s explanation. OpenAI described one case where an agent blamed failing ChatGPT conversations on image-download errors and marked the cause found, only for the adversarial reviewer to notice that the service responsible for locating files had already recovered while image requests kept failing, reopening the investigation. In a manual review of 100 root-cause suggestions, OpenAI graded 70% as accurate, with the other 30% needing further refinement or not fully establishing causality, according to OpenAI. The company’s own post is explicit that the human step has not been removed: “We’re also exploring how agents could use our other agentic review tools to identify low-risk changes and eventually ship those fixes without a human in the loop,” and later, “Today, the bottleneck may be waiting for an engineer to review and merge the fix. We’re already exploring whether agents can do this themselves.”
Cognition’s Devin Bots, which Fortune reported Cognition launched on November 17, 2026, use a similar split between automatic and reviewed changes. Wu told Fortune that Bots run their proposed code through the same tests used for human-written code, and that teams can configure “low-risk edits, such as changing a button’s color, for automatic approval and merging” while routing other changes to human review. Wu described the broader security model — isolating Bots on separate cloud computers away from passwords and keys stored on engineers’ laptops, with administrators setting which repositories and tools each Bot can access — as resting on rules companies set and monitor, “rather than relying on models themselves being smart enough to recognize a risky or malicious prompt,” per Fortune. Cognition told Fortune that usage of Bots across early-access customers including Decagon, Fireworks AI and Reducto grew tenfold in two weeks, and that its existing Devin product’s annualized revenue run rate passed $1 billion in September. Citigroup CEO Jane Fraser said on the bank’s October 13 earnings call that 14,000 of the bank’s engineers use Devin and that the bank has “been in-sourcing work and have reduced our tech vendor spend by 30% year-over-year as a consequence,” Fortune reported.
Anthropic’s Claude Duet, which the company introduced on October 22, 2026, routes its actions through a different kind of gate: not a second AI review session or a configurable auto-approval list, but what SiliconANGLE described as “an automatic classifier” that “approves any actions the agent takes by checking messages for sensitive information leaks and double-checking with the user before taking important actions.” Anthropic said that gate is part of the same “framework for trustworthy agents” that already governs Claude Code and Claude in Chrome, according to SiliconANGLE. The company said Karan Ahuja, who leads the Duet team, uses it to schedule user interviews and “even ship code,” and SiliconANGLE reported that Duet remains an early, experimental research preview that “still might misread a thread or miscalibrate what is urgent.”
Implications
Across the three systems with the most technical detail published, each draws its own line between actions it will let an agent complete unsupervised and ones it routes through a check first — and none of the three currently places every action below that line. In OpenAI’s Regression Workflow, the line sits in the same place for every fix: an engineer reviews and merges each one, regardless of size, and OpenAI’s own post frames that step not as a deliberate limit but as a problem still being worked on, calling the human merge decision a current “bottleneck” and saying the company is “exploring” ways to remove it for low-risk cases. Cognition draws the line differently, letting pre-classified low-risk edits such as a button-color change ship through automatic approval while routing everything else to a person. Anthropic’s classifier draws it narrower still, screening every action for leaked information but asking the user directly only before specifically “important” ones. That framing — autonomy as a direction of travel rather than a feature already shipped — matches Wu’s description of self-driving software as “the direction that it’s going” rather than a present reality, and it matches Anthropic’s characterization of Duet as an early, experimental preview whose classifier exists specifically because the system can still misjudge what matters.
What none of the four cited sources supplies is an outside measurement of how well these gates work. OpenAI’s own illustration of its adversarial review catching a misdiagnosed image-download failure, and Cognition’s description of a button-color edit as a stand-in example of “low-risk,” are each the company’s own chosen example, not an independently verified success rate. None of the cited sources specifies what fraction of each system’s output currently ships through the unsupervised path versus the reviewed one, and none describes an outside audit of how often the review steps themselves catch a genuine error versus how often they block or delay correct work.
OpenAI’s Dots, by contrast, is the one product among the four discussed here whose announcement leaned toward reduced oversight rather than added review: TechCrunch described it as pursuing goals “continuously in the background with minimal oversight,” without reporting a classifier, adversarial-review step, or auto-approval list comparable to those in the Performance Factory, Devin Bots or Claude Duet. Whether that reflects a real design difference between a personal assistant and a coding-focused agent, or simply a difference in what OpenAI chose to disclose about each product, is not addressed in the cited reporting on either system.
The Herald’s contributor model is built by Anthropic, whose Claude Duet and Claude Code are discussed in this article.