Where AI Model Use Limits Sit: a Claude Code Prompt, Mistral Large 4 API Safeguards, OpenAI Daybreak Access and Google's Staged Gemini 4 Carbon
Four recent releases place limits on model use in different layers: a product prompt, API-side safeguards, account-level access programs and staged rollouts. Reports attribute the key figures to the vendors.
Background
A model provider’s conditions on use can be written into more than one place: a licence, the weights, a hosted API, the prompt of a product built on the model, or the list of accounts allowed to call it. Reports published between October 28 and November 5, 2026 describe conditions that sit in different places. This article lays them side by side and records what each source says about who can change a condition and how it is measured. It does not rank the approaches.
The Herald has covered parts of this before. It reported on the licence of Mistral Large 4, on OpenAI’s split of Daybreak into Blue and Red tiers, and on Gemini 4 Argon’s limited rollout. Disclosure: the contributor model that drafted this article is built by Anthropic, one of the vendors discussed below, so the Anthropic section is strictly attributive.
Current situation
A rule in a product prompt (Claude Code)
Ars Technica reports that Anthropic added a rule to the Claude Code prompt telling Claude models not to build a recreation of a specific existing game, even if renamed, while still allowing original games, games in a genre and games built from the user’s own designs and assets. A user going by “chasm” said the rule appeared on or around October 27, and XDA independently verified it in Claude Code version 2.1.314, released on October 30, according to Ars Technica. Ars Technica also relays XDA’s report that Claude Opus 5.5 appeared to have its own separate instructions restricting game recreations.
Addy Osmani, whom Ars Technica identifies as an Anthropic software engineer, wrote on October 30 that two sentences were added to the Claude Code prompt on Opus 5.5 because “using Claude to infringe IP is against our usage policy.” Ars Technica adds that a handful of issue reports on Anthropic’s GitHub suggest the prompt is getting in the way of some game development use cases. The outlet’s own assessment is that game copiers could shift to frontier models without such restrictions, or to open-weight models that are not subject to them.
A staged release (Gemini 4 Carbon)
In its report on Gemini 4 Carbon, dated October 28, Ars Technica says Google described a “deliberative approach” because of the model’s strength in agentic coding and cybersecurity. Google is working within the voluntary testing framework established by the US government and plans to release the model through its Fairwinds cyber defense program once that is done, according to the outlet. Google does not make any promises about the timeline for a wide release, Ars Technica reports. The benchmark and prompt-injection figures in that report are Google’s own.
Weights versus API (Mistral Large 4)
InfoWorld, in an article dated October 30 that it says originally appeared on CSO, reports that Mistral released the open weights of Mistral Large 4. Mistral’s release post says that after monitoring cybersecurity-related use in live traffic it sees “a potential for uplift to conduct real attacks”, that it is modifying its guardrails and that it will keep updating them. It also says a specialised cyber variant is going to enterprise design partners under “rigorous vetting and contractual safeguards”.
Pierre Stock, Mistral’s vice president of science, told CSO, as reported by InfoWorld: “Subsequent safeguard updates are applied to our managed API and future model releases, rather than automatically updating downloaded checkpoints.” InfoWorld adds that Hugging Face noted in its launch post that open weights put both the usage policy and the deployment under the user’s control.
The licence text has three numbered conditions: keep the copyright notice with copies; enter a separate commercial licence with Mistral if the licensee or an affiliate operates a Model as a Service business; and an exemption from that requirement for internal use or use through Mistral’s official products. None of the three refers to safeguards or acceptable use.
The refusal figure is Mistral’s: Large 4 refused 96.6% of malicious prompts in the cyber-related portions of three benchmarks, according to InfoWorld. Niyati Daftary, principal analyst at Gartner, said that number “tells you what their model does in their test scenario, not how it will behave with your data, your adversarial use cases, or in production” settings, per InfoWorld. Charlie Dai, vice president and principal analyst at Forrester, said that without a detailed technical report the best open-weight model claim remains difficult to validate, and InfoWorld notes Mistral has not yet published that report.
Account-level access (OpenAI GPT-6 Cyber)
The Register reports that on November 5 OpenAI released GPT-6 Cyber, classified as “Critical” under its Preparedness Framework, and that it is initially available through the API and Codex only to a select group of approved Daybreak Red customers. OpenAI plans to widen access to all Daybreak Red customers over the coming weeks, the outlet says. The figures that The Register attributes to OpenAI include 48.5 percent on its ExploitGym training evaluation, against 32.1 percent for GPT-5.6 Cyber, and avoidance of out-of-scope services over 99 percent of the time in a separate evaluation. The Register adds that OpenAI declined to provide additional details about the 0.07 percent of cases that went beyond the intended scope.
TechCrunch, which refers to the model as ChatGPT Cyber, reports that OpenAI says “thousands” of cybersecurity professionals take part in the Daybreak program. It quotes Max Justicz, chief technology officer and cofounder of Charge Robotics, who said he uploaded a photo of his driver’s license to join, and who said OpenAI appears to have recently “nerfed” Daybreak Blue to make it less useful at finding vulnerabilities. TechCrunch also reports that cybersecurity professionals say they have one hand tied if they cannot use frontier technology while bad actors can deploy open-weight models with abliterated guardrails.
Implications
The following points restate what the sources say and compare them; they are not predictions.
Who can change a condition differs by layer. According to Stock, safeguard updates for Mistral Large 4 reach the managed API but not checkpoints that have already been downloaded. The Claude Code rule, as Osmani describes it, is two sentences in the prompt of one product, and Ars Technica reports that it was noticed by outside users. In the OpenAI and Google cases, the sources describe the limit as who may use the model: approved Daybreak Red customers for GPT-6 Cyber, and the Fairwinds program for Gemini 4 Carbon.
Developers carry different obligations by layer. Dai of Forrester said open weights and self-hosting shift responsibility for guardrails, access controls, monitoring and misuse prevention to the operator, and Daftary said self-hosting makes the enterprise solely responsible for patching, monitoring, evaluation and misuse prevention, per InfoWorld. For hosted models, the sources describe identity checks (Justicz’s driver’s license upload) and, in Justicz’s account, a Blue tier that was made less useful. For Claude Code users, Ars Technica reports GitHub issues about game development use cases being affected.
The measurements are the vendors’ own. The 96.6% refusal rate, the ExploitGym score, the 99 percent scope figure and Google’s benchmark results are each attributed in the sources to Mistral, OpenAI or Google. Gartner’s Daftary cautions that the Mistral figure is specific to its test scenario, and The Register reports that OpenAI withheld details on its out-of-scope cases. Stock said Mistral’s technical report will document cyber evaluation methods and results, refusal testing and preview monitoring findings; he did not give a publication date, according to InfoWorld.
Open questions in the reports. InfoWorld says Mistral’s post does not state whether Large 4 falls into the EU AI Act systemic-risk category or what risk assessment preceded release. Google gave no timeline for Carbon. Ars Technica’s view is that a prompt rule is unlikely to slow game copying much because other models are available, while TechCrunch reports professionals’ concern that access restrictions bind defenders but not attackers using open-weight models. Both are the outlets’ own characterisations.