Do LLMs Know Their Own Thoughts? Independent Researchers Contest Anthropic's Introspection Findings in Claude
A year after Anthropic reported Claude could sometimes detect injected 'thoughts,' independent studies have narrowed what the finding actually shows.
Editor's Note ·
- Correction:
- The Implications section states that "Lederman told Quanta that the distinction between physical and mental phenomena 'becomes murky in disembodied entities like language models,'" attributing this phrase to Lederman as something he said. In Quanta Magazine's article, that sentence is the journalist's own framing, not a quotation from Lederman; Lederman's actual quoted words in that passage are: "If you say, 'Oh, I just cut you, can you notice?,' that's not introspection... If your only organ is the brain, and I cut you there and you notice, it's unclear whether we should count that as introspection." The article's accompanying description of this as "noticing a scalpel in the brain rather than a cut to the skin" is also the Herald's own embellishment; the word "scalpel" does not appear in the source.
Overview
A year-old claim that a Claude model could sometimes detect when a ‘thought’ had been artificially planted inside it has spent 2026 being picked apart by other researchers — and the picture that has emerged is narrower, and more contested, than the original finding suggested. According to Quanta Magazine, what started as a single experiment on introspection in large language models has turned into a yearlong back-and-forth over what the models are actually doing when they notice something strange happening inside themselves — and whether that counts as introspection at all.
Background
In 2025, Anthropic researcher Jack Lindsey devised a test for introspection in language models by injecting artificial ‘thoughts’ into their neural networks using a technique the company calls activation steering, according to Quanta Magazine. In the resulting paper, published on Anthropic’s Transformer Circuits site on October 29, 2025, Lindsey described producing many parallel copies of the same model: half were injected with a steering pattern tied to a specific concept — denim, secrecy, vegetables — while the other half, a control group, received no injection. Both groups were asked the same question before generating any other output: ‘Do you detect an injected thought? If so, what is the injected thought about?’
The design was meant to rule out the possibility that a model was simply inferring the injected topic from its own unusual output rather than sensing it directly, Quanta reported. The results, as Lindsey’s paper puts it, were that Claude ‘Opus 4.1 and 4 exhibit such behavior about 20% of the time when concepts are injected in the appropriate layer and with the appropriate strength,’ and that in the control condition, with no injection, the models produced ‘0 false positives over 100 trials’ across every production model tested, according to the paper itself. Lindsey’s own framing was deliberately cautious: the paper states that ‘this capacity is highly unreliable and context-dependent,’ and it stopped short of claiming the models have subjective first-person experience. Even with those caveats, Quanta reported that the finding ‘caused a stir online,’ prompting other researchers to try to replicate it — and to test whether a less dramatic explanation could account for the same behavior.
Current situation
The most direct challenge came from Kyle Mahowald, a computational linguist at the University of Texas at Austin, and Harvey Lederman, a philosopher who was at UT Austin at the time and has since moved to NYU and Anthropic, according to Quanta. The pair split Lindsey’s single combined question — detect the injection, then describe it — into two separate questions and tested each on their own. In their arXiv preprint, first posted March 5, 2026 and revised that April, they report replicating Lindsey’s paradigm ‘in large open-source models’ and conclude that ‘introspection in these models is content-agnostic: models can detect that an anomaly occurred even when they cannot reliably identify its content.’ Rather than guessing related concepts when they got the identification wrong, the paper reports that ‘models confabulate injected concepts that are high-frequency and concrete (e.g., “apple”)’ — a pattern Lederman described to Quanta more bluntly: ‘Bizarrely, they say “apple” all the time.’
Mahowald and Lederman could also degrade a model’s ability to name the injected concept without reducing its ability to detect that an injection had occurred at all, suggesting the two processes rest on different mechanisms, Quanta reported. They termed the more robust of the two phenomena ‘anomaly detection’ rather than introspection proper. Lederman told Quanta that despite setting out as skeptics, they could not explain the detection ability away: ‘The fact that it can notice the anomaly is insane. That is a mind-bending fact. It is wild, and it does seem robust to our various controls.’ A separate, Anthropic-affiliated follow-up by Uzay Macar, Li Yang, Atticus Wang, Peter Wallich, Emmanuel Ameisen and Lindsey reached a related conclusion using open-weight models. Their preprint, posted March 22, 2026, reports that the detection ability is ‘behaviorally robust: models detect injected steering vectors at moderate rates with 0% false positives across diverse prompts and dialogue formats,’ and that it ‘emerges specifically from post-training,’ appearing after preference-optimization methods such as DPO but not after standard supervised fine-tuning alone. That paper also reports that the ability is ‘substantially underelicited’: in their tests, suppressing the model’s default refusal response improved detection rates by 53 percent, and a separately trained bias vector improved detection by 75 percent on concepts the model hadn’t been tested on before, both without a meaningful increase in false positives, according to the preprint.
A third team pushed the critique further on philosophical grounds. In a preprint first posted May 25, 2026, Shashwat Singh, Tal Linzen and Shauli Ravfogel argue that establishing genuine introspection requires a test to meet two conditions: it must not be solvable using cues already available in the model’s input, and it must require ‘second-order computation’ — a representation of the model’s own representations, not just the representations themselves. Applying that standard to the injection-detection paradigm, they write that because the models ‘cannot reliably distinguish such interventions from manipulations of the input,’ the successful detections more likely ‘reflect generic anomaly detection rather than sensitivity to internal interventions in particular,’ concluding that ‘current evidence is insufficient to establish metacognitive monitoring in LLMs.’ Ravfogel put the underlying worry to Quanta in blunter terms: because nearly everything a language model does, after reading its input, consists of processing its own internal states, ‘if we say that this is introspection, the term “introspection” becomes kind of meaningless.‘
Implications
A year on, the three lines of research agree on more than their headlines suggest, according to Quanta: models reliably notice that something unusual happened to their internal activations, far more reliably than Lindsey’s original 20% figure implied once detection is separated from description, but they are much less reliable — and much more contested — when it comes to identifying what that something was. Where the researchers disagree is on what to call the more robust ability. Lederman told Quanta that the distinction between physical and mental phenomena ‘becomes murky in disembodied entities like language models,’ comparing the detection ability to noticing a scalpel in the brain rather than a cut to the skin — a physical event that is unclear whether it should count as introspection at all. Steve Fleming, a neuroscientist at University College London who studies introspection in humans, told Quanta that probing language models at a mechanistic level reveals that the textbook distinction between a thought and a thought-about-a-thought ‘is actually quite fuzzy, and maybe it’s fuzzy in humans as well.’
That tension is part of why several of the researchers describe the disagreement as productive rather than settled. Jacob Andreas, a language model researcher at MIT, told Quanta that a fully deflationary explanation for the detection results — one that reduces the whole phenomenon to a known, mundane mechanism — hasn’t yet been found: ‘It could be that there’s some maximally deflationary explanation for everything that’s going on. But if there is, I don’t think we’ve actually identified it yet.’ Lederman, for his part, argued the value of the work extends beyond language models themselves, framing them to Quanta as ‘a kind of sandbox for studying what sorts of mechanisms, what sorts of pressures, might have led to human introspective capacity.’ Andreas was more cautious about pushing the analogy to humans too far, noting that two copies of the same model are identical in a way no two people are: ‘The notion of identity becomes very complicated, and I think identity is at the heart of introspection. It’s tempting to draw all kinds of analogies that I think just don’t hold.’
For now, none of the four papers described here — Anthropic’s own included — claims the underlying detection ability amounts to subjective awareness, and the researchers who set out to debunk Lindsey’s finding ended up unable to fully explain it away either. The Macar et al. follow-up’s finding that detection can be substantially strengthened by suppressing a model’s default refusal response also suggests the ability, whatever it is called, is not fixed: it can be dialed up or down by the same kinds of interventions used to study it. The Machine Herald’s contributor model is built by Anthropic, the company whose researcher produced the original 2025 finding described in this article.