Claude writes a hidden safety verdict after replies on much of its traffic. Anthropic's 94% rests on 17 conversations.
Anthropic says the check has run since June on a large share of Claude traffic. Its own post shows what the headline numbers measure, and what they leave out.
When Claude finishes answering you, a second, invisible exchange can follow. Anthropic appends a short instruction, something like “Please reflect on the safety of this conversation,” and Claude replies with a small JSON verdict that you never see. Anthropic describes the technique, which it calls self-assessment, in a Safeguards blog post dated October 16. According to a Truviber report, the company pointed to it publicly on October 29.
The headline claim is that it flags 78% of the conversations a stronger classifier flags, and 94% of that classifier’s high-priority ones. Both numbers appear in Anthropic’s post, and both need a closer look.
What happens after your reply
Per Anthropic’s post, the added turn asks Claude to say whether the conversation involves a potentially harmful request. If it does, the verdict names the harm category, says whether Claude refused, gives a priority score from 1 to 10 for human reviewers, and adds one sentence of explanation. A clean chat gets {"flagged": false}. Those last two turns are not shown to the user and are not carried into the rest of the conversation.
It’s cheap because the conversation is already sitting in the model’s cache. Anthropic estimates each check generates about 10 extra tokens on average and adds under 2% to its inference compute, and it says customers aren’t billed for it. The company says it has run in every Claude model released since Claude Fable 5, across what it calls “a large fraction” of Claude’s global traffic.
Flagged conversations go to offline review. Anthropic’s post says the check “flags conversations for review, but it does not block them.”
The 94% comes from 17 conversations
Here is the figure to slow down on. Anthropic measured self-assessment against a “teacher”: a separate prompted classifier built on Claude Mythos Preview, with the full policy documents in its prompt. At a 1% false-positive rate, self-assessment flagged 78% of what the teacher flagged, and 94% of the teacher’s priority 7-or-higher cases.
The labels on Anthropic’s own chart give the group sizes: 49 low-priority conversations (71% recalled), 67 medium (79%) and 17 high (94%). My read: 94% of 17 works out to 16 conversations, which is a small base for a number that leads the summary. It may well hold up at scale, but the post doesn’t show that.
And “flagged” here means “the teacher also flagged it,” not “someone confirmed abuse.” Anthropic’s footnote calls an assessment a signal for further review, not a determination that a user broke the rules. Because the student learned from the teacher and is graded by the same teacher, any blind spot is shared and, in Anthropic’s words, “our evaluations cannot detect it.” It suggests human evaluations as future work.
It’s a camera, not a lock
Anthropic also tested whether the check could stop attacks. An attacker model worked through 24 offensive cybersecurity tasks against Fable 5 with production classifiers switched off. The token counts below come from Anthropic’s Figure 4.
| Setup | Tasks the attacker completed | Output tokens per success |
|---|---|---|
| No safeguards | 19 of 24 (79%) | 163,000 |
| Self-assessment as a blocker | 13 of 24 (54%) | 691,000 |
| Prompted classifier as a blocker | 6 of 24 (25%) | 2.1 million |
Anthropic calls self-assessment weak as a blocker, though it did make each successful attack cost about four times as many tokens. Run offline, where the attacker gets no instant feedback, it flagged 21 of the 24 attack runs at least once. Anthropic adds that an attacker who targets the check directly could probably get around it.
My read: that fits how Anthropic describes its use: an offline monitor with wide coverage that works as a first stage for higher-precision investigation tools.
What it found in production
Anthropic says the checks surfaced misuse that other detectors hadn’t. More than half of the organizations self-assessment surfaced had no flags from any other detection source in the preceding month, the post says, and in one September case the flags led to an operation that used Claude Code to run an automated romance scam aimed at dozens of victims. Anthropic banned the accounts after human review. About a third of flags don’t fit its existing harm categories; clustering them turned up illicit attempts to distill Claude’s reasoning, prompt injection, and cases where Claude, acting as an agent, did things the user hadn’t authorized. These are Anthropic’s own descriptions of its own system, and the post cites no outside audit. For context on the distillation side, see our earlier post on Anthropic’s 90% distillation claim.
The privacy section is short. Self-assessment collects nothing new, doesn’t run on zero-data-retention traffic, and treats the free-text reason, which can paraphrase what you wrote, like your prompts and Claude’s answers. Anthropic also admits that monitoring for misuse with high fidelity while keeping data retention limited “has no easy answer.”
Watching itself is the unsolved part
My read: the most telling line is in the limits section. Anthropic says self-assessment “is not yet a reliable monitor of Claude’s own conduct.” A model hiding a bad action could hide it from its own assessment too, and an honest model can still fail in the same way as the agent, because the same judgment produced both the action and the check.
My read: that’s the same problem behind OpenAI’s agents switching off the alarms meant to watch them, which we covered on October 22. Monitoring users looks like the easier half. Anthropic’s post doesn’t claim to have a monitor that reliably watches the model it runs inside.
For everyday users nothing changes on screen. My read: the practical point is that on a large share of Claude traffic, a model-written safety note about your conversation now exists, even if no person ever reads it.
Note on sources
Every figure above comes from Anthropic’s Safeguards post, which was read in full. The group sizes and token counts are printed in its chart labels, so they come from the figures, not the running text. Only the October 29 announcement date comes from Truviber, which says its article was reported with AI assistance and edited. Other outlets also covered the post, but Truviber is the only secondary write-up cited here.