Notes on Inference Integrity
This article was created by Forethought. See all our research on our website.
Summary: Claude Fable’s deliberately triggered sandbagging shows that training-time targets are, by themselves, insufficient to guarantee particular LLM behaviors. To preserve the public’s reasonable confidence in LLM behaviors, LLM foundation model companies should take inference-time guarantees as seriously as their model specs.
When the system card for Anthropic’s Fable was published on June 9th, the card noted that using Fable for “frontier LLM development” would run contrary to the terms of service for the model. Therefore, the card continued, if a classifier on top of the Fable model detected that it was being used for such development, Fable’s behavior would be silently degraded “through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning.”
This change to Fable’s behavior was plausibly a system-level intervention that ran contrary to the training-time commitment within Claude’s Constitution. Claude’s Constitution states that Claude should help users “to the best of its ability or […] make any ways in which it is failing to do so clear, rather than deceptively sandbagging its response.” So although Claude’s Constitution tries to aim at this behavioral ideal, this system-level implementation of behavior steered Claude in the opposite direction.
Following public furor over this decision, Anthropic reversed course; Fable now falls back on Opus 4.8 for LLM frontier-development queries that trigger a classifier, and does so visibly.
This incident demonstrates that other invisible inference-time interventions are entirely technically feasible, even while keeping the training targets for some particular LLM the same:
An LLM trained to be honest could be less honest, if classifiers noted it was answering a sensitive question.
An LLM trained to be impartial, and not favor a particular company or government, might suddenly turn to favoring one of them, conditioned upon some classifier.
An LLM trained to never subvert user intent could suddenly start doing so, again conditioned on the same.
Note that such invisible inference-time interventions, unlike in the case of Fable’s sandbagging, might not be disclosed in any way.
Such undeclared changes to LLM behavior would be far easier to hide than undeclared changes to a training target. Such conditional changes could be rolled out or rolled back quickly. And such changes might influence only a very small fraction of users – one could in theory apply inference-time interventions to a specific demographic group, political party, or an individual person. So, not only do we live in a world where AI companies have not clearly pledged not to do such inference-time interventions; we also live in a world where, if they did so, it would be very hard for anyone else to know.
Furthermore, if AI companies build out the capacity and skill to conduct such inference-time interventions, other entities could lean on them to do the same thing for other purposes. As governments have leaned on social media to hide or promote certain kinds of speech, governments could lean on AI companies to alter LLM responses in some cases.
Without countermeasures, I think this dynamic broadly decreases the importance of prior work on model specs, including my own work.
What should be done about this?
AI companies themselves could take several different measures:
Don’t use hidden inference-time interventions. An AI company could pledge that it does not use undisclosed, conditionally applied inference-time interventions to degrade or alter answers. Afterwards, in the same way that it would be reasonable for an employee to whistleblow if they found that an AI company was violating its model spec, so also it should be reasonable for an employee to whistleblow if they found an AI company was applying undisclosed, invisible inference-time interventions. Such a pledge should also include some standard for prominence of all disclosed conditional interventions; they should be prominent, not in a previously-unknown section of a model card.
Replace model specs with system specs. An AI company could simply declare that the standard within their model spec or Constitution applies with equal weight to the model itself and to any particular deployment setup. Or they could propose a “system spec” in addition to the model spec, which would describe how their AI systems as a whole operate and behave.
Such voluntary measures should likely be succeeded by third-party verification in the future; but before then, such pledges can help provide the same – albeit rather weak – level of protection as model specs themselves. And in general, some AI-safety attention given to “model specs” should be redirected to work on “system specs.”
This article was created by Forethought. See all our research on our website.
 |
|