Something about how the model reads instructions has changed, so the prompt that was tuned for the old behaviour is now tuned for nothing. Rerun first, then change the prompt against the dataset, not against a hunch.
The instruction contract moved. Prompt edits are evaluated before they ship, like any other change.
Anthropic launched claude-fable-5-1 and claude-mythos-5-1 with a 1M token context window by default, 128k maximum output tokens and always on adaptive thinking, priced at $10 and $50 per million tokens. The notes state that tool_choice types any and tool are not supported and return a 400 error, and that thinking blocks are preserved only for the same model or a newer one.
Agent: tool_choice types any and tool now return an error, so the way the agent is made to call a tool changes. Arize Phoenix reads tool_path, the ordered tool calls with their arguments, and tool_io, what each tool was called with and what it returned, and warns that a correct looking answer can come from a broken path.
Extraction: Forcing a tool call is one way to fix an extractor's output shape, and that request now fails. DeepEval defines a golden as an input plus the expected result, which becomes a full test case once the application produces the actual output, so the same set reruns once the shape is settled.
Revisit the promptSearch your codebase for tool_choice values of any or tool, since those requests now fail, and check any code that replays thinking blocks across models.
Anthropic launched claude-opus-5 at $5 and $25 per million tokens, with a 1M token context window as both the default and the maximum and 128k maximum output tokens. The notes state that thinking is on by default, that disabling thinking is only allowed at effort high or below, which they call a breaking change from Claude Opus 4.8, and that Fast mode was removed for Claude Opus 4.7 so those requests return an error.
Agent: Thinking on by default, and a narrower way to turn it off, change how much the agent works out before it acts. Future AGI describes scoring an agent at four resolutions, end to end, trajectory, step and turn, and writes that a high end to end score can hide bad tool calls, while strong per step scores can still miss that the agent pursued the wrong goal.
Chat assistant: Future AGI recommends refreshing a golden dataset immediately after a model update, monthly by default and weekly during rapid change, sampling 200-500 recent production traces stratified by intent and persona and versioning the result as a new artifact.
Generation: If a model from this family also scores your outputs, the judging side moves too. Future AGI writes that a judge model can move 3-5 points on a fixed rubric in a single month without any change on your side.
Revisit the promptFind any request that disables thinking at effort xhigh or max, since that combination is now rejected, and remove the fast mode flag from Claude Opus 4.7 calls.
Google released gemini-3.6-flash and gemini-3.5-flash-lite as generally available. The same entry deprecates the sampling parameters temperature, top_p and top_k.
Classification: Classifiers usually pin sampling parameters in the request builder, so a deprecation here is a change to caller code rather than to the model you chose. Anthropic made the same kind of change from Claude Opus 4.7 onward, where a non default temperature, top_p or top_k returns a 400 error.
Extraction: Removing sampling parameters changes the request every fixture runs through. Braintrust tracks every change to a dataset so experiments can pin to specific versions, which is what keeps the run before the edit comparable with the run after it.
Revisit the promptSearch your request builders for temperature, top_p and top_k on Gemini calls and plan their removal, then rerun the affected cases with the parameters dropped.
Anthropic launched claude-sonnet-5 at an introductory $2 and $10 per million tokens with a 1M token context window and 128k maximum output tokens. The notes state that adaptive thinking is on by default, that manual extended thinking was removed and returns a 400 error, that non default sampling parameters return a 400 error, and that a new tokenizer produces about 30 percent more tokens.
Classification: Non default sampling parameters now return a 400 error, which breaks caller code without any model swap on your side. Anthropic first made this parameter deprecation from Claude Opus 4.7 onward.
RAG: A tokenizer that produces about 30 percent more tokens changes how much retrieved text fits in one prompt. Ragas defines faithfulness as the share of claims in a response that can be inferred from the retrieved context, so a smaller retrieved context can move the score by itself.
Summarisation: The same input now costs more tokens, so what reaches the summariser can change before the model does anything. Galtea warns against updating ground truth to match model behaviour when the summaries come back different.
Revisit the promptRecalculate token budgets and cost estimates against the new tokenizer, and strip manual thinking and non default sampling parameters from requests before they start failing.
Anthropic launched claude-opus-4-6 at $5 and $25 per million tokens. The notes state that adaptive thinking is recommended and manual thinking is deprecated, that an effort parameter with levels low, medium, high, xhigh and max launched, that a 1M token context window is available in beta with long context pricing above 200k input tokens, and that the media limit rose from 100 to 600 images or PDF pages with the 1M context.
Agent: An effort level replaces a thinking budget you used to set by hand, which changes what happens between the request and the answer. Arize Phoenix writes that end to end evaluation only inspects the two endpoints, the input and the final output, and that reasoning, tool selection and the decision path are where the interesting failures sit.
Generation: Moving from a manual thinking budget to adaptive thinking plus an effort level is a prompt change, and the text comes back different. Galtea warns against updating ground truth to match model behaviour.
Revisit the promptSwitch requests from manual thinking budgets to adaptive thinking plus an effort level, then rerun the cases where you previously tuned the thinking budget by hand.
Anthropic took structured outputs out of beta on the Claude API for Claude Sonnet 4.5, Claude Opus 4.5 and Claude Haiku 4.5. The notes state that the output_format parameter moved to output_config.format, and that structured outputs remain in public beta on Amazon Bedrock and Microsoft Foundry.
Extraction: The parameter that carries your schema moved, so the request changes while the fields you expect do not. DeepEval defines a golden as an input plus the expected result, which becomes a full test case once the application produces the actual output.
Classification: Where labels come back through a structured output, the shape of the response is part of the feature. Galtea recommends a tiered release gate in which blocking failures such as safety or policy violations block a deploy at any rate, while regressions gate on the change in fail rate.
Revisit the promptMove output_format to output_config.format in your request builders and rerun the schema validation cases on both the Claude API and any partner platform you use.
Change-log entries restate provider announcements with sources and dates. Impact notes cite public practitioner reports. The site publishes no measurements of its own on any model.
Questions about this action
Who decides the recommended action?
We do, from what the provider announced, and it is one of exactly four: rerun, re-pin, re-prompt or migrate. It is a statement about what a team does this week, not a prediction about your feature.
Does an action mean something is wrong with my feature?
No. It means something your feature depends on moved. Whether anything about your output changed is a question only a rerun against your own dataset can answer.
What happens if we are on a plan?
A tracked provider update triggers a rerun of your suite within 2 business days of the provider's announcement, and you get a written verdict. That is one of the six written guarantees on every plan.