Amazon engineers are reportedly distilling Anthropic models to cut costs before new token-based pricing kicks in

Worried about rising costs, some Amazon engineers are already distilling Anthropic models to build smaller, cheaper versions for internal use. That's according to a report from The Information. Distillation works by having a smaller model learn from a larger model's outputs. Amazon has certain rights to use Anthropic's models for this purpose, according to a person familiar with the matter, similar to Apple's arrangement with Google Gemini. Amazon does offer a distillation service on its Bedrock cloud platform, but Anthropic's Claude models aren't available there; only Amazon's own Nova models and Meta's Llama models are supported.

The effort ties back to a renegotiation of the partnership, according to The Information. Starting next year, Amazon will pay for Anthropic's models based on tokens processed rather than compute hours, which could push costs up sharply. An Amazon spokesperson pushed back, saying the changes from the expanded partnership won't raise costs. Anthropic points to lower prices relative to the performance its models deliver.

Amazon is reportedly exploring alternatives like OpenAI and its own Nova models. The company has invested up to $25 billion more in Anthropic and up to $50 billion in OpenAI this year.