Tag: Prompt caching

Token accounting for AI agents: tokens, context windows, and caching from first principles

A model has no memory - everything that looks like memory is your code resending it. Get that one sentence right and you understand tokens, context windows, and prompt caching, and why an agent's bill balloons.

Sep 23, 2026