LLM prompt caching costs: cache reads, writes and hit rates
Savings depend on actual cache hits, read prices, write charges and retention. Do not apply the cache-read price to every input token.
Separate ordinary input, cache reads and cache writes
These usage categories can have different unit prices. A provider may cache automatically or require explicit cache creation. It may also charge for storage time. A missing cache-write quote does not establish that writes are free.
Read the usage-field definitions carefully. Cached tokens may be a subset of total input tokens. If total input includes hits, subtract those hits before calculating ordinary input charges. Do not charge the same tokens as both ordinary input and cache reads.
Work through a cache-hit example
Assume total input is 10,000 tokens, including 8,000 cache-hit tokens. If ordinary input costs CNY 2 per million and cache reads cost CNY 0.2 per million, the input charge is 2,000 / 1,000,000 × 2 + 8,000 / 1,000,000 × 0.2 = CNY 0.0056. With no cache hits, input would cost CNY 0.02.
This is a hypothetical read-cost example, not a model quote. Output, initial writes, storage and rebuilding expired caches must be calculated separately. Apply the hit price only when actual provider usage identifies the cached portion.
Similar questions do not guarantee a hit
Cache rules can depend on matching prefixes, minimum token counts, model versions, retention times and request formats. Semantic similarity does not necessarily mean a prefix is reusable. Changing instructions or context can change matching behavior. Check the official API documentation.
Observe hit rates and charges with your real workload. Repeated, stable context is useful for testing. Do not remove information that needs to be updated just to increase hits; verify that responses remain correct.
Include the cost of establishing and retaining the cache
Compare the cost of the same workload without caching against uncached input + cache reads + writes + storage + output. Repeated-read savings must exceed the incremental cost of establishing and retaining the cache before there is a cost benefit.
If you do not know the actual hit rate, calculate both zero-hit and expected-hit scenarios. Model detail pages show observed billing items and conditions. Return to official documentation when an item is missing instead of assuming a zero price.
Include the cache write charge
Teaching assumptions: ordinary input CNY 2/million, cache reads CNY 0.2/million, and an incremental initial write charge of CNY 3/million. For 100 requests with 10,000 input tokens each, 8,000 reusable tokens: no-cache input costs CNY 2. With caching, ordinary input 0.4 + reads 0.1584 (99 requests) + initial ordinary prefix input 0.016 + incremental write 0.024 = CNY 0.5984. Add output and any storage charges. Actual write billing depends on the provider.