Cache-Hit Discounts

A cache-hit discount reduces the price of input tokens when an upstream provider recognizes a repeated input prefix. Token Market requires no extra switch. When the same provider detects a cache hit and offers cache-hit pricing, the platform bills the tokens in the actual cache-hit category.

1. Billing Calculation

Cache discounts apply only to input tokens, not output tokens. The input cost of a request usually has two parts: “uncached input tokens” and “cache-hit input tokens”:

Uncached input cost = uncached input tokens × input price
                   + cache-hit input tokens × cache-hit price
Output cost        = output tokens × output price
Total cost         = input cost + output cost

Cache-hit pricing depends on the model and provider. Discount rates, billing units, and availability can vary. Check current pricing and invoice details on the model page, and do not hard-code discount rates in application code.

2. Discount Requirements

A request receives a cache-hit discount only when both conditions are met:

• The request hits the upstream provider’s cache.
• The current model and provider offer cache-hit pricing, and the model’s billing details show that pricing.

A request may receive no discount when the provider lacks Prompt Caching, the input prefix differs, the cache expired, routing selected another provider, the provider omitted the cache-hit token count, or the provider has no cache-hit price. Similar text does not guarantee an identical prefix or a cache hit.

3. Verify Actual Charges

Check in this order:

1. Check the response’s usage data to confirm whether the provider reports cache-hit input tokens.
2. Check the actual model, provider, and token usage in the request logs to confirm routing used the expected provider.
3. Compare the standard input price, cache-hit input price, and output price on the model details page.
4. Use the final invoice as the source of truth. Response statistics help with troubleshooting; the invoice records the actual charge.

If the response omits a cache-hit category, do not infer a discount from the request prefix alone. Verify actual billing in the logs and invoice.

4. Improve the Cache Hit Rate

Cache hits usually depend on a stable input prefix. These practices can improve the hit rate:

• Keep system prompts, tool definitions, and prior conversation turns stable.
• Reuse the same fixed prefix for similar tasks, including role definitions, rules, output requirements, and tool schemas.
• Place changing user input, timestamps, random IDs, and experiment flags after the stable prefix;
• Do not reuse sensitive information or cross-tenant context solely to increase cache hits.

5. Interaction with Smart Routing

Cache state is a real-time, provider-specific billing capability. Price-first or automatic routing may select different providers, which changes cache-hit behavior. To measure caching consistently, run a low-volume test with the same model, provider, and prefix, and record the actual invoice.

6. Important Limitations

• The platform does not guarantee a cache-hit discount; each request depends on the actual upstream result.
• A cache hit reduces only the cost of qualifying input tokens, not output tokens.
• Prices, fields, and settlement rules may change by model or provider. Production systems should read current model details and billing data;
• Do not put an API Key, personal data, or private cross-user content in a reusable shared prefix.