How AI API Usage-Based Billing Works: Balance, Tokens, Cache, and Cost Spikes

AI API cost control is not only about whether a model is expensive. You need to understand what each request does: which model was called, how many input and output tokens were used, whether cache was involved, whether retries amplified traffic, and which account balance paid for it.

Four layers to separate

• Account balance: funds added to the account for future API calls.
• Model pricing: input, output, cache write, and cache read prices can differ by model.
• Usage records: model, time, tokens, status, cost, and key ownership.
• Balance changes: used to reconcile recharges, deductions, refunds, and incidents.

A simple cost formula

single request cost =
  input tokens / 1,000,000 * input price
+ output tokens / 1,000,000 * output price
+ cache write tokens / 1,000,000 * cache write price
+ cache read tokens / 1,000,000 * cache read price

Why cost spikes happen

• Growing context: full history, logs, or documents are repeatedly sent.
• Agent loops: one user request triggers dozens of model calls.
• Aggressive retries: 429 or 5xx errors are retried without backoff.
• Model switching: workloads move to a stronger model without changing strategy.
• Key leakage: unfamiliar projects, time ranges, or models appear in usage records.

Practical ways to reduce cost

• Tier tasks: use lighter models for classification, rewriting, and simple summaries.
• Control context: send only information needed for the current task.
• Reuse cache for system prompts, long documents, and fixed background context.
• Set limits on test keys, script keys, and agent keys.
• Review usage by key notes to identify which project is consuming balance.

Cost spike investigation order

• Check the time range: when did the spike start?
• Check the key: which project, environment, or tool made the calls?
• Check the model: did traffic move to a higher-cost or longer-context model?
• Check request type: did streaming, tool calls, or agent loops multiply requests?
• Act: disable suspicious keys, lower limits, export records, and contact the Support & feedback page in the console.

Team usage recommendations

• Use one key per project and one key per environment.
• Review balance changes and high-cost models weekly.
• Add max steps, max tokens, and budget thresholds for scheduled jobs and agents.
• Document cost incident response in your operations playbook.