How to reduce LLM API costs without making it worse

Most guides on how to reduce LLM API costs open with “use a smaller model”, which is the lever with the worst ratio of savings to quality loss. Here are seven, ordered by what they save per hour of engineering — starting with the requests you shouldn’t be sending at all.

Measure per-feature before you change anything

You can’t optimise a single line item. Tag every call with the feature that made it, then look at the breakdown.

Nearly always, one feature dominates in a way nobody predicted — a background summariser that runs on every save, a retry loop that fires three times, a system prompt included in a high-frequency call. Fixing that one thing usually beats every other item on this list combined, and you’d never find it from the invoice.

Log per call: feature, model, input tokens, output tokens, cached tokens, latency. An hour of work.

1. Don’t send the request

The cheapest token is the one you never spend. In our own Telegram beta, 56% of 656 real conversational requests were filler — “thanks”, “ok”, “got it” — answered by a local template path with zero model calls (measured March–May 2026). That share is traffic-mix-specific, but the class exists in every conversational product.

The bar for this filter must be high: misclassifying a real request as filler is a much worse failure than overspending on it. Handle only the unambiguous cases and default to sending everything else.

2. Cache aggressively

Two different mechanisms, both worth having.

Prompt caching, where the provider caches a long shared prefix — your system prompt, tool definitions, retrieved documents. Discounts on cached input are substantial. Put everything stable at the front of the prompt and everything variable at the end, or you get no hits at all. This one change often pays for itself the same day.

Response caching for identical requests. Deterministic questions over unchanged data don’t need re-inference. Hash the normalised input, keep a short TTL, and be careful the cache key includes anything that would change the answer.

3. Shrink the context, not the model

Long context is the quiet cost driver, and it’s usually accidental: whole documents pasted in when three paragraphs mattered, chat history resent in full every turn, tool definitions for a thousand tools when the model needs six.

Retrieve what’s relevant instead of sending everything. Summarise old turns rather than replaying them verbatim. And select tools at runtime rather than shipping the whole catalogue — which becomes its own engineering problem at scale.

Quality usually goes up here, because less irrelevant context means less distraction.

4. Route by task, not by tier

Now the model choice — but the framing matters. Rather than downgrading everything, match the request to the model that fits its shape: reasoning to a reasoning model, bulk classification to a small fast one, long drafting to a long-context one.

The spread between tiers is what makes this worth engineering. Anthropic’s published list prices, as of July 2026:

Model	Input $/1M tokens	Output $/1M tokens
Claude Opus 5	$5.00	$25.00
Claude Sonnet 5	$3.00	$15.00
Claude Haiku 4.5	$1.00	$5.00

A worked example, with the assumptions stated so you can rerun it on your own numbers. Suppose a workload of 1M calls/month averaging 500 input + 200 output tokens per call. On Opus 5 alone, each call costs $0.0075 — $7,500/month. If 40% of those calls are trivial (greetings, acknowledgements, simple lookups — measure your own share) and you route them to Haiku 4.5 at $0.0015 a call, you save 400,000 × $0.006 = $2,400/month, a 32% cut, without touching the quality of the calls that needed the frontier model. The same arithmetic at 60% trivial saves $3,600.

That’s a whole discipline; we wrote it up separately. The cost point is that routing captures most of the savings of downgrading without the quality cliff — and the table shows why: the tier gap is 5× per token, and it compounds with every call you move.

5. Cap output length

Output tokens usually cost several times input tokens. An unbounded max_tokens on a summarisation endpoint invites the model to write an essay.

Set explicit ceilings per call type and ask for the format you want. “Answer in at most three sentences” is a cost control as well as a UX decision.

6. Batch what isn’t interactive

Most providers offer a batch tier at a significant discount for work tolerating delayed completion. Nightly enrichment, backfills, bulk classification, evaluation runs — none of these need a synchronous response.

The engineering cost is a queue and a completion handler. If you have meaningful non-interactive volume, this is the largest single discount available.

7. Kill the retry amplifier

Check this before shipping. A retry-on-failure wrapper around a call that fails deterministically — malformed schema, context too long, content filter — turns one wasted call into three or five, forever.

Retry on transient errors (429, 5xx, timeout). Never retry on 400-class validation errors. Cap attempts, add backoff, and log retry counts so the amplifier is visible.

The order to actually do them

If you have a day: measure per-feature, fix whatever dominates, restructure prompts for caching, and audit your retry policy. That’s most of the savings for most teams.

If you have a week: add routing and batching.

What I’d skip early: hand-tuning prompts to shave tokens. It’s slow, it makes prompts worse to read, and it saves less than one caching fix.

Velaris classifies each prompt before it costs anything — see the router.