How to reduce LLM API costs without making your product worse

Disclosure up front: this guide is published by TierUp, an OpenAI-compatible API that routes requests by performance tier instead of model name. TierUp appears in section 6 alongside its alternatives, with its trade-offs stated as plainly as everyone else's. Sections 1–5 apply no matter whose API you use.

Most LLM bills aren't high because tokens are expensive. They're high because of three quiet defaults: every request goes to the flagship model, every request carries the full context, and nobody can say which feature the money went to. Fix those and the bill usually drops a lot — before you touch a discount, a router, or a cheaper vendor.

Here's the order of operations that actually works.

1. Right-size the model per task class

The single biggest lever. Teams pick a model when they prototype — usually the smartest one, because they're testing feasibility — and then every request inherits it forever. But production traffic isn't uniform. Sort yours into rough classes:

• Speed-class: classification, extraction, routing decisions, autocomplete, formatting. Small/fast models handle these fine.
• Balance-class: summarization, drafting, general chat, RAG answers over decent retrieval. Mid-tier models are usually indistinguishable from flagships here.
• Intelligence-class: complex generation, nuanced editing, hard extraction over messy input.
• Reasoning-class: multi-step planning, hard code generation, math, agent-loop decisions that everything downstream depends on.

The pricing spread between classes is roughly an order of magnitude. As one concrete pair: Sonnet-class retail runs around $3/$15 per 1M tokens in/out, while speed-class models run at a small fraction of that. If 70% of your requests are speed- or balance-class work being sent to a reasoning-class model, no caching trick will save as much as fixing that.

The cheap way to start: tag each call site in your codebase with its class, then downgrade the obvious ones and spot-check outputs. You don't need a formal eval harness to catch "we were using the flagship to title-case strings."

2. Cache what you're paying to recompute

Two distinct kinds, often confused:

• Prompt caching (provider-side): most major providers discount input tokens that repeat across requests — long system prompts, tool definitions, few-shot examples, big shared documents. To benefit, keep the stable part of your prompt byte-identical and at the front; don't interleave dynamic content (timestamps, user names) into the static preamble, or you break the cache on every call.
• Response caching (your side): if many users trigger semantically identical requests — "summarize this same article," canned onboarding questions — cache the completion itself, keyed on a hash of the normalized input. A boring KV lookup that skips the API call entirely beats any per-token discount.

3. Trim the context you send

Input tokens are usually the majority of spend in chat and RAG apps, and much of what gets sent is dead weight:

• Cap conversation history. Keep the last N turns verbatim and a running summary of everything older, instead of replaying the whole transcript forever.
• Tighten retrieval. Send the 3 chunks that matter, not 20 "just in case." Reranking is cheap compared to paying flagship rates to have irrelevant chunks ignored.
• Audit the system prompt. They accrete. If yours is thousands of tokens of edge-case instructions, you pay for every one on every request — prune, or split into per-feature prompts so each call carries only what it needs.
• Bound outputs. Set max_tokens deliberately and ask for terse formats. Output tokens are typically ~5x input price; verbosity is expensive.

4. Measure cost per feature, not per invoice

You can't fix what shows up as one line item. The provider invoice tells you totals; it can't tell you that your summarizer costs $0.002 per use while your agent loop costs $0.40 per run. Minimum viable cost accounting:

• Tag every request with the feature/call-site that made it (a metadata field or an internal proxy header).
• Log tokens in/out and computed cost per request.
• Once a week, look at cost per feature and cost per user action.

The distribution is usually badly skewed — one or two features dominate. That's where sections 1–3 pay off; everywhere else is noise. This is also how you catch runaway agent loops and retry storms before the invoice does. A daily spend cap at the account or proxy level is the seatbelt: it turns a $3,000 surprise into a $30 one.

5. Re-shop your model choice as prices move

This is the lever teams do once and never again. Model pricing has been on a steep downward ramp for years — the model that was the clear value pick six months ago rarely is today. But re-shopping has real friction: re-benchmark candidates, update model strings everywhere, re-run evals, watch for regressions. So it gets scheduled quarterly, then skipped.

Two honest options: make the migration chore cheap (centralize model choice in one config, keep a small eval set so switching is an afternoon), or delegate the choice to a routing layer — which brings us to the last section.

6. Routers and aggregators: LiteLLM vs OpenRouter vs TierUp

Three common ways to stop hardcoding a single vendor, with honest trade-offs. All three are good tools; the real difference is who owns the model choice. (For the wider field — Portkey, Helicone, Requesty, NotDiamond and friends — see our 2026 LLM router field guide.)

LiteLLM (self-hosted proxy/SDK) — open source; one OpenAI-style interface over ~every provider, with routing, fallbacks, budgets, and spend tracking you configure yourself.
Pros: full control, no per-token middleman, keys stay in your infra, deep customization.
Cons: you run it and you still own model choice — it makes switching easier but doesn't decide anything for you. Best for teams with infra appetite who want control. Model choice: yours.

OpenRouter (hosted unified API) — one hosted API over 400+ models with pass-through pricing plus a small fee on credits; mature, widely used, with configurable routing preferences and fallbacks.
Pros: zero ops, instant access to a huge catalog, battle-tested at scale.
Cons: by default you're still picking model IDs, so the re-shopping chore from section 5 remains yours. Best when you want breadth without ops. Model choice: yours (with routing assists).

TierUp (tier abstraction — our product) — an OpenAI-compatible API where the model name is replaced by a performance tier: tier-1 (Speed) through tier-4 (Reasoning). You classify the task (section 1); TierUp maps each tier to what we assess as the best-value model in that class, versioned server-side — when a better model ships, your tier upgrades with zero code changes, which automates section 5 too.
Pros: the churn-tracking chore disappears; pricing is flat per tier at ~50% under the retail price of the underlying models (tier 2: $1.50/$7.50 per 1M in/out vs ~$3/$15 retail Sonnet-class); tier 1 is currently free; daily spend guardrail built in.
Cons, stated plainly: you give up model pinning — responses strip provider and model details by design, so if you need a specific checkpoint, use LiteLLM or OpenRouter. TierUp is very early with ~zero production users. And the discount is transparently subsidized: TierUp routes through OpenRouter today and prices below cost on purpose while testing whether tier-routing is something developers actually want. That's disclosed on the site, and now here. Model choice: ours — that's the product.

The short version

1. Classify tasks and stop sending easy work to flagship models — biggest win.
2. Cache stable prompts (provider-side) and repeated responses (your side).
3. Trim history, retrieval, system prompts, and output length.
4. Attribute cost per feature and set a daily spend cap.
5. Re-shop model choice regularly — or delegate it to a layer that does it for you, with eyes open about who that layer is.

Want the arithmetic behind step 1 worked through with July 2026 prices? See Cheapest AI API in 2026: the real per-token math.