Skip to content
Chain Defenders
AI Systems

Where AI Spend Leaks

These are the main places we look for savings. Each one has a quality risk, and we say what it is. Vendor pricing changes often. Check current pricing for your provider and model.

LeverWhat it isQuality risk
Model routingSend simple requests to smaller modelsMedium
Prompt cachingReuse repeated prompt prefixesLow
BatchingUse batch APIs for non-urgent workLow
Context trimmingSend less text per requestMedium
Output limitsCap response lengthLow to medium
Retries and loopsRemove runaway repeat callsLow
Duplicate callsStop asking the same thing twiceLow
Result cachingStore and reuse answersMedium
Model swapsMove to a cheaper modelMedium to high

Model routing

Not every request needs your largest model. Classification, formatting and short lookups often work on a smaller one. The risk is that the router sends a hard request to a weak model. We test this on real examples first.

Prompt caching

Vendors state that repeated prompt prefixes can be billed at a discount. Whether this applies depends on provider and model. Quality risk is low because the model sees the same text. The saving depends on how much of your prompt repeats.

Batching

Vendors state that batch endpoints for non-urgent work are priced at roughly half of standard rates. Results come back later, not instantly. It suits reports, back-office jobs and data processing. It does not suit live chat.

Context trimming

Prompts often carry old history or unused documents. Cutting them lowers cost. The risk is removing something the model needed. We test before and after.

Output limits

Long answers cost more. Setting sensible limits helps. The risk is cut-off answers, so we check where limits bite.

Retries and loops

A failing call that retries many times, or an agent that keeps looping, can burn money fast. Fixing these has little quality risk and often large effect.

Duplicate calls

The same input is sometimes sent several times by different parts of a system. Removing the duplicates saves cost with no change to output.

Result caching

If many users ask the same question, the stored answer can be reused. The risk is stale answers. We set expiry rules.

Evaluation before model swaps

Moving to a cheaper model can save money, but quality can drop in ways that are hard to see. We build a small evaluation set from your real cases and compare results before any swap. If quality falls, we do not recommend it.

No promises

We do not claim a set percentage. What is possible depends on your usage.