Model selection is a routing problem
'Which model do we standardize on' is procurement thinking applied to a per-call decision. Escalation rules, premium criteria, and eval gates are one framework wearing three hats.
the frugal blog
Tool calls are the new cloud bill.
Provider routing, AI tool-call costs, model selection, and the unglamorous work of keeping agents reliable. Written by Brian Sparker, creator of Frugal.
Price ladders: a mental model for every API dependency
The free-cheap-premium ladder isn't a search trick. Extraction, transcription, embeddings, and code execution all have the same shape — here's how to build the ladder.
Why we log latency_ms on every response
One field on every tool response does two jobs: it routes the next call and it explains the last one. A worked example from a real 1.5-second search.
MCP servers are the new CLI tools
Unix won with small tools composed by a shell. Swap the shell for an agent and the pattern repeats — except pipes were free, and tool calls have a meter.
Anatomy of a deep research agent
Log a research run and a shape appears: one plan, a burst of searches, a train of extractions, one long synthesis. Each phase costs differently. Route accordingly.
The hidden cost of retries
Every retry is a repurchase at full price with worse odds than the first attempt. Here's the playbook: budgets per task, hard no-retry rules, and fallbacks.
When a premium model is worth it (and when it's just expensive)
The top-tier model is worth 10x on maybe a tenth of your calls. The trick is knowing which tenth — and it starts with what a wrong answer costs you.
The boring parts of the AI product are the product
Your model call is forty lines anyone can write. The moat is the unglamorous machinery around it — and the judgment encoded inside that machinery.
5 provider routing mistakes that quietly burn your budget
None of these show up as errors. They show up as an invoice that's 3–6× what the same traffic should cost, one reasonable-looking call at a time.
Graceful degradation for AI features
Every AI feature already degrades. The only question is whether you designed the degraded states or your users are discovering them for you.
How to evaluate an MCP server before you install it: a 10-point checklist
Installing an MCP server hands your agent new capabilities and someone else new visibility. Ten checks to run before it touches your config.
Build vs buy for agent infrastructure
Your evals are your product. Your retry loop is not. Splitting the agent stack along that line answers the platform question before a vendor call.
Unit economics for tool calls: budgeting an AI product
One click becomes forty API calls before anyone looks up. The budgeting discipline that survives that fan-out, worked through with real search rates.
Local models are underrated infrastructure
Everyone prices the model ladder from cheap-API up. There's a rung below that: weights on hardware you already own, at a marginal cost of zero.
Timeouts are a product decision
Somewhere in your config is a number deciding how long users stare at a spinner for an answer that may never come. Nobody senior chose it.
Routing tool calls vs routing model calls
The same ladder idea, two different physics. Why a router that treats search results and model completions identically gets both of them wrong.
Frugal is the open routing layer for AI tools
Cost was the wedge. Routing is the product. What's shipping, what changed, and what Frugal Cloud will add.
Benchmarks lie: evaluate models on your own traffic
A model's leaderboard position tells you how it does on the leaderboard's distribution, not yours. Fifty queries from your own logs beat it every time.
Hardening an MCP server's HTTP transport
The moment an MCP server leaves stdio for HTTP, it becomes a network service with everything that implies. The hardening playbook I actually run.
Instrumenting an agent: what to log from day one
Six fields, one JSONL file, first deploy. The instrumentation playbook that answers every question you'll be asked about your agent in month three.
Token prices fell 90%. Tool prices didn't.
Inference got an order of magnitude cheaper and everyone noticed. The per-call price of search, scrape, and browse barely moved, and almost nobody did.
Inside Frugal's router: how a search call picks its provider
One captured run, walked rung by rung: 0 hits in 529ms, a fall-through, 3 hits at $0, and exactly what lands in the ledger afterward.
Latency, quality, cost: pick the two your product needs
Autocomplete, overnight reports, and research tasks live in different corners of the same triangle. Products that pick one corner for everything overpay in all three.
5 failure modes every agent tool chain should survive
Your tool chain will break in exactly five ways. Here's what each one looks like in the logs, and what a chain that survives it does differently.
Orchestration patterns for multi-tool agents
Pipeline, fan-out/fan-in, escalation ladder, verify loop. Each pattern has a signature failure: a place the bill or the latency hides until production.
BYOK is a feature, not a compromise
Every middleman reselling API calls adds margin and a second data controller. The case for bring-your-own-key as trust architecture, tradeoffs included.
Search is most of your agent's bill
Everyone budgets for tokens. Then a teardown shows search at 54% of task cost. The math behind why research agents multiply search spend.
Free-first: why your default provider should cost $0
Paid providers should be the fallback, not the default. The case for making your agent's first call free, with the failure modes stated honestly.
6 questions to ask before switching LLM providers
Most teams discover their switching cost the day a price hike or outage forces the move. Ask these six questions while the answer is still cheap.
Zero hits is not an error
An empty result set arrives as a 200 and gets treated like a 500. The provider didn't fail — it answered. What you do next should depend on whether that answer cost money.
Evals before scale: the cheapest insurance in AI engineering
Thirty real queries with expected answers will save you more money than any discount negotiation. You can't swap models, cut costs, or route traffic without a way to score the output.
stdio or Streamable HTTP? Choosing an MCP transport
One transport is a subprocess and a pipe. The other is a service you now operate. Which MCP transport you pick is really a question about who connects, from where.
Cost observability for agents: put cost_usd on every result
Your invoice knows what you spent by provider and month. It has no idea what one agent run cost. Stamp the cost on the response envelope and the question answers itself.
Fallback chains: designing a provider ladder that doesn't page you
A backup provider you've never exercised is a config value, not a plan. The rules I use to order rungs, avoid paying twice, and break flapping providers out of the chain.
Small models first: the case for escalation
Escalation isn't a cost hack, it's an architecture: the cheap model handles the bulk, the expensive one earns its rung. The same ladder logic that works for tool calls.
Latency budgets for agent tool calls
An agent chains tool calls in sequence, and sequences are where tail latency compounds. How to set a budget, split it per call, and spend it on fallback rungs.
Shipping your first AI feature: a checklist for product teams
The demo is the easy 20%. This is the other 80% — seven checks that decide whether your AI feature survives contact with real users and a real invoice.
What is an LLM router — and do you actually need one?
The term gets used for two different things. Here's what a router actually contains — policy, health, fallback, accounting — and the honest cases where you should skip one.
7 ways to cut your AI agent's API bill this week
No replatforming, no procurement cycle. Seven changes you can ship by Friday that attack the parts of an agent bill nobody is looking at.
The MCP ecosystem in 2026: what's actually getting used
Forget the spec tour. This is what MCP looks like from the inside of a server that runs every day: the parts that stuck and the parts that didn't.
Designing AI agents that survive provider outages
Your agent is a distributed system with a single point of failure per dependency. The playbook for staying up when a provider doesn't.
How to choose a model for your agent workload
Model selection for agents isn't a leaderboard question. It's five workload questions, and most teams skip straight to the logo.
The rack-rate gap: why the same web search costs $0 or $0.005
Five providers, one capability, a price range from free to half a cent. An analysis of what you're actually paying for when your agent searches the web.
Provider routing for AI agents: the complete guide
Agents pick providers by what got wired in, not by price. Here's how a routing layer walks every tool call down a ladder — free first, paid as fallback.