How to Pick an AI Model Without Burning Your Budget
A new model drops roughly every week. Half of them get coverage. Most of them are not worth your time or your API budget. And almost none of the coverage answers the only question you actually have, which is: should I use this for the thing I’m doing right now, or keep using what I’ve got?
That’s the question this page is about. Not “what launched this week,” but “how do you decide what to run when there are a hundred names on the leaderboard and the prices fan out by a factor of ten.” I’ve been writing software for about twenty years, I’ve watched a lot of tooling hype come and go, and I can tell you the model space is the noisiest version of it I’ve ever seen. This is my honest method for cutting through it.
I learned it the expensive way, so you don’t have to.
Contents
The $15 Coffee Break
Here’s the thing that started all of this. I have a Pro Claude subscription that I love, but I kept running into the session timer on weekends when I actually have hours to code. So I went looking for a second rail, something I could swap onto through OpenRouter when Claude rate-limited me mid-bug-hunt.
That was a fine idea right up until I picked an unfamiliar model because it was cheap. I scrolled the OpenRouter listings, saw a price that was a fraction of what Claude charges, told myself I was being smart about costs, and shipped it. Then I walked away to grab coffee. I came back fifteen minutes later to a dashboard telling me I’d just spent fifteen dollars.
The model couldn’t handle tool calls. Instead of finishing a task and stopping, it looped. Called the same tools, got confused by the results, called them again. It wasn’t reasoning, it was a stuck record that happened to cost money per revolution. Fifteen bucks isn’t a fortune, but fifteen bucks per coffee break adds up fast.
The lesson burned into me that afternoon is the whole foundation of how I pick models now: cheap and broken is more expensive than expensive and correct. The sticker price on the listing is not the cost. The cost is the sticker price times how many times the thing fails, loops, or hands you garbage you have to redo. A model that costs ten times as much but works on the first try is often the cheap option once you do the real math. I wrote the full story of that disaster (and what I built to stop repeating it) in Building a Cost-Saving Skill That Accidentally Became Its Own Newsletter.
So when you read “this model is cheaper,” the right reflex is not “great, switch.” It’s “cheaper per token, sure, but is it cheaper per finished task?” Those are very different numbers, and only one of them is on the pricing page.
The Three Signals That Actually Tell You What to Use
The reason model selection feels impossible is that every source of information about models lies to you in its own specific way. Once you know how each one lies, you can triangulate. I cross-reference three kinds of signal, and the truth lives in the intersection, never in any single one of them.
Volume: where the money is actually flowing. Token counts on a public router like OpenRouter tell you what people are trying. This is useful because it’s real usage, real dollars, not a survey. But it lies in two ways. First, a brand-new model giving itself away during a free promotional window will spike to the top of the rankings and tell you nothing, because free usage is not the same as chosen usage. Second, volume tells you what people are testing, not what they kept. Plenty of models get a big curious spike and then get quietly abandoned the next week.
Head-to-head: what people prefer when they can’t see the label. This is the Arena style of signal, where two anonymous model outputs sit side by side and a human picks the better one. It’s great because it strips away brand loyalty. Nobody’s voting for Claude or GPT because of the logo, they’re voting on the answer in front of them. But it lies too: it measures preference in voting conditions, which is a single prompt and a snap judgment, not what happens when you actually live with a model across a long coding session where context management and tool reliability matter more than one slick paragraph.
Lived experience: what people say after weeks of real use. This is the Reddit and forum signal, the “I switched to X for production and here’s what broke” reports. It’s the most honest about the stuff that actually bites you, because it’s specific: real projects, real failures, real switching stories. But it’s loud-minority biased and slow. The people posting are disproportionately the ones who had a strong experience in one direction, and by the time a consensus forms, the model has often been superseded.
Each one is wrong on its own. Volume without preference shows you marketing stunts. Preference without lived experience shows you models that win one prompt and lose your afternoon. Lived experience without volume shows you whatever ten loud people on a subreddit are mad about this week. Put all three together and the picture sharpens fast. That triangulation is the entire job, and it’s the thing I eventually automated. More on that at the bottom.
How to Read the Arena Leaderboard (the Cheapskate Math)
The single most useful thing I figured out in all of this is how to actually read a head-to-head leaderboard, because almost everyone reads it wrong. They look at the top of the board, find the #1 model, and assume that’s the answer.
Here’s what’s really going on. The top of the leaderboard is compressed. The entire visible top end (the models everyone is fighting over) fits inside a tiny rating spread. The gap between the #1 model and the #8 model is often smaller than the noise. Meanwhile the prices for those same models fan out by an order of magnitude or more. You have models within a rounding error of each other on quality, where one costs ten or twenty times what another does.
So the question is almost never “what’s the best model.” The best model and the fifth-best model are, for most real work, indistinguishable. The question that actually saves you money is: what is the cheapest model that’s still in striking distance of the best, for the specific category of work I’m doing? Best-for-coding, best-for-writing, best-for-cheap-bulk-classification. Each of those has a different answer, and in each case the smart pick is usually not the one at the very top.
That insight is the centerpiece of The Cheapskate’s Guide to the Arena Leaderboard, which is the most important post on this page if you only read one. It’s the difference between paying frontier prices out of habit and paying frontier prices only when the task genuinely needs them.
The discipline is simple to state and hard to follow: stop optimizing for the top of the board and start optimizing for the best value at the quality bar your task actually requires. Most tasks do not require the top of the board.
Cost vs Capability: Which Model for Which Job
Once you stop treating “best” as a single number, you start matching models to jobs, and the spending drops without the quality dropping with it. The framework I use is boring on purpose:
Match the model to the cost of being wrong. If you’re doing throwaway exploration, prototyping, or bulk work where an occasional bad output is cheap to catch and redo, run something cheap and good-enough. If you’re doing production agentic coding where a subtle wrong answer compounds into hours of debugging, that’s exactly where the premium is worth paying, because the expensive failure isn’t the API bill, it’s your afternoon.
This is the real shape of the question people are actually searching, things like “is the premium model worth the price bump over the mid-tier one for production coding.” The honest answer is: for low-stakes work, almost never; for high-stakes agentic work where reliability and tool-calling discipline matter, often yes. The price premium buys you fewer loops, fewer hijacked tool calls, and fewer confident wrong answers, and those are the things that actually cost money downstream.
Watch out for the words that stopped meaning anything. “Flash,” “mini,” “cheap,” “fast.” These used to be reliable signals for “smaller and dumber but good enough for easy stuff.” They’ve drifted. Some of the cheapest models on the internet are now genuinely competitive, and some models wearing the budget-tier naming are priced like anything but. I dug into exactly this drift in The Cheapest Model on the Internet Is Winning, where the naming conventions have come completely unmoored from the pricing.
Capability is not the only axis. Reliability is its own thing. A model can be brilliant and still useless to you if it can’t follow a tool-calling protocol, or if it confidently lies to your face about something it didn’t do. The smartest model in a category can also be the most willing to make something up, and a model that’s slightly less capable but never goes off the rails is worth more in an agentic loop. Raw capability and operational reliability are separate columns, and the leaderboard only shows you one of them.
And sometimes the newest, smartest model is a liability. When Anthropic shipped its most capable model yet, the same release also made it easier to hijack. “Most capable” and “safest to point at your codebase” are not the same property, and the gap between them is exactly the kind of thing the launch coverage skips.
The Model Roundup: How I Keep This Current
Everything above is the evergreen framework. The problem is that the answers change every single week, even though the method doesn’t. The cheapest-good-enough coding model in striking distance of the top is a different model this month than it was last month. So I needed something to keep the picture current without spending every Saturday morning doing it by hand.
That something is a Claude Code skill that runs the triangulation for me, cross-references the three signals, and produces a five-minute brief I read before I open OpenRouter. It started as a pure cost-saving tool and accidentally turned into a weekly blog post, which is the Model Roundup series you’ll find on this page. I told the whole story of building it, including the self-editing adaptation log that keeps it from rotting, in Building a Cost-Saving Skill That Accidentally Became Its Own Newsletter.
The roundups are the living layer under the framework. Each one applies the method above to whatever actually happened that week. Read newest first:
If you want the method, read this page. If you want this week’s answer, read the latest roundup. The two are designed to work together.
So Which Model Should You Use?
Here’s the honest version, the one the launch coverage never gives you.
Stop chasing the top of the leaderboard. The top is compressed and the prices are not, so paying for #1 out of habit is just lighting money on fire for a quality difference you can’t feel. Figure out the cost of being wrong for the specific job in front of you, and buy exactly that much capability and no more. Run something cheap and good-enough for the low-stakes work, and pay the premium only where a confident wrong answer would cost you an afternoon. Trust the intersection of the three signals, never a single ranking. And remember that the sticker price is not the cost, because cheap and broken bills you twice.
Do that and you’ll spend less, ship more, and stop refreshing the rankings every time a new name trends. The articles below are the long version: the specific weeks, the specific models, the specific times I was wrong and said so. Read the ones that match whatever you’re about to build on.
And if you’d rather just be handed the current answer every week instead of working it out yourself, that’s exactly what the Model Roundup is for.
