Gemini 2.5 Deep Think Tops the Reasoning Benchmarks. Now What?
On June 22, Google launched Deep Think mode for Gemini 2.5 Pro, and the headline numbers are real headline numbers: 82.4% on GPQA Diamond, the graduate-level science reasoning benchmark, ahead of Anthropic's Fable 5 (79.1%) and OpenAI's GPT-5.5 (76.3%). It also posted 89.8% on MMLU-Pro — reportedly the highest of any publicly available model — and state-of-the-art results on LiveCodeBench V6 and Humanity's Last Exam.
Deep Think isn't a new model. It's a new way of spending compute: instead of one chain of thought, it generates multiple reasoning paths in parallel and converges on an answer, trained up with reinforcement learning on step-by-step reasoning. Same weights, more inference. It's the clearest example yet of the industry's pivot from "train a bigger model" to "think longer at answer time."
A note on rigor before we go further: a HumanEval Plus score for Deep Think has been circulating in aggregator posts. We could not trace it to a primary source, so we're not repeating it here. That's the correct default posture for launch-week benchmark numbers — including, frankly, some of the ones above, which so far come from Google's own reporting and secondary coverage rather than independent evaluation.
What the benchmark race tells you
Something real, actually. GPQA Diamond is hard to game — it's graduate-level physics, chemistry, and biology written specifically to resist lookup. A six-point jump over GPT-5.5 on it suggests parallel inference-time reasoning genuinely works on problems with deep, verifiable structure: math, science, competition-style coding. Google's IMO results back this up — the research version of Deep Think reportedly performs at gold-medal standard, with the consumer version around bronze.
If your workload actually looks like this — formal verification, quantitative research, hard algorithmic problems — the reasoning race is delivering real capability gains, and Deep Think deserves a spot in your eval queue.
What it doesn't tell you
Three things the leaderboard conveniently omits:
Cost. Google hasn't published Deep Think pricing. Standard Gemini 2.5 Pro runs $2.50 input / $15 output per 1M tokens; estimates for Deep Think range from roughly 4x standard inference to 3–5x, per other analysis — all speculation until a price sheet exists. Parallel chains of thought means you're paying for every path explored, not just the one that wins. A benchmark chart with no cost axis is half a chart.
Latency. Extended parallel reasoning takes time. Fine for a research assistant; disqualifying for anything interactive. No leaderboard shows you the p95.
Whether you need it at all. Most production LLM traffic is extraction, summarization, classification, routine codegen. None of it resembles GPQA Diamond. Paying a 4x premium for graduate-level physics reasoning on "categorize this support ticket" is how inference bills quietly triple. As the analysis linked above notes, GPQA measures exactly one capability — formal scientific reasoning — and extrapolating a narrow win to overall superiority is a category error.
There's also an access asterisk: at launch, Deep Think is limited to Google AI Ultra subscribers via an app toggle with daily prompt caps. API access is promised in "coming weeks," so you can't put it in production today even if you want to.
The practical takeaway
Deep Think is best understood not as "Gemini got smarter" but as a new price/performance point on Google's curve — a very high tier that trades money and time for reasoning depth. That framing suggests the right response: don't ask "is this the best model?", ask "which of my requests justify this tier?" For most teams the honest answer is a small single-digit percentage of traffic — and routing that slice to a Deep Think-class tier while everything else runs on cheaper models is exactly the kind of split that tier-based routing (our whole thing at TierUp) exists to make boring.
When API pricing lands, redo the math with real numbers. Until then, the 82.4% is impressive, and it's still only half a chart.
