Devstral 2 2512 vs GPT-5 Mini

Summary of wins in our testing: GPT-5 Mini wins 5 benchmarks (strategic_analysis 5 vs 4, faithfulness 5 vs 4, classification 4 vs 3, safety_calibration 3 vs 1, persona_consistency 5 vs 4). Devstral 2 2512 wins 2 benchmarks (constrained_rewriting 5 vs 4, tool_calling 4 vs 3). The remaining five tests tie (structured_output 5/5, creative_problem_solving 4/4, long_context 5/5, agentic_planning 4/4, multilingual 5/5). Detailed context and ranks (all scores are our 1–5 internal tests):

• Constrained rewriting: Devstral 2 2512 = 5, GPT-5 Mini = 4. In our testing Devstral is tied for 1st in constrained rewriting ("tied for 1st with 4 other models"), while GPT-5 Mini ranks 6th of 53. This matters when you must compress or strictly meet character/format limits.
• Tool calling: Devstral 2 2512 = 4, GPT-5 Mini = 3. Devstral ranks 18 of 54 (many models share scores) vs GPT-5 Mini at 47 of 54 — Devstral makes more accurate function selection and argument sequencing in our tool-calling tasks.
• Strategic analysis: GPT-5 Mini = 5, Devstral 2 2512 = 4. GPT-5 Mini is tied for 1st on strategic analysis, so it handles nuanced tradeoff reasoning with numeric detail better in our tests.
• Faithfulness: GPT-5 Mini = 5, Devstral 2 2512 = 4. GPT-5 Mini is tied for 1st for faithfulness in our ranking; expect fewer source hallucinations on factual summarization tasks.
• Classification: GPT-5 Mini = 4, Devstral 2 2512 = 3. GPT-5 Mini ranks tied for 1st on classification—better routing/labeling in our classification tasks.
• Safety calibration: GPT-5 Mini = 3, Devstral 2 2512 = 1. GPT-5 Mini ranks 10 of 55 vs Devstral at 32, so GPT-5 Mini more reliably refuses harmful prompts while permitting legitimate ones in our tests.
• Persona consistency, long-context, structured-output, creative problem solving, agentic planning, multilingual: ties — both models scored equally (examples: structured_output 5/5, long_context 5/5). Both tied models rank at or near the top for long-context, structured output, and multilingual in our rankings. External benchmarks: GPT-5 Mini also has third-party scores — 64.7% on SWE-bench Verified, 97.8% on MATH Level 5, and 86.7% on AIME 2025 (all reported by Epoch AI). Devstral 2 2512 has no external benchmark entries in the payload. Use these external figures as supplementary evidence when comparing coding/math performance.

The main price gap is input tokens: Devstral 2 2512 charges $0.40 per mTok input vs GPT-5 Mini at $0.25 per mTok; both charge $2.00 per mTok output. The input-only delta is $0.15 per mTok. At 1M tokens/month (1,000 mTok) that’s $150 more for Devstral; at 10M tokens (10,000 mTok) it’s $1,500 more; at 100M tokens (100,000 mTok) it’s $15,000 more. Teams that stream large volumes of prompts (embedded search, heavy user inputs, analytics pipelines) should care about this gap. Small-scale projects or those dominated by output tokens will see a smaller relative impact because output pricing is identical ($2.00/mTok).

Choose Devstral 2 2512 if you need stronger tool-calling and the best constrained-rewrite performance (e.g., agentic coding workflows, strict-format outputs) and can accept higher input costs ($0.40/mTok). Choose GPT-5 Mini if you need safer, more faithful outputs, stronger classification and strategic analysis in production (wins 5 of 12 benchmarks in our tests), or if input-cost savings ($0.25/mTok) matter at scale.