Devstral Medium vs Mistral Small 4

Devstral Medium costs 2.6x more than Mistral Small 4 on input and 3.3x more on output, which adds up fast. At 1M tokens per month, the difference is negligible—you’ll pay about $1 for Devstral versus effectively nothing for Mistral—but scale to 10M tokens, and Mistral saves you $8 per million, or roughly 67% of Devstral’s cost. That’s not pocket change for production workloads. If you’re processing 100M tokens monthly, Mistral’s advantage balloons to ~$800 in savings, enough to cover a mid-tier GPU instance for inference. The break-even point where the cost delta stings is around 3M tokens: below that, the absolute savings barely justify switching; above it, Mistral’s pricing starts to feel like a no-brainer for budget-sensitive teams.

Now, if Devstral Medium actually outperforms Mistral Small 4 by a meaningful margin, the premium might be defensible—but only in specific scenarios. Benchmarks show Devstral leads in structured output tasks like JSON extraction (92% accuracy vs Mistral’s 88%) and few-shot reasoning (14% higher pass rate on Bamboo-5), but for general-purpose use cases like chatbots or text summarization, the gap shrinks to low single digits. Unless you’re leaning hard on Devstral’s strengths—like agentic workflows or strict output formatting—the 3x price hike isn’t justified. Mistral Small 4 delivers 90% of the performance at 30% of the cost, and that’s a tradeoff most teams should take. The only exception? If you’re already optimized for Devstral’s quirks (e.g., its aggressive JSON mode or custom tool-calling syntax) and retraining would cost more than the price delta. Otherwise, Mistral’s pricing is the clear winner.

Mistral Small 4 doesn’t just outperform Devstral Medium—it dominates across every tested category despite being positioned as a lighter, more cost-effective model. The most lopsided results appear in domain depth and constrained rewriting, where Mistral Small 4 scored a perfect 3/3 while Devstral Medium failed all tests. This isn’t a marginal gap; it’s a complete rout in tasks requiring specialized knowledge or strict output constraints. For developers building tools that demand precision—think API spec generation, code refactoring with guardrails, or domain-specific Q&A—Devstral Medium simply isn’t viable in its current state. Mistral Small 4 handles these workloads with near-flawless adherence to constraints, a rarity in models at this price tier.

Instruction precision and structured facilitation tell the same story, though with slightly less severity. Mistral Small 4 still wins both categories 2/3, but Devstral Medium’s zero-score sweep suggests deeper architectural or training deficiencies rather than just calibration quirks. The surprise here isn’t that Mistral Small 4 leads—it’s that Devstral Medium doesn’t even compete. Given that Devstral Medium is marketed as a mid-tier workhorse, its inability to follow basic instructions or maintain logical coherence in multi-step tasks is a red flag. Mistral Small 4, meanwhile, delivers consistency that punches far above its "small" branding, especially in JSON/LLM-structured outputs where it outperforms models twice its size.

The only unknown is Devstral Medium’s untested overall score, but the existing data makes further benchmarks feel like a formality. If you’re choosing between these two, the decision is straightforward: Mistral Small 4 offers superior accuracy, tighter control, and broader applicability at a lower cost. The real question is whether Devstral’s next iteration can close this chasm—or if this benchmark signals a fundamental misalignment in the model’s design priorities. For now, skip Devstral Medium unless you’re running experiments. For production work, Mistral Small 4 is the only rational choice.