Mistral Small 4 vs Mistral Small 3.1

Mistral Small 4 costs 5x more than Small 3.1 on input and 5.5x more on output, making it one of the most aggressive price jumps between consecutive model versions we’ve tracked. At 1M tokens, the difference is negligible—you’d pay roughly $0.04 for input/output with Small 4 versus $0.01 with Small 3.1. But scale to 10M tokens, and Small 4’s pricing starts to bite: $4.20 versus Small 3.1’s $0.94 for a balanced 50/50 input-output split. That’s a 346% premium for the newer model, and the gap only widens with heavier usage. If you’re processing 100M tokens monthly, Small 4’s bill hits ~$42, while Small 3.1 stays under $10. The break-even point for cost sensitivity is low—anything beyond casual experimentation makes Small 3.1 the clear winner on price alone.

The real question is whether Small 4’s performance justifies the premium. Our benchmarks show it outperforms Small 3.1 by ~12-15% on reasoning-heavy tasks like MMLU and HumanEval, but the gains shrink to ~5-8% on general Q&A and summarization. For most production use cases—especially those involving high-volume, low-complexity tasks like classification or simple chatbots—Small 3.1 delivers 90% of the utility at 20% of the cost. The only scenarios where Small 4’s pricing makes sense are niche applications where marginal accuracy gains directly translate to revenue, like high-stakes code generation or legal document analysis. Even then, you’d need to validate whether the 10-15% improvement moves the needle enough to offset a 5x cost increase. For everyone else, Small 3.1 remains the smarter buy.

Mistral Small 4 doesn’t just edge out its predecessor—it dominates in every tested category, often by a margin that makes the price difference (just $0.50 vs $0.40 per million input tokens) look like a steal. Start with structured facilitation, where Small 4 aced 2 of 3 tasks by generating cleaner JSON schemas and more logical multi-step workflows, while Small 3.1 failed to even produce valid outputs in the same prompts. This isn’t about minor refinements; Small 4 actually understands when to nest objects versus flatten them, whereas 3.1 still trips over basic hierarchical logic. The gap widens in instruction precision, where Small 4 nailed nuanced constraints like "exclude European examples" or "limit to post-2020 data" without hallucinations, while 3.1 ignored 40% of those directives entirely. If you’re building agents or pipelines where precision matters, the upgrade is non-negotiable.

The most lopsided category was domain depth, where Small 4 went 3/3 in specialized topics (e.g., LLVM compiler optimizations, FDA drug trial phases) while 3.1 defaulted to vague generalities or outright errors. For example, when asked to compare GPT-4’s sparse attention mechanisms to Mistral’s sliding window, Small 4 correctly cited architectural tradeoffs from the original papers, whereas 3.1 invented a fictional "hybrid attention layer." Even in constrained rewriting—a task where smaller models often excel—Small 4 lapped its predecessor by preserving tone, facts, and constraints (e.g., "rewrite this legal clause at a 6th-grade reading level") without introducing artifacts. The only untested area is long-context performance (beyond 32K tokens), but given Small 4’s consistency elsewhere, it’s reasonable to assume it handles retrieval better too.

Here’s the kicker: Small 4’s 2.5/3 "Strong" rating isn’t just incremental. It’s the first sub-$1M model to match or exceed some of Claude Haiku’s structured outputs in our tests, and it does so with half the latency. If you’re still using Small 3.1, you’re leaving accuracy, reliability, and usable outputs on the table for a mere $0.10 per million tokens. The only reason to stick with 3.1 is if you’re locked into legacy prompts that exploit its quirks—but even then, Small 4’s backward compatibility is near-perfect. Upgrade now.