Devstral Medium vs Llama 3.3 70B Instruct

Across our 12-test benchmark suite, Llama 3.3 70B Instruct wins 5 categories, Devstral Medium wins 1, and the two tie on 6.

Where Llama 3.3 70B Instruct wins:

• Long context (5 vs 4): Llama ties for 1st among 55 tested models; Devstral ranks 38th. At 30K+ token retrieval tasks, this is a meaningful gap for document-heavy applications.
• Tool calling (4 vs 3): Llama ranks 18th of 54; Devstral ranks 47th. Function selection, argument accuracy, and sequencing — core to agentic and API-integrated workflows — are substantially stronger on Llama.
• Safety calibration (2 vs 1): Llama ranks 12th of 55; Devstral ranks 32nd. Devstral's score of 1 sits at the bottom quartile of all models we've tested (p25 = 1), meaning it more frequently fails to refuse harmful requests or over-refuses legitimate ones.
• Creative problem solving (3 vs 2): Llama ranks 30th of 54; Devstral ranks 47th. For generating non-obvious, feasible ideas, Llama is the stronger choice.
• Strategic analysis (3 vs 2): Llama ranks 36th of 54; Devstral ranks 44th. Nuanced tradeoff reasoning with real numbers favors Llama.

Where Devstral Medium wins:

• Agentic planning (4 vs 3): Devstral ranks 16th of 54; Llama ranks 42nd. Goal decomposition and failure recovery — the backbone of autonomous agent loops — is where Devstral earns its keep. This is its clearest differentiator.

Ties (6 categories): Both models score identically on structured output (4), constrained rewriting (3), faithfulness (4), classification (4), persona consistency (3), and multilingual (4). Neither has an edge in these areas.

External benchmarks: Llama 3.3 70B Instruct has third-party scores available from Epoch AI: 41.6% on MATH Level 5 (ranking last — 14th of 14 — among models with this score in our dataset) and 5.1% on AIME 2025 (ranking last — 23rd of 23). These scores place it at the bottom of math-capable models in our dataset. No external benchmark data is available for Devstral Medium in the payload. Neither model should be the first choice for competition-level math.

Devstral Medium costs $0.40/M input and $2.00/M output tokens. Llama 3.3 70B Instruct costs $0.10/M input and $0.32/M output tokens — 4x cheaper on input and 6.25x cheaper on output. At 1M output tokens/month, that's $2.00 vs $0.32 — a $1.68 difference that barely registers. At 10M output tokens/month, the gap grows to $16.80 ($20.00 vs $3.20). At 100M output tokens/month — a realistic scale for production APIs — you're looking at $200.00 vs $32.00, saving $168.00 monthly by choosing Llama 3.3 70B Instruct. For high-volume applications where Devstral Medium's agentic planning advantage (4 vs 3) isn't mission-critical, that cost gap is hard to justify. Developers self-hosting or running inference at scale will feel the price delta most acutely.

Choose Llama 3.3 70B Instruct if you need a cost-efficient general-purpose model: it wins on tool calling (4 vs 3), long-context retrieval (5 vs 4), strategic analysis (3 vs 2), creative problem solving (3 vs 2), and safety calibration (2 vs 1), all at $0.32/M output tokens. It's the right call for document analysis, multi-tool API agents, safety-sensitive deployments, and any workload where you're processing tens of millions of tokens per month.

Choose Devstral Medium if agentic planning is your primary workload — specifically goal decomposition and failure recovery in autonomous agent loops, where it scores 4 vs Llama's 3 and ranks 16th of 54 models. That advantage comes at a steep premium ($2.00/M output vs $0.32/M), so it only makes sense if agentic planning quality directly impacts your application outcomes and your volume is low enough that the 6.25x cost multiplier is acceptable.