Codestral 2508 vs Gemini 2.5 Pro

Overview: across our 12-test suite Gemini 2.5 Pro wins five benchmarks, Codestral 2508 wins none, and seven benchmarks are ties. Details by test (scoreA = Codestral, scoreB = Gemini):

• Faithfulness: tie (5 vs 5). Both models rank highly; Codestral’s faithfulness is tied for 1st of 55 models (tied with 32 others) and Gemini shares that top rank as well — good for tasks that must stick to source text.
• Persona consistency: Gemini wins (3 vs 5). Gemini ties for 1st in persona consistency (tied with 36 others out of 53) while Codestral ranks 45 of 53 — Gemini holds the edge for sustained character or persona-driven chat.
• Constrained rewriting: tie (3 vs 3). Both models score equally; neither pulls ahead for aggressive compression or hard-character-limit rewriting.
• Strategic analysis: Gemini wins (2 vs 4). Gemini ranks 27 of 54 on strategic analysis vs Codestral’s 44 of 54 — practical effect: Gemini produces stronger multi-step tradeoff reasoning with numbers.
• Creative problem solving: Gemini wins (2 vs 5). Gemini is tied for 1st on creative problem solving (tied with 7 others), so it generates more non-obvious, feasible ideas in our tests.
• Structured output: tie (5 vs 5). Both are tied for 1st (tied with 24 others out of 54) — reliable JSON/schema compliance for both models.
• Long context: tie (5 vs 5). Both tied for 1st on retrieval at 30K+ tokens; Codestral’s context window is 256,000 vs Gemini’s 1,048,576, so Gemini supports larger files but both score top in our long-context retrieval tests.
• Multilingual: Gemini wins (4 vs 5). Gemini is tied for 1st in multilingual (tied with 34 others); choose Gemini when parity across many languages matters.
• Tool calling: tie (5 vs 5). Both tied for 1st (tied with 16 others) — strong for function selection and argument accuracy in integrations.
• Classification: Gemini wins (3 vs 4). Gemini is tied for 1st in classification (tied with 29 others), making it more reliable for routing and labeling tasks in our suite.
• Safety calibration: tie (1 vs 1). Both scored poorly on safety calibration in our tests (rank 32 of 55), so expect similar refusal/permissiveness behavior and plan guardrails accordingly.
• Agentic planning: tie (4 vs 4). Both models scored the same and share rank 16 of 54 — comparable decomposition and failure recovery abilities. External benchmarks (supplementary): Gemini 2.5 Pro scores 57.6% on SWE-bench Verified and ranks 10 of 12 on that external coding benchmark, and it scores 84.2% on AIME 2025 (rank 11 of 23), according to Epoch AI. Codestral has no external SWE-bench or AIME scores in the payload; these Epoch AI results support Gemini’s strength on third-party coding/math tests but do not replace our 12-test internal signal.

Costs per million tokens (input+output): Codestral 2508 = $0.3 + $0.9 = $1.20/M; Gemini 2.5 Pro = $1.25 + $10 = $11.25/M. At 1M tokens/month that’s $1.20 vs $11.25; at 10M it’s $12 vs $112.50; at 100M it’s $120 vs $1,125. The ~9x priceRatio in the payload (0.09) means large-scale labeling, CI/test generation, or high-throughput code-completion pipelines will see meaningful savings with Codestral. Teams running low-volume, high-value reasoning, multimodal research, or tasks where Gemini’s unique strengths matter should budget for the higher monthly spend on Gemini.

Choose Codestral 2508 if: you need a cost-efficient, production-grade coding model with top tool-calling, long-context handling and faithfulness for high-throughput completion, test generation, or CI tasks — especially when budget matters (≈ $1.20/M tokens). Choose Gemini 2.5 Pro if: your priority is stronger strategic reasoning, creative problem solving, classification, persona consistency and multilingual performance, or you require multimodal inputs and a 1,048,576-token context window — accept the higher cost (≈ $11.25/M tokens) for those capabilities.