Claude Haiku 4.5 vs GPT-5 Mini

Test-by-test summary (our 12-test suite):

• Wins for Claude Haiku 4.5: tool_calling 5 vs GPT-5 Mini 3 (Haiku tied for 1st of 54; GPT-5 Mini ranks 47/54). Expect more reliable function selection, argument accuracy, and sequencing from Haiku in our tests. Agentic_planning: Haiku 5 vs GPT-5 Mini 4 (Haiku tied for 1st; GPT-5 Mini rank 16/54) — Haiku better at goal decomposition and recovery.
• Wins for GPT-5 Mini: structured_output 5 vs Haiku 4 (GPT-5 Mini tied for 1st of 54; Haiku rank 26/54) — GPT-5 Mini is stronger at JSON/schema compliance for API-driven outputs. constrained_rewriting 4 vs Haiku 3 (GPT-5 Mini rank 6/53; Haiku rank 31/53) — GPT-5 Mini handles tight character compression better. safety_calibration 3 vs Haiku 2 (GPT-5 Mini rank 10/55; Haiku rank 12/55) — GPT-5 Mini shows a small edge in refusing harmful requests while permitting legitimate ones.
• Ties: strategic_analysis, creative_problem_solving, faithfulness, classification, long_context, persona_consistency, multilingual — both models score identically on these tests in our suite (e.g., strategic_analysis 5 tied for 1st; long_context 5 tied for 1st), so expect comparable behavior on nuanced reasoning, long-context retrieval (30K+), multilingual output, and faithfulness.
• External benchmarks (Epoch AI): GPT-5 Mini scores 64.7% on SWE-bench Verified, 97.8% on MATH Level 5, and 86.7% on AIME 2025 — these third-party results back GPT-5 Mini’s strength on coding/math tasks. Claude Haiku 4.5 has no external scores in this payload. Overall: Haiku’s standout in tool-calling and agentic workflows; GPT-5 Mini’s strengths are structured-output, constrained-rewriting, safety calibration, and strong external math/coding measures.

Raw per-million-token rates: Claude Haiku 4.5 charges $1 input / $5 output per mTok; GPT-5 Mini charges $0.25 input / $2 output per mTok (priceRatio 2.5). With a 50/50 input-output token split, effective cost per million tokens is about $3.00 for Claude Haiku 4.5 vs $1.125 for GPT-5 Mini. At that split, monthly costs are: 1M tokens = $3.00 vs $1.13; 10M = $30 vs $11.25; 100M = $300 vs $112.50. In an output-heavy workload (all output tokens): 1M = $5 vs $2; 100M = $500 vs $200. Teams with high volume (10M+/month), narrow margins, or consumer-facing pricing should care most about GPT-5 Mini’s cost advantage; teams that prioritize tool orchestration and agentic workflows may justify Haiku’s higher per-token price.

Choose Claude Haiku 4.5 if: you need best-in-class tool calling and agentic planning (Haiku scores 5 on tool_calling and agentic_planning and ranks tied for 1st), you require ultra-low latency or the particular Haiku behavior described, and you can absorb the higher token cost. Choose GPT-5 Mini if: you need reliable JSON/schema output, constrained rewriting, better safety calibration, lower per-token costs (input $0.25 / output $2 vs Haiku $1 / $5), or stronger third-party math/coding signals (SWE-bench 64.7%, MATH Level 5 97.8%, AIME 2025 86.7 per Epoch AI).