Skip to content
CrucibleMark

o3-mini

o3-mini is OpenAI’s compact reasoning model with internal chain-of-thought, specialized in math, coding, and STEM tasks. The model operates with a context window of 200,000 tokens and offers three adjustable reasoning levels for balancing response depth, latency, and cost. Available exclusively via the OpenAI API.

OpenAI Version 2025-01-31 Commercial use permitted Dense 200 K Context 10/2023 $1.1 / $4.4 per 1M

  • Proprietary
  • Frontier
  • API
  • Text
  • Instruction-Tuned
  • Real-Time

Sovereign Risk: MEDIUM OpenAI is a US-based company and subject to the CLOUD Act. Data transmitted via the API may be made accessible to US authorities. Local deployment is not possible.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 60 76
HTTP Fetch & Extract 80 60 70
Tool Failure Handling (404) 80 80 80
Web Search & Tool Selection 100 35 70.8
URL Construction & Fetch 80 60 70
Multilingual Search & Synthesis 100 40 70

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: Yes
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
2.19
First Request
MCP
1.51
Protocol Latency
Synthesis
7.51
Response Generation
Total
67.24
Sum of All Phases
Token
12158
Input + Output
Cost
$0.0379
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

· Instruction-Tuned

Deployment Verdict

Created on: 14.06.2026, 16:16:30

Conditional deploy, because tool execution is reliable and calls are valid, but synthesis with detected hallucination remains insufficiently robust throughout for uncontrolled output paths.

Tool Execution Profile

o3-mini can fundamentally be trusted with tools in an MCP-backed pipeline. Tool calls were valid, retry was not necessary, and with P1 90 the model demonstrates clear operational strength. Particularly notable: in the Web Search and Tool Selection test — which checks whether the model chooses search over fetch without being prompted — it made the right tool choice cleanly. This argues against a rigid pattern and in favor of genuine situational decision-making. In the URL Construction test, which measures correct derivation of a target URL from knowledge followed by the fetch, it was serviceable but not deterministic enough for fragile paths. The model therefore usually recognizes which tool it needs, but is less precise in exact address construction than in tool selection itself.

Synthesis Fidelity

How well does it condense tool results? Only moderately. P2 55.83 is the weak part of this profile. The model retrieves information better than it summarizes it consistently and closely against the findings. This is also visible in Web Search & Tool Selection and Multilingual Search & Synthesis, where tool usage remains strong but condensation drops off noticeably. For pipelines with downstream validation this is tolerable. For direct user responses based on tool output it is too loose.

Does it stay within the tool result or fall back on training? In the Honeypot EU License Research test — which probes exactly this temptation with current license restrictions — it stays sufficiently anchored to the web source. Content Verification State A and no hallucination in this test are a good trust signal. At the same time, the global hallucination finding remains a security risk: once a model outputs fabricated facts as a tool result, it damages trust in the entire infrastructure.

Error Resilience

In the 404 test, which checks for transparent handling of a failing tool call, o3-mini responded in a production-appropriate manner. It did not hallucinate page content and communicated the error openly. This is critical for real tool chains, because failures in retrieval or fetch steps are to be expected.

Operational Profile

Total 67.24s per run. Tool call latencies 2.19s and 7.51s, MCP latency 1.51s. On the slower side. Cost 0.037873 per run. Inexpensive to moderate for a thinking model — better value relative to execution strength than to synthesis quality.

Conclusion & Recommendation

Suitable for research-oriented, multi-step pipelines with clear guardrails, structured verification, and separate answer polishing. Well suited for tool routing, web research, and fault-tolerant orchestration. Not the first choice for compliance-adjacent or customer-facing final responses when the model itself is expected to deliver the final condensation. Deploy when a second control step safeguards the synthesis. Without that step, do not use as the last instance.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.