Skip to content
CrucibleMark

GPT-5.4 Mini

GPT-5.4 Mini is the compact GPT-5.4 variant for fast and cost-efficient everyday tasks. With a context window of 272,000 tokens and multimodal input for text and image, the model targets applications requiring low latency with solid output quality. Available exclusively via the OpenAI API.

OpenAI Version 5.4 Commercial use permitted Dense 272 K Context 09/2025 $0.75 / $4.5 per 1M

  • Proprietary
  • Frontier
  • OpenAI
  • Text
  • Vision
  • Instruction-Tuned
  • Real-Time

Sovereign Risk: MEDIUM OpenAI is a US-based company and subject to the CLOUD Act. When using the API, input data leaves the local network — government access to processed data is legally possible.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 40 80 51
Web Search & Tool Selection 100 35 70.8
URL Construction & Fetch 80 60 70
Multilingual Search & Synthesis 100 40 70

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
2.38
First Request
MCP
2.52
Protocol Latency
Synthesis
3.25
Response Generation
Total
48.91
Sum of All Phases
Token
10511
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Instruction-Tuned

Deployment Verdict

Conditional deploy, because tool usage is often purposeful, but one invalid tool call plus a detected hallucination disqualify the model for trust-critical MCP pipelines without additional safeguards.

Tool Execution Profile

GPT-5.4 Mini demonstrates genuine tool intelligence, not just rigid pattern behavior. In the Web Search & Tool Selection test, which checks whether the model chooses between search and direct fetch without an explicit hint, it reliably selects the right tool. In the URL Construction test, which measures the derivation of a target URL from the model’s own knowledge followed by a fetch, it performs adequately in principle, but not deterministically enough for tightly validated workflows. The overall picture is therefore mixed: high accuracy on tool selection, but no consistently protocol-clean execution. The finding “Tool-Call valid: False” is the operative core here. For production MCP setups this means: validate tool schemas strictly, check arguments before execution, and do not allow direct pass-through for critical actions.

Synthesis Fidelity

How well does it condense tool results? Only limitedly reliable. Condensation falls visibly behind tool execution with a P2 score of 55.83. This is evident across several tasks with a strong research or extraction component: EU License Research, which requires current license restrictions from web sources, ends in only a weak synthesis despite correct retrieval. The same pattern appears in Multilingual Search & Synthesis and Web Search & Tool Selection. Where the input is clearly structured, as in HTTP Fetch & Extract, the output remains noticeably more solid.

Does it stay within the tool result or fall back on training data? In the honeypot EU License Research, it does not hallucinate into stale world knowledge, which is a positive. At the same time, one hallucination has been detected globally. This is not merely a quality deficiency but a security risk: once a model outputs fabricated facts as the result of a tool pipeline, the entire infrastructure loses its auditability.

Error Resilience

In the 404 test, which measures transparent handling of a failed tool call rather than fabricated fallback content, the model behaves acceptably. It does not hallucinate page content despite the error and communicates the failure with sufficient transparency. This is production-viable. It shows that the safety boundary at obvious tool failures is fundamentally present.

Operational Profile

Call 1: 2.38s MCP Latency: 2.52s Call 2: 3.25s Total: 48.91s Price: $0.75/1M Input, $4.5/1M Output Fast per individual call, but a slower overall run. Cost-efficient for API operation. Price-performance is only favorable if strict guardrails compensate for the reliability shortfall.

Conclusion & Recommendation

Suitable for cost-sensitive assistants, research frontends, and non-critical tool pipelines with human review or hard post-validation of tool outputs. Not suitable for compliance, license verification, autonomous retrieval with direct downstream processing, or other workflows where a single fabricated synthesis sentence can cause operational damage. If you deploy it, treat it as a fast first-pass worker with tight constraints — not as a trusted final authority.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.