Skip to content
CrucibleMark

GPT-OSS 120B (Thinking)

GPT-OSS 120B has been OpenAI’s largest Open Weights model since August 2025, released under the Apache 2.0 license with free commercial use. The MoE bundles 116.8 billion total parameters at only 5.1 billion active parameters per token and runs on a single high-memory GPU thanks to native MXFP4 quantization. Three reasoning levels and native tool use in the Harmony format round out the profile.

OpenAI Version 1.0 Commercial use permitted MoE 116.8 B (5.1 B active) 131 K Context locally tested

  • Open Weights
  • Server
  • vLLM
  • Text
  • Native Quantisierung
  • Harmony-Format
  • Interactive

Sovereign Risk: LOW OpenAI is a US company; the model is released as Open Weights under Apache-2.0. Local deployment completely eliminates any API data leakage to OpenAI servers, which is why the risk is rated as low despite US jurisdiction (CLOUD Act).

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 15 49
HTTP Fetch & Extract 80 60 70
Tool Failure Handling (404) 80 100 87
Web Search & Tool Selection 100 80 91
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 60 80

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
12.18
First Request
MCP
1.06
Protocol Latency
Synthesis
35.38
Response Generation
Total
291.68
Sum of All Phases
Token
15002
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Native Quantisierung · Harmony-Format

Deployment Verdict

Conditional deploy: The model executes tools competently in most cases, but the detected hallucination in the honeypot and the invalid tool call undermine trust for unsupervised production pipelines.

Tool Execution Profile

GPT-OSS 120B demonstrates genuine tool intelligence, not merely rigid call patterns. In the Web Search & Tool Selection test — which checks the choice between search and direct retrieval without hinting — it reliably identifies the need for web_search. That is a strong signal for dynamic MCP pipelines. In the URL Construction test, which requires deriving the correct target URL from internal knowledge and then executing fetch, it remains usable but not deterministic enough. P1 80 is sufficient for that, but not reassuring.

Across tasks, the operational hit rate is high. EU License Research and Multilingual Search & Synthesis both achieve full tool execution. Nevertheless, the overall picture is not clean, because at least one tool call was formally invalid. Since no retry was needed, this looks less like a recurring formatting issue and more like a one-off protocol error. For MCP, that means: orchestration-capable, but not blindly trustworthy.

Synthesis Fidelity

How well does it consolidate tool results? Only partially. P2 55 indicates that the model often produces usable summaries, but not with the consistency required for reliable final outputs. The pattern is uneven: Tool Failure Handling (404) is exemplary at P2 100, HTTP Fetch & Extract is merely average at P2 60, and EU License Research drops sharply at P2 15.

Does it stay within the tool result or fall back on training data? No, not reliably. In the honeypot EU License Research task — which is designed to force current license restrictions from web sources — the model hallucinates instead of adhering to the retrieved content. This is not merely a quality shortcoming; it is a security risk. When a model outputs fabricated facts as the result of a tool pipeline, the entire infrastructure loses its audit trail.

Error Resilience

Here the model is production-ready. In the 404 test, which checks for transparent handling of a failed tool call, it communicates the error correctly and does not fabricate page content. This is precisely the behavior required in production pipelines: a visible failure rather than plausible misinformation.

Operational Profile

Call 1: 12.18s. Call 2: 35.38s. MCP latency: 1.06s. Total: 291.68s.
Slow relative to the overall quality delivered.
Cost/run: local. Economical to operate, but time-intensive.

Conclusion & Recommendation

Suitable for local MCP setups with human review, error gates, and downstream fact-checking. Particularly useful where tool selection matters more than final synthesis. Not suitable for compliance, policy, licensing, or other high-trust pipelines where the model must strictly preserve tool results. Anyone deploying it should verify responses against raw tool output and enforce formal tool call validation.

This analysis was generated automatically based on benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.