Skip to content
CrucibleMark

GPT-OSS 120B

GPT-OSS 120B is OpenAI’s largest Open Weights model as of August 2025, released under the Apache 2.0 license with free commercial use. The MoE architecture bundles 116.8 billion total parameters while activating only 5.1 billion per token, and runs on a single high-memory GPU thanks to native MXFP4 quantization. Three reasoning levels and native tool use in the Harmony format round out the profile.

OpenAI Version 1.0 Commercial use permitted MoE 116.8 B (5.1 B active) 131 K Context locally tested

  • Open Weights
  • Server
  • VSPK
  • Text
  • Configurable-Reasoning
  • MXFP4
  • Native-Quant
  • Harmony
  • Interactive

Sovereign Risk: LOW OpenAI is a US company; the model is released as Open Weights under Apache-2.0. Local deployment completely eliminates any API data transfer to OpenAI servers, which is why the risk is rated as low despite US jurisdiction (CLOUD Act).

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 15 49
HTTP Fetch & Extract 80 60 70
Tool Failure Handling (404) 80 20 59
Web Search & Tool Selection 100 35 70.8
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 35 67.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
8.05
First Request
MCP
0.93
Protocol Latency
Synthesis
40
Response Generation
Total
293.89
Sum of All Phases
Token
14909
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Configurable-Reasoning · MXFP4 · Native-Quant · Harmony

Deployment Verdict

Conditional deploy: The model executes tools with clear intent in many cases, but is not cleared for trust-critical MCP pipelines due to detected hallucination and inconsistent Tool-Call validity.

Tool Execution Profile

Tool usage shows genuine selection capability, but no consistently clean protocol discipline. In the Web Search & Tool Selection test — which checks the choice between search and direct retrieval without an explicit hint — the model reliably picks the right tool. That argues against a rigid pattern. It also reaches for the operationally correct tools in Multilingual Search & Synthesis and EU License Research.

The execution layer is weaker. Tool-Call valid: false matters in MCP environments because even isolated form or parameter errors can throw orchestrators off track. In the URL Construction test, which checks independent derivation of the target URL followed by a fetch, performance is usable but not deterministic. The model thus demonstrates tool intelligence, but not reliable call precision across all cases.

Synthesis Fidelity

How well does it distill tool results? Only to a limited degree. The P2 score of 45 reveals a clear pattern: raw retrieval is stronger than post-processing. In HTTP Fetch & Extract the model synthesizes solidly, and likewise in the URL Construction test. In several research-heavy tasks, however, it drifts from extraction into free reconstruction. For pipelines that require exact summaries from tool output, that is too imprecise.

Does it stay within the tool result or fall back on training? No — and that is the core risk. In the honeypot EU License Research test, which checks whether current license restrictions are answered from web sources rather than training knowledge, the model hallucinates. This is not a mere quality deficiency; it is a security issue. When a model presents invented or pre-learned facts as the result of a tool-based lookup, the entire tool infrastructure loses its trustworthiness.

Error Resilience

In the 404 test, which checks for transparent behavior when a tool call fails, the model does not fabricate page content. That is the correct production reflex. The contextual framing remains thin, but it holds the critical line: surface the error rather than generate substitute facts. For robust orchestration, that is acceptable.

Operational Profile

Call 1: 8.05s. Call 2: 40.00s. MCP latency: 0.93s. Total: 293.89s. Slow relative to the overall confidence delivered. Cost/run: local. Inexpensive to operate, but the time costs are high.

Summary & Recommendation

The model is suitable for internal tool pipelines with human review — such as research pre-stages, URL resolution, search-to-fetch routing, and non-binding summaries. It is not suitable for compliance, license verification, regulatory information, incident analysis, or any pipeline in which tool results are passed on as reliable facts. Anyone deploying it should enforce hard validation of tool calls, source gating, and a downstream response check.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.