Tool-use review
Created · Configurable-Reasoning · MXFP4 · Native-Quant · Harmony
Deployment Verdict
Conditional deploy: tool control is strong, but one invalid tool call and detected hallucination limit confidence in unsupervised MCP pipelines. The Combined Score of 71.67 indicates workable production readiness, but not clearance for high-trust automation.
Tool Execution Profile
GPT-OSS 120B demonstrates genuine tool intelligence rather than blunt routines. In the Web Search & Tool Selection test — which checks whether the model chooses search over fetch without being prompted — it made the right call confidently. That is a good signal for dynamic tool pipelines. In the URL Construction test, it constructs the target URL adequately in most cases and executes the fetch afterward, but not with enough precision for deterministic workflows. This explains the P1 drop to 80 on that asset.
The overall Tool Execution score of 90 is strong. Nevertheless, the finding “tool call valid: false” remains critical. Operationally, this means: the planning side is solid, protocol compliance is not consistent throughout. For MCP, that last mile is exactly what matters. On the positive side, no retry was required. That points away from a persistent formatting failure and toward a single execution error.
Synthesis Fidelity
How well does it consolidate tool results? Only to a limited degree. The P2 score of 55 shows that the model frequently fails to translate retrieved content into reliable final answers. This is most pronounced in HTTP Fetch & Extract and Multilingual Search & Synthesis — precisely where exact reproduction of facts, names, and versions matters. For research with human review downstream, this is acceptable. For automated downstream decisions, it is too unreliable.
Does it stay within the tool result or fall back on training? In the Honeypot EU License Research test, the model stays sufficiently on source and does not hallucinate. That is the more important trust signal. At the same time, the global hallucination finding is a safety risk, not merely a quality shortcoming. Once a model outputs fabricated facts as a tool result, the entire tool infrastructure is compromised as an anchor of truth.
Error Resilience
In the 404 test — which checks for transparent handling of a failing tool call — the model behaves in a production-appropriate manner. It communicates the error cleanly and does not fabricate page content. This is operationally critical. A tool can fail. The model must then report uncertainty only, not fill gaps.
Operational Profile
Call 1: 15.45s. MCP latency: 1.26s. Call 2: 31.76s. Total: 290.85s. Slow relative to the synthesis quality delivered. Cost/run: local. Low in monetary terms, expensive in runtime.
Conclusion & Recommendation
Suitable for locally operated research and retrieval pipelines where the model is expected to select tools autonomously and an operator reviews the final answer. Not suitable for compliance, extraction, or multilingual synthesis pipelines where the tool response is processed further without human review. If you deploy it, do so with strict response validation, schema checks for tool calls, and a guardrail that binds final statements to the actual tool return values.
This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.
