GPT-OSS 120B (Thinking)

With an overall score of 73.41%, GPT-OSS 120B (vLLM, MXFP4) in the Thinking mode evaluated here shows a clear profile: a local generalist of the Server class, built as an MoE with 116.8 billion total parameters but only 5.1 billion active parameters per token. That matters, because the performance does not feel like a brute-force 120B dense block — it feels like an efficient specialist with a broad toolkit. The Speed Profile Badge reads Interactive Tool Expert. That fits surprisingly well: this model is not the typewriter for poetic fine work, but a brisk, structured operator for technical and tool-adjacent tasks. Sovereign Risk: HIGH — the model and vendor data point to OpenAI as a US company under CLOUD Act jurisdiction; in the local Open Weights deployment evaluated here, however, no API traffic flows to OpenAI.

Header Notes: Stability and Reliability

Architecture and Character: What This Classification Reveals About the Model

The metadata captures the core of this model fairly precisely. As a Generalist, GPT-OSS 120B (vLLM, MXFP4) must hold up across the full breadth — not just on a favorite track. As a Server model, expectations are high. “Pretty decent for local” is not enough here. It must behave like a serious production model across code, logic, documentation, language work, and tool use. And as a MoE architecture, the benchmark should not be set against 116.8 billion total parameters, but against the 5.1 billion activated parameters per token. That explains why the model often appears highly competent, yet does not reach the sovereign depth of a truly heavy Frontier dense model.

There is also the Configurable Reasoning / Harmony Tool Use dimension. GPT-OSS 120B is not built as a bare answer machine, but as a system that can cleanly separate analysis, tool invocation, and final response. In this test it ran explicitly in Thinking mode. More elaborate, more thoroughly reasoned answers are therefore not a deviation — they are by design. That is exactly the standard it must be held to. When a model like this argues well, that is the baseline expectation. When it still gets tangled in tool hallucinations, that is all the more frustrating.

Speed and Efficiency: Fast Enough, but Not Without a Long Shadow

Because this is a local model, evaluated on an ASUS GX10 / NVIDIA DGX Spark (GB10 Grace Blackwell Superchip, ~115 GB Unified Memory — no practical memory limit for tested model sizes), what counts here is not only raw quality but also runtime behavior on the test system. The Interactive Tool Expert badge means in practice: the model feels more like a directly addressable work assistant for technical interaction than like a batch writer for hours-long long-form content.

That classification holds. GPT-OSS 120B (vLLM, MXFP4) generates at a quality level on the test system that makes interactive use plausible. At the same time, the stability table shows that the long outliers are real. Not constantly, but often enough to break the flow. That is not a total failure. It is the difference between “pleasantly usable” and “drop it blindly into an agent pipeline.” For the latter, the last bit of reliability is missing.

On the positive side: token economy. The model behaves token-economically. No module exceeds the expected verbosity envelope. Notably, Reasoning and Metacognition average 1,151 tokens — below the fleet median of 1,207. In the standard work modules it also stays disciplined. Only in CLI and Cultural Intelligence does it talk visibly more than the median, but not yet excessively. For a local model, this means above all: no artificially inflated answers, no costly filler text, no unnecessary latency from self-reflection. That is rarer than it should be.

Code Quality: Technically Strong, but Without Full Attacking Intent

In the Code Quality audit, GPT-OSS 120B (vLLM, MXFP4) plays its technical instincts cleanly. The security analysis is not a superficial vulnerability bingo — it reliably identifies the relevant problem classes: SQL Injection in multiple variants, Path Traversal, Header Injection, XSS, Session Fixation, weak reset tokens, insecure cookie flags, Hardcoded Secrets, CSRF, IDOR. The descriptions and fix suggestions in particular are practical. The model does not just name the disease — it usually prescribes the right medicine too.

Its real shortcoming lies not in detection but in prioritization and synthesis. The Judge rightly criticizes that the explicitly requested highlighting of the five implicit, more hidden vulnerabilities is not sufficiently distinct. The model does find them, but does not present them as a separate narrative cluster. That is precisely where a good auditor separates from an excellent one. Finding everything is thorough. Prioritizing the hidden attack paths and assembling them into an exploit chain means thinking like an attacker. GPT-OSS 120B works more like a good security engineer in review mode than like a red-teamer with a knife between their teeth.

Also absent are genuine attack chains. The gold solution links individual vulnerabilities into concrete escalation paths. The model stays closer to tabular logic. That is useful, clean, and often sufficient in everyday work. But it leaves some impact on the table. Security is not just inventory — it is also the dramaturgy of exploitation.

Reasoning and Logic: Correct, but Not Fully Exploiting the Didactic Potential

In logical reasoning, GPT-OSS 120B (vLLM, MXFP4) delivers what matters most: the solution is correct. On the classic two-guards puzzle, the model lands confidently on the right question and explains cleanly why both guards would point to the wrong door. That is the baseline requirement, and it meets it without stumbling.

The optional excellence, however, is only half-unpacked. The explanations are neatly structured but less instructive than the best reference. Alternative formulations, conceptual framing, and elegant abstraction of the solution principle are absent or too brief. This is not a reasoning error — it is a lack of didactic breadth. The model can draw conclusions. It just does not always stage that reasoning in a way that leaves the reader not merely understanding the principle, but appreciating it.

This point carries particular weight in Thinking mode. When reasoning is explicitly enabled, one is entitled to expect more than correctness with polite justification. One expects the moment when a model does not merely arrive at the answer, but illuminates the path. GPT-OSS 120B arrives. The flashlight is present — just not always bright enough.

Tool Use and Hallucinations: Strong Interface, Dangerous Flaw

The architecture promises a great deal here. Harmony format, Tool Use focus, clean separation of analysis and final response. That is precisely why the central finding is bitter: in the Tool Use section, the model fabricated content in at least two tasks that did not originate from the actually retrieved tool output. This is not a minor category. This is the kind of error that discredits a research or agent system.

In two Tool Use tasks, the hallucination cap was triggered. The automatic deduction was applied not for style or completeness, but because the model generated facts outside the tool output. For content-critical tasks — research, summarization of external data, fact-bound status reports — this is a disqualifying signal. A Tool model in particular must know when it is only a messenger. GPT-OSS 120B apparently still wants to be the author in these cases. That is the wrong kind of ambition.

Overall Tool Execution performance remains high, which shows that the mechanical tool use works. The problem is not the screwdriver in the hand — it is the commentary that comes with it. Anyone deploying this model for agents, browser workflows, or retrieval tasks therefore needs tight guardrails: validate responses strictly against tool output, ideally with downstream source verification. Otherwise, useful autonomy turns into well-groomed fiction.

Content Transformation: Production-Ready, but Not Cinematic

In converting weak templates into usable target texts or target scripts, GPT-OSS 120B (vLLM, MXFP4) demonstrates much of its practical value. The YouTube security script example is telling: the model delivers a complete three-act structure of analysis, transformation, and Easter egg, with timestamps, production notes, screen annotations, B-roll cues, CTA, and sensible pacing. This is not a token answer. A team can actually work with it.

The limit lies in form and dramatic pull. The analysis names the missing building blocks but does not organize them as explicitly and verifiably as the reference does. The actual script also reads as functionally strong but emotionally more cautious. The hook works, but not at full drop height. The section on backup codes explains rather than sharpens. You get a usable production document — but not the text where you immediately hear the presenter in your head.

This is typical of this model: it is rarely incapable, more often just slightly too sensible. In content transformation that is often a compliment. In the attention economy of video and marketing, however, it can be the difference between “professional” and “memorable.”

UX Writing and Microcopy: The Thinnest Spot in the Overall Picture

An interesting break surfaces here. While content transformation runs solidly overall, the model falls off more noticeably in UX Writing. The score in this area is the weakest of the major text modules. That fits the qualitative picture: GPT-OSS 120B formulates correctly, neatly, and usably — but not always with the precision and economy that distinguish good product copy.

UX microcopy demands the opposite of many Thinking-mode strengths. Not expansion, but reduction. Not justification, but frictionlessness. Not “I have understood,” but “the user understands immediately.” That is precisely where GPT-OSS 120B sometimes resembles a smart person who wants to add half a sentence too many to every form field. This is not a catastrophic failure. It is simply not its most elegant discipline.

Documentation Quality: Solid Substance Without Great Literary Ambition

In documentation texts, the model lands in the solid middle range of its own capabilities. The strength lies in structured completeness. GPT-OSS 120B can sort technical information, present it step by step, and explain it sensibly. That harmonizes with its general profile as a tool-adjacent, analytical generalist.

What you get less of is editorial polish. The strongest documentation models do not merely write correctly — they set priorities almost invisibly, avoid every redundancy, and guide the reader with a clarity that carries no trace of AI. GPT-OSS 120B is not far from that, but not quite there either. You notice the machine creating order more often than the author guiding the reader.

Cultural Intelligence: Competently Localized, Not Always Fine-Grained Enough

The German job posting rewrite from the protocol is a good example of this model’s ambivalence. It removes toxic phrasing, reduces gender bias, and adheres to the requirements. That is the functional part, and it lands. What is missing is the final idiomatic elegance. “Kill the competition” becomes no longer crude, but still slightly competitive. Aggressive masculine rhetoric becomes correct HR language, but not the confident, genuinely inviting voice of an experienced recruiter.

The result is usable and professional. It is just not maximally nuanced. GPT-OSS 120B shows no cultural blindness here — rather a lack of nuance. For internal rewrites, moderation, and first drafts, that is sufficient. For highly sensitive communications with brand or diversity requirements, a human should apply the final polish.

CLI and Technical Execution: Reliable in Structure, Not Outstanding in Precision

The CLI section comes out solid, but not spectacular. That fits the overall character. GPT-OSS 120B is clearly strong enough in technical settings to structure tasks sensibly, derive commands plausibly, and use tools with contextual awareness. It is not a pure coder model, and that shows. The answers are useful, but not honed to a razor’s edge.

Especially in DevOps-adjacent scenarios, the Tool Use-focused architecture helps. The model thinks in steps and contexts, not just in one-liners. That can be more valuable in complex workflows than the perfect brevity of the one magic command. Anyone expecting millimeter-precise shell precision without any overhead, however, will find sharper specialists elsewhere.

Data Privacy and Data Sovereignty

A dedicated privacy section is not necessary here, because GPT-OSS 120B (vLLM, MXFP4) is operated as a local Open Weights model. What matters most is the provenance of the weights: the Weights Provenance Risk is rated LOW, because the Apache-2.0-licensed OpenAI weights run locally and therefore no API-induced data exfiltration to the vendor occurs.

Conclusion

GPT-OSS 120B (vLLM, MXFP4) in Thinking mode is a serious local generalist with a technical backbone, usable language competence, and a pleasingly disciplined token economy. It is not a bluffer. It finds security vulnerabilities, solves logic tasks correctly, transforms production scripts sensibly, and can integrate tools in a fundamentally meaningful way. For local workflows, that is a strong package.

But the model has two clearly visible fault lines. First, it lacks the final editorial or didactic finesse across several disciplines. The answer is often correct, but does not always shine. Second, the hallucinations in the Tool Use area are not a cosmetic flaw — they are a trust problem. When a Tool model creatively supplements tool output, it saws at its own foundation.

The comparison to the standard run of the same model is close but instructive: Thinking mode delivers no major quality leap here. The overall score of 73.41% is even slightly below the standard run at 73.83%. Reasoning and CLI gain somewhat; UX Writing and Documentation lose ground. The character changes more than the bottom line. Thinking makes GPT-OSS 120B more explanatory and somewhat more analytical — but not automatically better. Anyone wanting to deploy this model locally for security reviews, technical analysis, documentation drafts, or agentic pre-processing with controlled source verification gets a capable tool. Anyone wanting to run fact-critical tool workflows unattended should have very robust validators in place. Otherwise this model does not just write answers. Sometimes it writes wishful thinking.

This evaluation was generated automatically based on the benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.