GPT-OSS 120B

With an overall score of 73.83%, GPT-OSS 120B (vLLM, MXFP4) is a serious generalist with a clearly recognizable profile, but without the effortless polish of the top models. That fits the classification: Server-class, Generalist, MoE architecture with 116.8 billion total parameters, of which only 5.1 billion are active per token. What you get is not the raw force of a fully active 120B dense model, but the targeted work of an expert system on a diet. The Speed Profile Badge reads Interactive Tool Expert: designed for dialogic tool tasks with interactive demands, not for stoic bulk writing. The tested variant ran in Standard mode, meaning without the Thinking toggle enabled. Shorter, more direct answers are the expected baseline here, not a shortcoming.

Header Notes: Stability and Reliability

The stability rating comes out split. There are no failures, no silent dropouts, no retry lottery. That is worth more than it looks at first glance, especially with local Open Weights models. At the same time, response time in the long tail of the top five percent varies far too widely for truly snappy workflows. Anyone wanting to use GPT-OSS 120B (vLLM, MXFP4) as a tool for interactive tool chains gets reliability, but not always rhythm.

Architecture and Mode: a MoE Generalist with a Configurable Head

The pre-assigned category captures the character with surprising precision. This model is a generalist, not a specialist. It belongs to the Server class and uses a Mixture-of-Experts architecture, or MoE for short. That means the model nominally carries a very large number of weights, but activates only a small fraction of them per token. More relevant than the large number on the box are the 5.1 billion active parameters. That is exactly what performance should be measured against.

This explains a lot. GPT-OSS 120B (vLLM, MXFP4) comes across as competent in multiple disciplines, often even elegant, but rarely massively superior. It has breadth, not the brute force of a Frontier dense model. The Configurable-Reasoning tag is also important, even though this test run explicitly took place in Standard mode. The model can in principle ramp up deeper inference, but was measured here without that lever engaged. Anyone not seeing maximum performance in Reasoning throughout is not looking at a defect, but at the deliberately sober operating mode.

Add to that Native-Quant and MXFP4. This is not merely an implementation note, but part of the character. The model is tuned for efficient local operation without feeling like a folded-up emergency fallback. Harmony and Tool-Use finally point to an architecture that separates analysis, tool calls, and final answer more cleanly than many older instruct models. In practice, that helps. It does not, however, immunize against hallucinations when external tool results are incorrectly assembled or embellished.

Speed and Token Economy

As a local model on the reference system ASUS GX10 / NVIDIA DGX Spark (GB10 Grace Blackwell Superchip, ~115 GB Unified Memory — no practical memory limit for tested model sizes), GPT-OSS 120B (vLLM, MXFP4) shows an unusually attractive speed profile for its nominal size. The Interactive Tool Expert badge is well chosen here. The model does not feel like a batch writer that first takes a deep breath and then unrolls a manuscript. It is more oriented toward direct interaction, tool responses, and structured follow-up work.

More important than raw speed is economy. Here the model works with discipline. No module exceeds the expected verbosity envelope. This is particularly noteworthy in the Reasoning area: the answers are not terse for the sake of terseness, but terse enough to avoid drifting into explanatory padding. In the remaining modules, GPT-OSS 120B (vLLM, MXFP4) likewise stays within reasonable text volumes. For a model with configurable reasoning, that is not a given. Some candidates talk themselves to death. This one does not.

Code Quality: strong security instinct, slightly lacking in sharpness

In the Code Quality module, GPT-OSS 120B (vLLM, MXFP4) clearly belongs among the better all-rounders. It reliably identifies the major issues: SQL injection in multiple places, plaintext passwords, path traversal, privilege escalation via cookies, session fixation, IDOR, weak token generation, CSRF, hardcoded secrets, debug leaks. This is not mere diligence, but a sign that the model maps security patterns broadly and cleanly.

The weakness lies one level deeper. Of all things, a comparatively classic XSS issue in the welcome output was missed. That is not a cosmetic scratch. When a model overlooks a standard OWASP finding in a systematic security analysis, confidence in its completeness shrinks immediately. On top of that, the view for attack chains is missing. Individual problems are named, but not consistently condensed into a realistic exploit path — for example, from IDOR through password reset to account takeover. That is precisely where solid bug detection separates from genuine security competence.

The form is positive. The required Markdown table was correct, the explanations stayed concise, and severity ratings were mostly plausible. For developers who first need an audit framework, that is useful. For a team that needs prioritization under time pressure, the final precision is still missing. GPT-OSS 120B (vLLM, MXFP4) reliably sees the smoke. Where exactly the fire needs to be extinguished first, it does not always explain with the necessary sharpness.

Logic and Reasoning: correct, clean, slightly too well-behaved

In the Reasoning module, the model shows one of the more appealing sides of its character. It solves the classic guard puzzle correctly, explains the double inversion mechanism cleanly, and remains readable throughout. The answer is tidy, consistently in German, and avoids unnecessary embellishment. For Standard mode, that is a good signal. GPT-OSS 120B (vLLM, MXFP4) does not think out loud until sunset — it delivers.

Precisely because the model is classified as a configurable reasoning system, however, what is missing must also be named. The explanation stays at the correct but rather sober level. Visualization, robustness argument, pedagogical depth, and the generalization as a pattern of self-referential questions are only touched on or omitted entirely. The result is right. The didactic elegance falls behind stronger Reasoning candidates.

That said: the concrete metacognition test shows that the model used the required <thought> tags in this case and did not refuse the format specification. That is not a minor point. Models from a Harmony-adjacent school of thought tend to stumble on exactly these benchmarks because they are internally organized differently from what the prompt demands. Here the model was cooperative enough not to fail on its own architectural posture.

Content Transformation and UX Proximity: practically strong, but not always refined enough

The strength of GPT-OSS 120B (vLLM, MXFP4) in content transformation lies in its practical usability. The long video script test demonstrates this well. The model delivers a complete, editorially usable script with timestamps, production notes, pause markers, CTA, B-roll ideas, and an Easter egg. That is more than mere text production. It is application-oriented work.

But here too: usable is not identical to brilliant. The preceding analysis remained too generic and less scannable than the reference. The script felt functional, but slightly too sprawling in its pacing. The title promised five minutes; the text ran longer. Such differences sound minor but are not a detail in production reality. Video formats often die from wrong length faster than from mediocre language.

Linguistically the model is solid, at times even pleasantly unpretentious. What it occasionally lacks is the idiomatic finesse that turns correct German text into genuinely precise German text. This is also visible in the reformulation of toxic passages in recruiting texts. The model reliably defuses openly problematic terms and corrects gender imbalances solidly. At the same time, it leaves residual aggression like “challenging the competition” standing, where a more cultivated solution would have neutralized the tone entirely. This is typical of this model: rarely badly off, but not always with the final feel for cultural undertone.

Cultural Intelligence: competent, but not maximally calibrated

In the Cultural Intelligence area, GPT-OSS 120B (vLLM, MXFP4) demonstrates that it does recognize discriminatory or toxic elements in source material and converts them into a more professional form. The German language is handled correctly, the answer stays free of language mixing, and the task is formally completed cleanly. Gender-neutral reformulation also succeeds here in principle.

The open flank is cultural fine-tuning. The model removes the coarse splinters but does not sand down every sharp edge. Especially in German-language recruiting, the difference between “formally correct” and “genuinely inviting” is larger than many models assume. The reference resolves this more psychologically inclusive, more idiomatic, and with more feel for inviting framing. GPT-OSS 120B (vLLM, MXFP4) by contrast remains somewhat technocratic. It sanitizes the text. It does not fully transform it into a text that radiates trust.

Tool-Use, Hallucinations, and Safety Trust: this is where it gets serious

The model’s biggest warning light does not hang over general writing, not over coding, not even over reasoning. It hangs over Tool-Use. Three identified hallucination violations in tool tasks are not operational noise — they are a concrete breach of trust. In the affected tasks, the model generated content that did not originate from the retrieved tool result but was fabricated. The score was capped there by the hallucination cap. That is methodologically strict and substantively entirely correct.

For readers outside the benchmark context, the translation into everyday language: when a model researches within a tool chain, queries data, or summarizes external results, it must not add anything invented. If it does, the answer is devalued for fact-critical applications. That is exactly what happened here, and not just once. In the Tool-Use area, GPT-OSS 120B (vLLM, MXFP4) must therefore be treated with considerably more skepticism than its overall impression would suggest.

There is also a particularly uncomfortable finding: in one Tool-Use task, the model reported success but produced no visible response text. That means either a silent refusal, a silent failure in the runtime chain, or a purely internal reasoning output without a presented final answer. In any case, there was no assessable output. For agent frameworks, this is insidious because the pipeline formally reports “green” while nothing is actually there. That is the kind of failure that triggers pagers at night.

Because the model carries the architecture tags Harmony and Tool-Use, this finding carries double weight. One may reasonably expect a system with those labels to handle tool results with discipline. Exactly that discipline is not stably present here. Anyone wanting to use GPT-OSS 120B (vLLM, MXFP4) for research, reports, or automated data summaries needs downstream validation. Without a safety net, that is too much trust in a model that has already clipped the curb in this area.

Documentation, CLI, and the Overall Impression

The individual sub-scores paint a model that is broadly accessible as a generalist. In CLI and documentation tasks it reaches solid, at times good performance, but not the mechanical precision with which top models make themselves unassailable in DevOps-adjacent benchmarks. That fits the overall picture again: GPT-OSS 120B (vLLM, MXFP4) is not a specialized tool, but a well-trained generalist with technical affinity.

That is precisely why the combination of reasonable code quality, usable documentation competence, and local operation is attractive. The model does not fail on trivial formatting questions, does not produce sprawling walls of text, and stays at a workable level across most disciplines. It is a capable colleague. Just not one you should hand the keys to the server room and the fact-checking desk simultaneously without oversight.

Data Privacy and Data Sovereignty

Since GPT-OSS 120B (vLLM, MXFP4) is deployed as a local Open Weights model, the data privacy situation here differs from that of a cloud API. The weights originate from OpenAI, a US company, and the weights provenance risk is nonetheless rated as LOW. The decisive reason is simple: in local deployment, no user data flows to OpenAI servers. For European companies, this is a real advantage, because data sovereignty is technically secured through the operating model rather than merely contractually.

Conclusion

GPT-OSS 120B (vLLM, MXFP4) is an interesting model with a mature profile. As a local generalist in the Server class, it delivers broad competence, very good token discipline, high practical utility in transformation tasks, and a solid to good technical baseline intelligence. The MoE architecture with only 5.1 billion active parameters explains why the model feels more efficient than its total parameter count would suggest. It is not a bluffer. But it is not an all-rounder either.

The weaknesses are precisely nameable. In security contexts, the final eye for completeness and attack chains is occasionally missing. In Reasoning, the model is correct but not exceptionally deep. In language and cultural tasks it works cleanly, but not always with a fine ear. And in Tool-Use sits the actual liability: repeated hallucinations based on external results and one case with no visible answer. For content-critical, tool-assisted processes, that is not a cosmetic flaw — it is a red line.

A direct comparison with the also available Thinking run reveals an instructive difference: Standard mode achieves the slightly higher overall score and feels more balanced, while Thinking mode performs somewhat stronger in logic and tool tasks but loses smoothness in UX-adjacent and documentation tasks. That points to an unusually clear recommendation. Anyone wanting to use GPT-OSS 120B (vLLM, MXFP4) locally as an all-round assistant, technical writing partner, or structured code reviewer is better served in Standard mode. Anyone working specifically on trickier reasoning tasks can consider the Thinking toggle. For unattended tool pipelines with factual requirements, this model remains — despite many qualities — a candidate requiring supervision. That is not a dismissal. It is a verdict of respect with a warning label.

This evaluation was generated automatically based on the benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.