GPT-OSS 120B (Thinking)

With an overall score of 74.36% and the Speed Profile Badge Interactive DevOps Expert, GPT-OSS 120B presents itself as an ambitious generalist with a full toolbox: fast enough for dialogue, technical enough for serious work, but not without rough edges you can run into day to day. For a Server-class model with MoE architecture, the fair benchmark is not the total size of 116.8 billion parameters but the active capacity of 5.1 billion parameters per token. That is precisely where this model draws its character: remarkably efficient, often clever, sometimes too terse, and not as reliable on factual fidelity in Tool-Use moments as you would want from a model bearing the OpenAI logo.

Header Notes: Stability and Reliability

Architecture and Classification

GPT-OSS 120B enters here in an interesting dual role. On paper it is classified as Thinking-Optional — a model family that supports extended reasoning in principle. The specific test run, however, was conducted explicitly in Thinking Mode. This is not a minor detail; it shapes the overall impression: you see more structured, deliberate responses, but you also have to live with noticeable outliers in response time.

Add to that the MoE architecture — a Mixture-of-Experts approach. Put simply: not all weights are active for every token, only a selected subset. This saves compute and explains why a model of this nominal size feels more like a disciplined specialist unit than a sluggish tanker. The benefit lies in efficiency and specialization, not in sheer brute-force capacity. For evaluation purposes this means: GPT-OSS 120B must be measured against other strong generalists, but with its active model capacity in view.

The metadata tags Native-Quant and Harmony are also more than labels. Native quantization means here that the model is built for quantized operation and was not simply trimmed down after the fact. Harmony, in turn, refers to the internal response schema with clearly separated analysis and output channels. This shows up positively in testing: reasoning feels mostly ordered, not improvised. And the Tool-Use tag is earned — with a caveat. The model can use tools. It just does not always adhere strictly enough to their results.

Speed and Efficiency

The Interactive DevOps Expert badge is a useful shorthand. It says: this model is not built solely for overnight batch runs; it is meant to respond in real time within technical workflows. Qualitatively, that holds. GPT-OSS 120B feels fast enough for interactive use on the test system under normal conditions, without collapsing into hasty short answers. The downside is equally clear in the data: the tail is long. The model responds well most of the time, but in a meaningful share of cases it pulls the user out of their flow.

GPT-OSS 120B is at least token-economical. No module exceeds the expected verbosity range. This is noteworthy for a Thinking run, because many reasoning models lose themselves in sheer text volume. That does not happen here. In the CLI area and in Cultural Intelligence, the model does produce visibly more text than the fleet median, but stays cleanly within reasonable bounds. This is not a weakness — more of a signature: slightly inclined to explain, rarely verbose.

Code Quality and Security: Competent, but Not Forensic

In the Code Quality module, GPT-OSS 120B displays a classic strength of good generalists. It catches a lot, structures things neatly, and delivers usable tables rather than vague generalities. The visible response in the reviewed security audit was fully in German, cleanly formatted as a Markdown table, and included the required columns. That is the baseline. The higher bar would have been not just naming the vulnerabilities but prioritizing them with the necessary sharpness. That is precisely where the model becomes vulnerable.

The Judge credits 20 identified vulnerabilities, indicating broad problem coverage. That sounds strong, and on first pass it is. But on critical points, GPT-OSS 120B stays too shallow. Particularly notable is the miscalibration around IDOR — unauthorized object access via manipulable parameters. What should have been flagged as a critical escalation chain gets downgraded to a validation side note. Similarly with SQL injection in the password reset path: present in the material, but not cleanly extracted as a standalone critical finding. For a developer checklist, that is sufficient. For a reliable security review, it is not.

The second problem is the absence of an attack narrative. Good security models do not just list gaps — they show how two medium-severity flaws combine into a full compromise. GPT-OSS 120B delivers fixes and labels, but almost no attack chains, almost no proof-of-concept thinking, almost none of that second layer where security moves from diligent work to genuine expertise. The model knows where things are burning. It just does not always explain how the fire spreads through the building.

Precisely because GPT-OSS 120B enters as a generalist, this result is still respectable. A score in the mid-70s for Code Quality, plus strong formatting discipline, shows: for code reviews, bug lists, initial audits, and developer communication, the model is well usable. Anyone who needs to reliably prioritize threat models, exploit paths, or severity ratings should place a second instance alongside it. GPT-OSS 120B is a solid analyst here — not a digital incident responder.

Reasoning and Logic: Right Thinking, Not Always Deep Enough

In the Reasoning section, GPT-OSS 120B visibly benefits from the activated Thinking Mode. The model solves the logical guard puzzle correctly, explains the double inversion cleanly, and stays linguistically clear throughout. That is worth more than it sometimes sounds in an era of reasoning marketing. Many models produce impressive volumes of text on logic tasks and still stumble over the basic structure. GPT-OSS 120B does not. It understands the mechanics.

What is missing is the second layer. The Judge rightly criticizes the low didactic depth: no verification table, no alternative formulations, little abstraction toward the general principle behind the solution. The model answers the question. It does not teach it. For users who simply want to move forward, that is entirely sufficient. For learning or explanation contexts, potential is left on the table.

The mode comparison within the same model family matters here. This report covers the Thinking run, and it differs clearly from the standard run of the same model: the overall score is higher, primarily because UX Writing, CLI, and Tool-Use improve. In return, the model gives up some directness in Documentation Quality and Content Transformation. This fits the picture. With thinking activated, GPT-OSS 120B feels more deliberate and strategic — but not automatically more elegant across every writing format.

Content Transformation: Usable, but with a Dangerous Tendency to Fabricate

In the Content Transformation module, GPT-OSS 120B illustrates why you should read benchmark protocols and not just scores. At first glance the performance is solid: the model produces a usable video script in German, complete with timing, production notes, and a broadly plausible flow. It hits the rough shape. In a production context, that would already be more than a blank template.

The Judge, however, identifies a real fault line. In the analysis phase, GPT-OSS 120B stays too generic and does not break down the required elements with sufficient granularity. More critically, in the so-called Easter Egg section: the model invents a free trial for a premium password manager that does not exist in the given context. This is not charming embellishment — it is a hallucination error with product proximity. Exactly these kinds of fabrications poison research, tutorial, and campaign work, because they look professional and are therefore dangerous.

In one Tool-Use task, the model hallucinated content that did not originate from the actually retrieved tool result. The P2 score was consequently capped by the hallucination penalty. For content-critical tasks such as research, factual reports, or automated summaries, this is not a cosmetic flaw — it is a disqualifying criterion.

Beyond that, stylistic consistency is not perfect either. The script remains functional but analytically underexploited, with a somewhat generic call to action and less editorial precision than the best comparable work. GPT-OSS 120B can deliver here. It just does not deliver with the final degree of care. And in content work, unfortunately: one fabricated line can ruin ten good paragraphs.

UX Writing and Documentation: Solid, but Without Shine

The module scores paint a mixed but readable picture. In UX Writing, GPT-OSS 120B clearly improves in the Thinking run. Responses appear more structured and consistent with thinking activated. This helps with microcopy that needs to be not just friendly but precisely targeted. The model is still not a brilliant specialist for tone and voice. It writes sensibly rather than elegantly.

In Documentation Quality, the picture is reversed. The Thinking run falls behind the standard run. This is not a contradiction — it is a familiar pattern: more deliberation does not automatically produce better documentation. Especially with instructions and structured knowledge transfer, additional internal complexity can translate into slightly more cumbersome, less streamlined text. GPT-OSS 120B documents usably, but without the cold precision that distinguishes strong documentation models. You rarely get garbage, but you also rarely get the page you would copy directly into the manual.

Cultural Intelligence: Correct, Respectful, Linguistically Confident

In the Cultural Intelligence module, the model shows one of the more appealing sides of its personality. The reviewed response stays fully in German, defuses toxic formulations, works more inclusively, and maintains a professional tone. This is not spectacular, but it is reliable. The Judge’s criticisms are primarily about nuance: plural instead of singular, slightly bureaucratic word choices, imprecise inclusive spelling with minor formatting errors. This is not a systemic failure — it is a loss of polish.

For tasks like these, what matters is whether a model avoids escalation without tipping into empty PR language. GPT-OSS 120B manages that most of the time. It does not come across as visionary here, but as civilized. For HR-adjacent reformulations, sensitive adaptations, and linguistically respectful revisions, this is a usable profile.

CLI and Tool-Use: Strong on Access, Weaker on Fidelity

The module scores for CLI and Tool-Use look very good at first glance. In the CLI benchmark, GPT-OSS 120B achieves a strong result in the Thinking run, confirming the badge as a DevOps-oriented model. This suggests it does not merely circle technical tasks linguistically but can structure them operationally. Anyone looking for shell commands, workflow plans, or step-by-step technical guidance will generally get substance.

But in Tool-Use, the model’s central ambivalence becomes apparent. The architecture promises a lot here: native Tool-Use, Harmony structure, reasoning-oriented separation of analysis and final response. In practice, a single hallucination finding is enough to erode trust. Because Tool-Use lives on fidelity to the source. A model may phrase things elegantly, may condense, may prioritize. But it may not fabricate content that the tool never returned. That is exactly what happened here.

The overall picture is therefore: GPT-OSS 120B is capable with tools, but not foolproof. For technical agents with human oversight, that is acceptable. For unsupervised, content-critical tool pipelines, it is too risky.

Data Privacy and Data Sovereignty

Since GPT-OSS 120B is operated here as a purely local Open Weights model, no API-driven data transfer to an external provider occurs during active use. The provenance of the weights remains relevant: the risk is rated LOW according to the Model Card, because while OpenAI as a US company is subject to the CLOUD Act, local deployment avoids precisely the sensitive point — namely, the transmission of productive inputs to OpenAI servers. For European organizations, this is the good news: the legal origin remains US, but operational data sovereignty rests with the operator.

Conclusion

GPT-OSS 120B is a local model, evaluated natively on ASUS GX10 / NVIDIA DGX Spark (GB10 Grace Blackwell Superchip, ~115 GB Unified Memory — no practical memory limit for tested model sizes). In this configuration it shows a clear profile: a capable Server-class generalist whose MoE architecture extracts a remarkable amount of competence from relatively lean active capacity. The Thinking run lifts the model noticeably. It makes GPT-OSS 120B better at logic, UX-adjacent formulation, and technical interaction. The standard run of the same model family remains more direct and is even somewhat cleaner in individual writing modules — but weaker overall.

Its strengths lie in structured technical work, solid code analysis, usable reasoning, and decent tool integration. Its weaknesses are about depth rather than breadth: security findings are often correct but not sharply enough prioritized; explanations are accurate but not always instructive; content work can be functional but tips into hallucination at the wrong moment. That is the critical point of this model. Not speed, not stability — but the question of whether you can blindly trust it on tool-sourced facts. The answer is: better not.

For local developer setups, technical assistance, code reviews, CLI support, and general knowledge work, GPT-OSS 120B is a serious option. For automated research, fact-critical content pipelines, or security audits without human follow-up, it lacks the final degree of reliability. A capable model, then, with character and utility — but also with that slight tendency toward improvisation that looks interesting in the lab and can get expensive in production.

This evaluation was generated automatically based on the benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.