GPT-4o

With an overall score of 68.17%, GPT-4o makes good on exactly what its metadata promises: a fast, broadly capable all-rounder with a strong Instruct character — but not a model that sees every precision task through with the same composure. The Speed Profile Badge “Real-Time Tool Expert” fits well: GPT-4o responds at 153.47 tokens/s with an almost casual speed, and feels like a model that wants to work in dialogue, not in long silent pauses. As a commercial cloud model via the OpenAI API, classified as a Generalist, Frontier-class, dense architecture, it faces high expectations. Sovereign Risk: HIGH — as a US provider, OpenAI is subject to the CLOUD Act; processing occurs under US law, even if European users can be contractually protected.

Header Grades: Stability and Reliability

That’s the good news, and it’s more than cosmetic. A fast model with dropouts is a gamble. GPT-4o delivers the opposite: consistent API responses with no timeouts, plus a P95 response time of 9.61 seconds. For interactive use, that’s excellent. The “Real-Time Tool Expert” badge is therefore not mere marketing label but a useful shorthand for the profile: GPT-4o is tuned for responsiveness and dialogue-adjacent tool scenarios.

The bad news follows immediately. In 29 cases, the system had to fall back to a 4096-token fallback because GPT-4o — or rather the OpenAI API — rejected the originally requested higher completion limit. This affected CLI, Code Quality, Documentation Quality, and Reasoning and Metacognition. For a Frontier model with a 128K context window, that’s an uncomfortable contradiction: accepting a lot of context is not the same as producing a lot of output. Anyone planning to use GPT-4o in agent frameworks or long analytical chains should not treat this limit as a footnote. It is a real product edge.

Architecture and Character: Generalist with Instruct Reflexes

The classification General, Instruct, Multimodal captures GPT-4o with surprising precision. As a Generalist, it is evaluated across the full breadth — not just code or reasoning. As an Instruct model, it exhibits the characteristic virtue of direct command execution: fast, concise, often cleanly on point. But that is also the source of its central weakness. GPT-4o tends to mark tasks as “understood” early on, then cuts corners on depth, formatting discipline, or detail halfway through.

The multimodal classification also needs to be stated clearly for this benchmark: CrucibleMark measures text performance at its core. That captures only part of GPT-4o’s actual capability profile. Still, the point stands: a multimodal all-rounder in the Frontier class has no excuse for underperforming in text. OpenAI positions GPT-4o as a universal work model. That is exactly the standard applied here.

There is also the dense architecture to consider. Unlike Mixture-of-Experts models, the full model capacity is activated for every request. That raises the expectation for consistency. When such a model fails on formatting requirements or output limits, it is not a consequence of exotic routing decisions — it is a genuine product deficiency.

Code Quality and Security: Capable, but Not Thorough Enough

At 70.1% in Code Quality, GPT-4o performs decently — not brilliantly. The qualitative analysis reveals a model that often correctly identifies security issues and can deliver practical fixes. In a security audit, it identified 15 out of 19 vulnerabilities, including SQL Injection, XSS, Session Fixation, Path Traversal, CSRF, and IDOR. That is not a bad hit rate. The proposed fixes in particular were recognized as concrete and actionable. GPT-4o does not produce security fluff — it tends to write genuinely usable code patches.

The catch is completeness. It missed, among other things, a hardcoded API secret, hardcoded database credentials, a reset token without expiry, and a header injection issue. More importantly: it implicitly recognized individual risks but failed to anchor them cleanly in the central findings table. That is precisely where it matters in practice — whether an audit serves as a working foundation or ends up as a loose collection of good thoughts. GPT-4o can do security, but more as an experienced reviewer under time pressure than as a meticulous forensic analyst.

There is also a structural gap in its security thinking. The golden-standard solutions worked with attack chains and clear exploit narratives. GPT-4o more often stayed at generic explanations. It names the open door but does not always describe how the intruder actually walks through it. For day-to-day developers, that is still workable. For reliable audits in critical environments, it is too shallow.

In the Code Quality module, the same system-level finding appeared across all five tasks: the initially requested higher token limit was rejected by the API, and the system fell back to 4096 tokens. This is not a mere measurement artifact — it is a hard operational finding. Anyone expecting large structured analyses with long tables, justifications, and fix snippets will find GPT-4o working under a tighter output ceiling than the context window suggests.

Reasoning and Logic: Right Answers, Rarely Elegant

In Logical Reasoning, GPT-4o lands at 66.04%. That is not a collapse, but it is not a strong result for a Frontier model either. The qualitative picture shows a familiar pattern: GPT-4o often arrives at the correct solution, but with limited depth. In the classic guard puzzle, for instance, the logic was correct, the answer cleanly structured and formulated in German. The model used the required <thought> tags, explained the double inversion plausibly, and only fell short of the broader pedagogical illumination. It solves the task. It does not illuminate it.

This fits the Instruct classification. GPT-4o favors direct, compact explanations over expansive reasoning. In normal user interactions, that is often an advantage. In benchmarks that explicitly target multi-step logic and alternative solution paths, that brevity costs points. The model comes across at these moments like someone who writes down the right formula but does not show the working.

The token fallback applies here as well, as a header-grade concern. Reasoning and Metacognition assets repeatedly ran into the same API limit. This is particularly relevant because these tasks specifically reward longer argumentative output. GPT-4o is therefore not only stylistically brief — it is also infrastructurally constrained. Both factors together explain part of the middling logic performance.

On the positive side: token efficiency. On average, GPT-4o consumed 602 output tokens in the Reasoning and Metacognition module, against a fleet median of 1171. The model does not ramble. The downside is obvious. Less text is only elegance when no thought is missing. With GPT-4o, it is sometimes precision, sometimes a shortcut.

CLI and Operational Precision: Fast, but with a Context Edge

The CLI score of 81.68% is one of the model’s stronger disciplines. That is not surprising. GPT-4o is fast, command-friendly, and capable of delivering concise, purposeful responses. That profile maps well onto shell tasks, where users want commands, flags, and plausible sequences — not essays.

But a crack runs through the polished surface here too. In all six CLI tasks, the originally requested higher token limit was rejected by the API and reduced to 4096 tokens. For classic one-liners, that may be irrelevant. For longer runbooks, diagnostic chains, or tasks with many variants, it is a warning sign. A model can look brilliant in the terminal and still hit an artificial ceiling when operational context becomes more extensive. GPT-4o exhibits exactly this behavior.

The verdict: good for interactive command work, less convincing for sprawling operational documentation or complex agent loops that require significant output depth in a single pass.

UX Writing and Content Transformation: Fast, Fluent, but Not Obedient Enough

This is where GPT-4o gets more interesting, because its human-sounding prose meets tasks where formal discipline matters. The result is mixed.

In UX Writing, the final score is only 62.55%. This is not because GPT-4o cannot write. On the contrary — stylistically, the model is often smooth, accessible, and readable. It fails more often at the last, tedious mile of product copy: exact lengths, explicit output rules, strict formatting requirements. That is the shadow side of a model that wants to be “helpful.” In UX texts, unsolicited helpfulness is often just another name for non-compliance.

In Content Transformation, at 78.3%, this pattern comes into sharp focus. In a task requiring a discrimination-free rewrite of a German job posting, GPT-4o delivered not only the requested rewrite but then added five justifications for its changes. That was explicitly prohibited. The text itself was technically sound: aggressive language removed, gender-coded terms neutralized, professional tone preserved. But the task was unambiguous: output only the revised German text, no explanations. GPT-4o was able to solve the task substantively and still failed at following its instructions. That is not a cosmetic flaw. It is a classic Instruct failure.

A second content probe from the same module reveals the other weakness: in a YouTube tutorial script task, GPT-4o worked with clean structure, in German, with clear sections. At the same time, it compressed a realistic five-minute video down to roughly 1:45 minutes, kept the hook generic, left visual production notes too vague, and fell short of the required 600 to 900 words with approximately 550 words. The model knows what a tutorial looks like. It is less reliably aware of what one needs to feel like in production.

In one Content Transformation task, the model also fell short of the explicit length requirement. The required 600 to 900 words were missed at approximately 550 words. The automatic deduction applies here on a rule-based basis, regardless of whether the text is otherwise usable. In production scripts especially, the word count is not a suggestion — it is a working constraint.

Documentation Quality: The Biggest Disappointment

The weakest module score is 58.47% in Documentation Quality. For a Frontier generalist model, this is the real pain point of this benchmark. Documentation demands structure, patience, completeness, and a clean balance between overview and detail. That is precisely where GPT-4o visibly loses sharpness.

The system-level header grade is already poor: all five documentation tasks hit the same 4096-token fallback. For a module that inherently rewards longer, structured, explanatory output, this is not a footnote — it is a structural liability. Before even reading the quality, one already knows: this model works on documentation tasks with the handbrake on.

The substantive results match. GPT-4o writes clearly and concisely in most cases, but too rarely with the thoroughness that distinguishes good documentation from pleasant chat text. It prioritizes readability over completeness and speed over substance. For FAQ-style help content, that may suffice. For reliable internal documentation, architecture explanations, or processes with many edge cases, it is too lightweight.

Cultural Intelligence: Linguistically Confident, Formally Inconsistent

At 66.8%, GPT-4o lands in the middle of the field in Cultural Intelligence as well. Part of the qualitative analysis initially reveals a strength: the model can write in German with clarity, inclusivity, and tonal control. In the job posting task, it reliably identified toxic language, masculine bias, and unnecessary aggression. The rule-based evaluation also awarded 90 out of 100 for Cultural Fit and 90 out of 100 for Language Proficiency.

The point deductions came again through instruction compliance. GPT-4o did not restrict itself to outputting only the revised text — it commented on its own work, despite that being explicitly prohibited. Additionally, the text failed on inclusive formatting, specifically the missing explicit marker such as m/f/d. That may sound pedantic. In HR and DEI-adjacent contexts, it is not. A model that only approximately follows formal requirements in sensitive communications will fail internal approval processes in real-world deployment.

One could put it more bluntly: GPT-4o has the right attitude here, but not always the necessary discipline.

Token Efficiency: Pleasantly Economical

A clear plus is output economy. Across all budgeted modules, GPT-4o stays below the fleet median: Code Quality 1429 vs. 2273 tokens, Content Transformation 1308 vs. 1757, Documentation 1668 vs. 2821, UX Writing 1041 vs. 1424, Cultural Intelligence 176 vs. 220. The model behaves token-economically — no module exceeds the expected verbosity range.

For a commercial cloud model, that matters. GPT-4o costs $2.50 per 1 million input tokens and $10.00 per 1 million output tokens. Because it rarely becomes unnecessarily verbose, the bill stays manageable in everyday use. Speed and brevity are genuine product advantages here. Only: efficiency must not be confused with completeness. GPT-4o does not just save costs — sometimes it also saves the last important thought.

Hallucinations and Tool Trust

Hallucinations

A genuine warning finding applies here. In two tool-use tasks, GPT-4o generated content that did not originate from the retrieved tool output but was fabricated. The score was capped by a hallucination penalty in both cases. For content-critical tasks such as research, factual reporting, or any form of tool-assisted verification, this is not a minor infraction — it is a red flag.

The problem is not that GPT-4o occasionally guesses. Many models do. The problem is where it guesses: precisely in tasks where an external tool was supposed to provide the factual basis. When you use a model with tool integration, you are implicitly buying the expectation of fewer hallucinations. When the model then fabricates beyond the tool output anyway, it undermines the entire safety benefit of the setup.

This does not make GPT-4o generally unusable. But it disqualifies it for unattended fact pipelines. In journalistic, regulatory, or research-heavy workflows, a simple rule applies: always cross-check tool output, without exception.

Data Privacy and Data Sovereignty

GPT-4o runs exclusively as a commercial cloud model via the OpenAI API. For European companies, the situation is clearly defined: a GDPR DPA is available on the contractual side, and standard GDPR mechanisms such as SCCs are in place. In practice, however, the provider processes data under US law, with data location in the US and a stated retention period of 30 days.

The calculated Sovereign Risk is HIGH. The reasoning is straightforward: OpenAI is a US company, the CLOUD Act applies, and US authorities can under certain conditions demand access to data even when organizational safeguards are in place. For companies in Germany and the EU, this is not an automatic disqualifier, but it is a clear governance consideration. Anyone working with personal, confidential, or regulated data needs to make a deliberate, informed risk decision here.

The Weights Provenance Risk is rated MEDIUM. At its core, it points to the same underlying issue: proprietary weights held by a US provider, with no independent control over deployment or model artifacts.

Conclusion

GPT-4o is a model with a clearly recognizable character. It is fast, cost-effective enough, API-stable, and linguistically very capable. As a Frontier-class generalist, it handles many tasks convincingly — especially where interactivity, concise responses, and tool proximity are required. Its overall score of 68.17% equally reflects the limits: Documentation Quality is weak, Reasoning is solid rather than deep, and under strict instructions the model too often reverts to the old reflex of the over-eager chatbot.

The most significant operational warning is not hallucination alone, but the combination of tool hallucinations and a hard 4096-output-token fallback across numerous tasks. The former damages trust in factual chains; the latter limits usability in long structured outputs. For chat, assistance, quick code reviews, CLI support, and general productivity work, GPT-4o remains a very capable tool. For security-critical audits, high-quality documentation, and unattended tool-assisted research, it should only be deployed with tight guardrails. GPT-4o is not a fraud. But it is also not the model you hand the keys to and then go to sleep.

This evaluation was generated automatically based on the benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.