Hermes 4 405B

Hermes 4 405B is a Frontier-class generalist, densely built, with 405 billion active parameters and a clear profile as a direct instruct model whose extended thinking would in principle be available, but was deliberately not activated for the benchmark. The result of 67.07 percent with the speed profile badge Real-Time Tool Expert reveals a model that is fast, steerable, and often useful — but one that doesn’t always translate the aura of a 405B giant into corresponding thoroughness. Served via OpenRouter as a cloud Open Weights model, Hermes feels in practice less like an unshakeable reference case and more like a highly talented operator with a slight impatience. Sovereign Risk: LOW — the weights originate from Nous Research in the USA and are publicly available; the provenance risk is low, even though US jurisdiction inherently includes the CLOUD Act.

Header Notes: Stability and Reliability

Architecture and Character: Instruct with the Handbrake on for Thinking

The metadata Instruct, Thinking-Optional fits the observed behavior surprisingly well. Hermes 4 405B responds frequently in a tight, goal-directed manner, without the didactic elaboration one expects from genuine thinking models. This is methodologically important: the benchmark measures the default mode. Extended thinking was disabled for fairness reasons. Anyone hoping for deeper derivations is therefore not seeing the full theoretical potential here, but the out-of-the-box behavior of a large, instruction-friendly chat model.

That is precisely why the verdict cuts both ways. For everyday instruction tasks, this directness is an advantage. In logic, security, and more complex transformation tasks, however, that same quality can tip into under-explanation, linguistic sloppiness, or premature shortcuts. A 405B dense model in the Frontier class should be held to that standard. The excuse of a small form factor does not apply here. All parameters are active with every response. The capacity is there. Not every decision the model makes puts it to best use.

Performance: Fast Enough for Interaction, Fast Primarily Thanks to Provider Infrastructure

The Leaderboard value of 44.08 tokens per second represents a very solid interaction speed for users. Combined with the Real-Time Tool Expert badge, this means: Hermes 4 405B is better suited to quick, tool-adjacent requests than to leisurely long-form analysis. The badge is not a marketing ribbon — it is a fairly accurate shorthand for the real deployment profile. This model wants to respond, not ruminate.

The context matters: this is a cloud Open Weights model via OpenRouter. The measured speed is therefore primarily a benchmark of the provider’s cloud infrastructure plus network latency, not some abstract intrinsic property of the weights file itself. Anyone seeing 44.08 tokens per second is mostly seeing a well-connected cloud endpoint. That is pleasant in everyday use, but also dependent on the operator. The single timeout during the test run is a reminder that performance and availability in the cloud always remain a pact with someone else’s infrastructure.

The token economy is a positive. Hermes does not overshoot the expected range in any module. Particularly notable is the lean approach in the reasoning area, where the model averages 592 output tokens — well below the fleet median of 1174. That can be credited as efficiency. One must add, however: sometimes it is simply brevity where depth is called for. Frugality is a virtue until it becomes omission.

Code Quality and Security: Useful, but Not an Auditor with a Hunter’s Instinct

In the Code Quality area, Hermes 4 405B reaches 67.64 percent. That is not a total failure, but also not a score that automatically commands respect from a Frontier model. The qualitative logs reveal a fairly clear pattern: the model identifies a lot, formulates cleanly, structures neatly in table form, and delivers concrete fixes. The problem is completeness. In a security audit with 19 relevant vulnerabilities, Hermes found only 15. That is a gap of 31 percent. Adequate for a helper. Not enough for an auditor.

The judge logs name the omissions very concretely: reflected XSS, header injection after output, hard-coded secrets, missing CSRF protection, root access without a password, and a reset token without an expiry time were all missed. More problematic still is the severity assessment. Path traversal and IDOR were rated too leniently, even though both are correctly classified as critical in the reference solution — because they can chain with other weaknesses into real takeover paths. That is precisely where table-filling diligence separates from genuine security understanding. Hermes describes individual leaks but too rarely constructs the threat picture from them.

This does not mean the model is worthless in a security context. On the contrary: for initial analyses, structured review tables, and the rapid collection of typical vulnerabilities, it is productive. But no one should make the mistake of reading its output as the conclusion of an audit. Hermes 4 405B is a diligent junior with good terminology and tidy Markdown — but without the suspicious instinct of an experienced pentester.

CLI and Tool Proximity: Light and Shadow Behind the Badge

Tool and CLI proximity is one of the more appealing traits of the model in the overall picture. The Real-Time Tool Expert badge does not come from nowhere, even if the concrete CLI score of 75.67 percent does not mark dominance. Hermes works concisely, follows instructions mostly directly, and wastes little output space. That is exactly what you often want in shell-adjacent or agentic flows. A model that does not write an essay about grep before naming the command saves nerves.

At the same time, the tool score of 60.0 percent should not be glossed over. For a model that, by its own positioning, is supposed to be optimized for tool use, JSON, and structured output, there is room for improvement here. The result is better than arbitrary improvisation, but not the kind of icy precision you would blindly factor in for autonomous agents. Anyone deploying Hermes in tool loops should validate response formats and secure critical steps. The model is ready to work — but not yet ready for an unsupervised shift.

Reasoning and Logic: Often Correct, but Too Often Too Brief and Linguistically Imprecise

With 66.26 percent in logical reasoning, Hermes 4 405B displays an interesting, mildly frustrating quality. It is often right. It just does not always explain its path in a way that makes you want to follow it. This again fits the instruct character in default mode: straight to the solution, sometimes with too little scaffolding for the reader.

The guard puzzle from the logs is exemplary. The model was substantively correct. It chose an equivalent form of the classic solution, and the internal reasoning was logically sound. But in the final German output, the explanation remained thin. The judge rightly noted missing verification, no clean case distinction, and insufficient pedagogical elaboration. You can use the result. You just would not want to reprint it as a teaching example.

More significant is the language dimension. In two metacognition tasks within the reasoning module, an automatic Language Mismatch occurred: the model responded predominantly in English even though German was required. This is not a cosmetic flaw. It is an instruction-following problem. Particularly in tasks that simultaneously demand language, structure, and reflection, Hermes apparently loses the language specification first. This language failure is not an isolated outlier. Across multiple tasks in the reasoning area, the model shows a consistent pattern: when simultaneous constraints on language, length, and format are imposed, the language constraint is the first condition to be dropped.

In one affected reasoning task, the explicit German language requirement was violated; the system applied the rule-based deduction automatically. Substantive quality then tells only half the story, because the benchmark deliberately penalizes such violations harshly. That is methodologically correct. An assistant that responds in the wrong language has already failed the task in many real-world environments.

UX Writing: Professional, Usable, Slightly Bloodless

In the UX Writing & Microcopy module, Hermes stands at 68.93 percent. This is a domain where the model can deploy its instruct virtues reasonably well. The logs rightly note that real problems were identified, jargon was reduced, progressive disclosure was applied, and content was trimmed for mobile surfaces. The work is professional, competent, and clean within budget. You do not get a rambling wall of text, but a structured result.

But something is left on the table here too: personality, warmth, edge. The judge describes the output as competent but stiffer than the reference. A switch between formal “Sie” and informal “du” within a single table is not a catastrophe, but for microcopy it is precisely the wrong place for carelessness. UX copy lives on tonal consistency. Stumbling there is not stumbling on a side stage.

The larger point is completeness in the details. In the analyzed flow optimization, the third step remained structurally incomplete, even though the rest of the concept was coherent. That is typical Hermes: the core idea lands, the execution tears just before the finish line. Not a disaster. But also not the polish one should take for granted at this model size.

Content Transformation: Good Foundations, but Language Discipline Drags the Score Down

With 70.49 percent, Content Transformation is one of the stronger areas for Hermes 4 405B. The log shows that the model gets a fair amount right substantively: gap analysis, timestamps, production notes, hook, pattern interrupt, call to action. Technically, that is quite solid. The output was also not bloated, remaining clearly within budget at 2646 tokens.

Then comes the catch — and it is not a small one: in a video script task, Hermes responded almost entirely in English even though German was explicitly required. The system flagged an automatic Language Mismatch violation. In one Content Transformation task, the model ignored the language instruction and lost points on a rule-based basis, regardless of whether the script itself was functional. That is exactly how a hard constraint must be treated. If you order a German campaign and receive an English script, you have not received a “partially correct” result.

The language failure is not an isolated outlier here either. Across multiple tasks with combined constraints, Hermes shows a consistent pattern: when language, format, and creative structure are required simultaneously, the language specification is remarkably often the first condition dropped. For content teams, this is a real risk, because such errors in production pipelines are not elegant — they are expensive.

Documentation Quality: Solid Structure, Limited Depth

Documentation Quality comes in at a rather sobering 63.73 percent. This fits the broader observations: Hermes can document, but not always with the patient thoroughness that distinguishes good technical documentation. The average token usage of 2087 tokens against a fleet median of 2838 initially looks efficient. In practice, however, the output often reads more like compressed common sense than thorough illumination.

For readers, this means: summaries, reformulations, and structured documentation blocks are where Hermes is comfortable. Where reliable derivation, exhaustive edge cases, or the didactic spelling-out of complex states are required, the air gets thinner. This is not accidental — it is a direct consequence of its instruct inclination in default mode. Hermes prefers to answer rather than to unfold.

Cultural Intelligence: Formally Correct, Not Quite Humanly Warm Yet

At 61.04 percent, Cultural Intelligence is no showcase discipline. The logs do show that the model often meets the hard requirements: removing toxic terms, reducing gender bias, delivering output strictly in German without explanatory text. That is the good news. The less good news: the language frequently stays in “clean but generic” mode. Inclusivity is achieved, but not always convincingly embodied.

The example at hand makes this clear. Hermes correctly replaced problematic terms, found the neutral designation “Fachkraft,” and delivered a professional version. What was missing was the warmth of an invitation, the explicit signaling of openness, and the linguistic elegance that turns mere rule compliance into genuinely inclusive communication. The model sounds here more like an HR handbook than like human insight. You can work with it. You just will not automatically win hearts with it.

Data Privacy and Data Sovereignty

For this review, the concrete usage scenario counts: Cloud Open Weights via OpenRouter. The model and vendor data show a calculated Sovereign Risk of LOW. The reasoning is sound: the weights originate from Nous Research in the USA but are openly available; the provenance risk is therefore low and not tied to a proprietary vendor API. According to the Vendor Card, Nous does not operate its own public API; data privacy risks arise only through the third-party provider actually used.

That is precisely where the practical point lies for European companies. Nous Research is based in San Francisco, USA. The origin of the weights therefore falls under US jurisdiction in principle, even if the Vendor Card does not specify a fixed hosting regime for the open weights model itself. The reference to the CLOUD Act remains relevant: US authorities can, under certain conditions, demand access to data even when it is physically processed outside the USA. For German and European companies, this is not a theoretical edge case — it is compliance reality.

The available card data lists the data location as “Local or third-party hoster”, a data retention of 0 days, and no GDPR DPA on the vendor side. For enterprise use, that is only half the picture, because actual data protection depends on the cloud provider used here. No verified provider details for the deployment infrastructure were available in the card data. Anyone wishing to deploy Hermes 4 405B in regulated environments must separately review the data protection terms of the specific endpoint operator. Without a robust data processing agreement, GDPR-compliant enterprise operation quickly becomes not a technical problem, but a procurement problem.

Conclusion

Hermes 4 405B is an interesting model with a clearly recognizable character: fast, concise, instruction-friendly, often useful, and rarely hallucinating — but not as sovereign as its 405 billion dense parameters might suggest. The overall score of 67.07 percent is not bad, but it is also not a performance that commands awe in the Frontier class. Its strengths lie in structured directness, usable tool proximity, solid code analysis, and economical output. Its weaknesses lie in depth, security completeness, and above all in language compliance under multiple simultaneous constraints. Across all tests, no notable hallucinations — the model would rather invent nothing than embarrass itself.

The recommendation is therefore precise. For general assistance, structured text work, quick initial technical analyses, and interactive tool workflows, Hermes 4 405B is a serious candidate — particularly if you are looking for a cloud Open Weights model via OpenRouter. For security audits, format-critical multilingual workflows, and unsupervised agentic pipelines, caution is warranted. Anyone deploying Hermes productively should add a validation layer on top: check language output, cross-read critical findings, verify tool outputs. With those guardrails in place, this model is useful. Without them, it occasionally feels like a very large engine that upshifts too early in the wrong places.

This evaluation was generated automatically based on the benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.