DeepSeek V4 Pro

With an overall score of 72.56%, DeepSeek V4 Pro shows the profile of a large, ambitious cloud Open Weights model via OpenRouter: a lot of depth, a lot of reach, but also friction. The Speed Profile badge Batch Tool Expert fits surprisingly well. This model doesn’t want to sprint charmingly — it wants to methodically work through complex tasks. As a Frontier model for reasoning with a Mixture-of-Experts architecture — 1,600 billion total parameters with 49 billion active parameters per token — it must be held to high standards. Sovereign Risk: HIGH — DeepSeek is a Chinese provider; Chinese jurisdiction and the warning explicitly reaffirmed by Germany’s BSI on 04.02.2025 apply expressly to cloud API operations.

Header Notes: Stability and Reliability

Architecture and Frame of Expectations

The upfront classification as Thinking, Thinking-Optional, Agentic-Orchestrator, and Long-Context is not mislabeling here — it’s a useful compass. DeepSeek V4 Pro was tested in this run in n/a mode, meaning its default behavior without an explicit thinking switch applied by the benchmark pipeline. This matters because the model card indicates the model fundamentally supports multiple reasoning modes, but the benchmark deliberately measures the behavior a regular API user actually gets without special configuration.

Add to that the structural underpinning: the primary use case is reasoning, the size class is Frontier, and the parameter architecture is MoE. What matters, then, is not the enormous 1,600 billion total parameters, but the 49 billion active parameters per token. That is exactly where expectations should be calibrated. The model doesn’t come in as a raw bulldozer but as a selective specialist with many experts under the hood. In theory, this promises efficiency and specialization. In practice, DeepSeek V4 Pro delivers above all one thing: strategic competence with occasional lapses in form.

Performance Profile: A Lot of Thought, No Rush

The Batch Tool Expert badge already signals the typical use case: batch processing rather than nervous back-and-forth in an editor window. Qualitatively speaking, the generation speed is moderate to low. For a reasoning-oriented Agentic-Orchestrator, that is not automatically a flaw. Such models often plan more internally, even when the visible output doesn’t explode. The measured speed should also be read as an infrastructure value of the cloud provider — here, the endpoint via OpenRouter — not as an abstract property that transfers one-to-one to other deployments.

What can be held against the model, however, is not its pace per se, but the variance. A slow model can be reliable. A slow model with critical tail latency and multiple timeouts becomes an unpredictable colleague who delivers brilliant thoughts and then simply doesn’t show up to the meeting.

Reasoning and Logic: Substantial, but Not Always Obedient

For a model with a reasoning focus, the logic domain is the core. Here, DeepSeek V4 Pro is good, but not flawless. The qualitative logs make its character very clear: the model often genuinely understands tasks, even when it stumbles unnecessarily during execution. A good example is the guardian task. The solution was correct in substance. The classic double-inversion was understood. The problem was not the thinking, but the packaging: the visible answer came entirely in English, even though German was explicitly required.

That is not a minor slip — it is a genuine instruction-following weakness. In production environments with a fixed target language, this is not an academic penalty but a failure. Anyone who needs a German result can do exactly nothing with a correct English answer.

Metacognition Compliance (Reasoning): The model refuses to use the explicitly requested <thought> tags in 3/5 metacog tests, citing a consistent policy statement. The reasoning content is partially correct in substance — the score deduction results from the format refusal, not from reasoning errors. For comparison: in the tag-free reasoning_5* tests, the model achieves an average score of approximately 65.71%, which corresponds to the level of solid Frontier models. CrucibleMark deliberately evaluates native zero-shot instruction compliance as a real-world characteristic. This deduction is methodologically intentional.

This passage matters because it sharpens the actual finding: DeepSeek V4 Pro is not unintelligent. It is occasionally willful. That may be tolerable in analytical tasks. In agent workflows where exact formats, languages, or markup specifications are processed downstream by machines, it is dangerous.

In two tasks in the reasoning metacognition domain, the model also ignored the explicit language instruction and responded in English. This is not an isolated outlier. Across multiple tasks in the reasoning domain, the model shows a consistent pattern: when faced with simultaneous constraints on language, length, and format, it drops the language constraint first.

In one task in the reasoning metacognition domain, the model additionally violated the fixed language requirement of German. The automatic score penalty here is not merely qualitative in nature — it is rule-based. reasoning_metacog_001 and reasoning_metacog_003 each produced a LANGUAGE MISMATCH. The substantive quality of the answer becomes secondary at that point, because the task was already formally missed.

Code Quality and Security: Technically Strong, Operationally Too Verbose

In the security-adjacent Code Quality module, DeepSeek V4 Pro shows its best side. It identifies vulnerabilities broadly, prioritizes sensibly, and formulates fixes concisely and usefully. The PHP security analysis log is exemplary: 21 table rows, correct Markdown structure, cleanly in German, solid coverage of SQL Injection, XSS, Path Traversal, Type Juggling, Session Fixation, IDOR, and Header Injection. This is not a smoke grenade. The model knows what it’s looking for.

The security competence is particularly noteworthy because it doesn’t stop at the obvious OWASP classics. Implicit issues are recognized. Type Juggling on the API key, predictable reset tokens, mail header injection, and session fixation are not lucky guesses. This is a model that can not only name attack paths but plausibly rank them. That is very useful for audits, code reviews, and initial risk triage.

The catch follows immediately: DeepSeek V4 Pro solves such tasks with more text than necessary. In the Code Quality domain, it produces an average of 9,626 tokens against a fleet median of 2,921. That corresponds to a factor of 3.3 compared to the average across all tested models. For API use, this means proportionally higher costs for identical output. Worse still: this overhead is not merely an economic cosmetic flaw — it correlates here with the observed sluggishness.

There is also a practical shortcoming at the module’s core focus. The model delivers a good table, but when it comes to the five implicit vulnerabilities, it too often stops at naming them rather than synthesizing them. The judges rightly flag missing attack chains and a formulated overall assessment. For a human reviewer, that is still acceptable. For teams that want to derive a prioritized action list directly from a single response, something is missing.

CLI and Tool Proximity: Solid, but Not Razor-Sharp

The combination of reasoning focus and Agentic-Orchestrator metadata makes DeepSeek V4 Pro a natural candidate for solid CLI and tool tasks. The partial score reflects exactly that: usable, but not outstanding. The model can structure, plan, and think in tool logic. What it lacks somewhat is the clinical precision of a true direct executor.

That should be assessed fairly. Agentic-Orchestrator models are often designed to decompose tasks and delegate to specialized sub-agents. Weaknesses with exact one-liners or meticulous format schemas are therefore less damning than they would be for pure instruct or coder models. In real-world deployment, you’d want such a model to write the strategy rather than blindly sign off on the last regular expression. The problem is simply: the benchmark also measures direct answering. And there, the final sharpness is sometimes missing.

Content Transformation: Surprisingly Strong, Then Willful Again

In the Content Transformation domain, DeepSeek V4 Pro shows one of its more appealing faces. The video script log reads like the work of a model that doesn’t just check off requirements but understands production reality. Timestamps were complete and plausible, pause markers and editor notes were present, the tone hit the spoken-word requirement precisely, the hook landed, the pattern interrupt was cleanly placed, and the CTA and Easter egg were not just present but usable. This is not filler text — it is workable material.

This task in particular shows why the model shouldn’t be reduced to pure table values. DeepSeek V4 Pro can work surprisingly close to production-ready in creative transformation tasks. It sometimes chooses different emphases than the reference solution — more actionability over analytical parallel structure, for instance. That is a legitimate stylistic choice and often the more useful one in practice.

But here too, formal discipline eventually breaks down. In one task in the Content Transformation domain, the model exceeded the explicit word limit of 250 words by 45%. The system applied an automatic deduction of 16.32 points, or 20%. The substantive quality of the answer becomes irrelevant at that point. The penalty applies regardless.

The length problem is not an isolated outlier. Across multiple tasks in the Content Transformation domain, the model shows a consistent pattern: when faced with simultaneous constraints on language, length, and format, it drops the word limit first. Anyone using it editorially will often get good material — but will need to stand by with scissors.

UX Writing and Documentation: The Price of the Urge to Explain

The weaker module scores in UX Writing and Documentation Quality fit the overall picture. DeepSeek V4 Pro wants to explain, contextualize, hedge. In security and reasoning, that is an advantage. In concise product copy, microcopy, or documentation requiring precision, the same trait acts like a built-in resistance to brevity.

The model is not unusable there. It is simply not disciplined enough. Anyone expecting tight user-facing text, clear labels, or sober documentation with a narrow scope will frequently receive more material than the task calls for. This is the literary equivalent of a screwdriver with a preacher mode. For teams with an editorial review loop, that may be tolerable. For automated generation against hard text budgets, it is unnecessary friction.

API Cost Profile

DeepSeek V4 Pro is a cloud Open Weights model via OpenRouter. Its verbosity is therefore not just a stylistic trait — it’s a bill. In the Code Quality domain, the model produces an average of 9,626 tokens against a fleet median of 2,921. That corresponds to a factor of 3.3 compared to the average across all tested models. In the CLI domain, it produces 1,448 tokens versus 312, a factor of 4.64. In Cultural Intelligence, it’s 1,639 versus 290 — a factor of 5.65.

In short: the model doesn’t just talk a lot — it does so systematically across multiple modules. Because the pricing structure of $0.435 per 1 million input tokens and $0.87 per 1 million output tokens may look attractive, DeepSeek V4 Pro is nonetheless not a cheap model once its actual output volume is factored in. A low token price loses much of its appeal when the model generates three times as many tokens as necessary.

Hallucinations and Safety Character

Noteworthy is what DeepSeek V4 Pro does not do: it does not stand out for aggressive hallucinations. Its weaknesses lie more in format discipline, language fidelity, and stability than in wild fabrication. For security and analysis tasks, that is a relevant trust bonus. A model that sometimes talks too much is annoying. A model that additionally invents nonsense with confidence would be untenable. DeepSeek V4 Pro does not commit that second sin to any notable degree here.

Data Privacy and Data Sovereignty

The data privacy situation is the part of this model that good prompting cannot fix. According to the vendor card, applicable law is Chinese law — specifically PIPL, CSL, and DSL. The stated data location is China plus EU and US cloud partners. For European users, this represents a relevant third-country transfer risk without an EU adequacy decision. No publicly stated GDPR DPA is apparent. For organizations that must operate in GDPR compliance, this is not a peripheral concern — it is a concrete procurement obstacle.

The calculated Sovereign Risk is HIGH. The rationale is clearly stated: as a Chinese company, DeepSeek is subject to the National Security Law. This can enable state access to data and models. Additionally, the explicit BSI warning of 04.02.2025 against using the DeepSeek cloud service for official or sensitive data applies. The retention period remains unclear at -1 days — effectively not reliably disclosed. The weights provenance risk is also HIGH and does not differ from the deployment situation — it confirms it.

Conclusion

DeepSeek V4 Pro is a model with a strong character. As a reasoning-oriented Frontier candidate with MoE architecture and a massive 1000K context window, it can be taken seriously analytically. It identifies security vulnerabilities cleanly, writes surprisingly close to production-ready in transformation tasks, and demonstrates the strategic depth one would expect from an Agentic-Orchestrator. At the same time, it lacks the operational discipline that turns a strong model into a reliable tool. Timeouts, critical tail latency, language switching despite German instructions, and systematic word-limit violations are not footnotes. They are the actual verdict.

DeepSeek V4 Pro is recommended for batch analyses, security reviews, conceptually demanding tasks, and Long-Context scenarios where a human will review the results regardless. It is less suited for tightly timed interaction, format-strict agent pipelines, and any environment where language or length constraints must hold without follow-up review. Across all tests, no notable hallucinations — the model would rather invent too little than too much. That speaks well of it. But reliability is worth more in practice than dignity, and that is precisely where DeepSeek V4 Pro falls short of its potential.

This evaluation was generated automatically based on the benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.