DeepSeek V4 Pro

With an overall score of 72.56 percent and the Speed Profile Badge “Batch Tool Expert”, DeepSeek V4 Pro makes its intentions very clear: not a frantic chat all-rounder, but a large cloud Open Weights model via OpenRouter that works through tasks rather than brushing them aside. The metadata fits: reasoning as primary purpose, Frontier class, MoE architecture with 1,600 billion total parameters and 49 billion active parameters per token, plus a context window of 1,000,000 tokens and a training cutoff of 2025-05. The run was conducted using the endpoint’s factory default behavior; no switchable thinking mode exists in this test, although the model family generally supports configurable reasoning variants. Sovereign Risk: HIGH: DeepSeek is a Chinese company subject to China’s National Security Law (NSL), which may allow state access to data and models. On 04.02.2025, Germany’s BSI explicitly warned against using the DeepSeek cloud service: user data is stored on Chinese servers; use for official or sensitive data is not recommended. This warning applies without restriction to cloud API deployments.

Header Notes: Stability and Reliability

Architecture and Character: Thinking Big, Not Always Executing Cleanly

The tag combination here is not decoration but explanation. DeepSeek V4 Pro is simultaneously classified as a Thinking, Thinking-Optional, Agentic Orchestrator, and Long-Context model. That may sound contradictory at first, but in practice it describes a very clear character: this model plans more visibly, writes more extensively than necessary, and often behaves like a strategist who would rather dispatch sub-agents but has to handle everything itself in the benchmark.

That softens some criticism. When a model like this fails to deliver the most elegant direct output on a strictly formatted single task, it is less embarrassing than it would be for a pure instruct model. Conversely, expectations for analysis, decomposition, prioritization, and depth are higher. That is precisely where DeepSeek V4 Pro is good — but not infallible. Its MoE design also means: what matters for performance evaluation is not the 1.6 trillion total parameters, but the 49 billion active parameters. The model therefore operates with considerably less active capacity than the raw total size suggests. For a Frontier model, top-tier performance is still a reasonable expectation. DeepSeek V4 Pro delivers that only in patches.

Performance: Workbench Rather Than Dialogue Partner

The Speed Profile Badge “Batch Tool Expert” already signals where this model belongs: in longer, less time-critical workloads where structure and endurance matter more than immediate responsiveness. The measured generation speed should therefore not be read as an abstract model value, but as the performance profile of the cloud provider OpenRouter for this endpoint. Such figures are infrastructure data from the provider, not merely an expression of model architecture.

In practice, this means: DeepSeek V4 Pro does not feel like a quick conversation but like a request to a calm back office. That is acceptable for documentation, longer transformations, or multi-step analysis. For interactive loops with many retries, agents with tight timeout windows, or users who grow restless after ten seconds, it is the wrong machine. The critical tail latency and the five dropouts across the full run matter more here than any average value. A model that occasionally shines but too often simply fails to respond reliably is not a sedan chair in practice — it is a bus with an irregular schedule.

Reasoning and Logic: Often Correct on Substance, Not Disciplined Enough on Compliance

In the reasoning module, DeepSeek V4 Pro reveals the core of its ambition. It solves logical tasks correctly with notable frequency, and even where scoring suffers, the reasoning error is not always the actual damage. A logged example illustrates this well: on the classic guards puzzle, the solution was factually correct. The model correctly identified the double inversion and named the right question. The catch: the visible answer came in English, even though German was explicitly required. That is not a marginal slip — it is instruction-following missed in plain sight.

Metacognition Compliance (Reasoning): The model refuses to use the explicitly requested <thought> tags in 3/5 metacog tests, citing a consistent policy statement. The reasoning content itself is partially correct — the score deduction results from format refusal, not from reasoning errors. For comparison: in the tag-free reasoning_5* tests, the model achieves an average score of approximately 65.71 percent, consistent with the run’s general performance level. CrucibleMark deliberately evaluates native zero-shot instruction compliance as a real-world characteristic — this deduction is methodologically intentional.

The two automatic language violations in the metacognition section sharpen this picture. In two tasks within the reasoning module, DeepSeek V4 Pro ignored the explicit language instruction and responded in English. This is not an exotic isolated incident but a structural signal: when language, format, and reasoning are required simultaneously, this model loses the language requirement first. Anyone integrating it into German-language production pipelines should not rely solely on logical correctness but should programmatically verify responses for language conformance.

Code Quality and Security: Wide Coverage, but Too Much Internal Overhead

In the code and security domain, DeepSeek V4 Pro performs better than its overall score might suggest. Its strengths lie in breadth and technical attentiveness. In an audit of insecure PHP code, the model identified not only the obvious vulnerabilities such as SQL injection, XSS, or path traversal, but also more implicit issues such as type juggling on the API key, session fixation, IDOR, and mail header injection. The table was cleanly structured, in German, and substantively sound. That is not a small thing. Many models see only the OWASP basics here. DeepSeek V4 Pro also catches the quieter flaws that become expensive later.

At the same time, a recurring weakness surfaces: the model often delivers the right list, but not always the best prioritization or the most useful synthesis. In the security example, the summarizing attack chain was missing, as was an explicit explanation of why the five implicit vulnerabilities are particularly dangerous in combination. It knows a great deal, but it does not always curate its findings the way a security review would require in practice. That is the difference between good finding and good advice.

There is also a structural efficiency problem. In the code quality module, DeepSeek V4 Pro produces an average of 9,626 tokens against a fleet median of 3,104. That is 3.1 times the benchmark average and even 1.6 times the allocated budget. The model does not solve these tasks concisely and precisely — it does so with very expensive internal elaboration. For API usage, this simply means: higher costs without proportionally better results.

Content Transformation: Strong in Execution, Weak on Discipline

The qualitative logs from the content transformation domain show DeepSeek V4 Pro at its most appealing. The model can deliver production-ready material that is immediately usable. The best example is a German-language tutorial script on two-factor authentication: realistic timestamps, cleanly placed pause markers, credible spoken-word tone, complete editor cues such as [SHOW], [CLICK], [CIRCLE], [ARROW], [B-ROLL], [MUSIC], [TEXT ON SCREEN], plus hook, pattern interrupt, troubleshooting, CTA, and even a community Easter egg. That is not merely correct. That is editorially usable.

Which makes the weakness all the more conspicuous: DeepSeek V4 Pro loses its grip under hard constraints. In one task in the content transformation domain, the model exceeded the explicit word limit of 250 words with 363 words — 145 percent of the limit. The system applied an automatic deduction of 20 percent, or 16.32 points, against the achievable sub-score. The substantive quality of the response is therefore irrelevant — the penalty applies regardless.

The length problem is not an isolated outlier. Across multiple tasks in the content transformation domain, the model shows a consistent pattern: when language, length, and format are required simultaneously, it drops the word limit as the first constraint. For creative or semi-structured editorial work, that is annoying but manageable. In automated pipelines with fixed fields, character limits, or publishing schemas, it is a genuine defect.

UX Writing and Documentation: Not Bad, but Rarely Elegant

In UX writing and documentation quality, DeepSeek V4 Pro delivers solid mid-range results — but without the assurance one would ideally expect from a Frontier reasoning model. The core problem is not a lack of language ability. It can write. The issue is rather that it too often expands where it should compress, and analyzes where it should decide.

For documentation, that can be useful — especially with more complex material. Anyone needing a first, comprehensive draft gets substance. Anyone needing precise user-facing text, concise help copy, or finely calibrated product microcopy is working against the model’s natural gravity. You sense the thinker here, not the copy engineer. That is not a damning verdict, but it is not a compliment either.

API Cost Profile

For a cloud Open Weights model via OpenRouter, verbosity is not merely a stylistic question — it is a bill. According to the model info, DeepSeek V4 Pro costs $0.87 per 1 million input tokens and $1.74 per 1 million output tokens. That is competitive for a Frontier model. However, it is partially offset by the model’s own tendency toward verbosity.

The overhead is particularly striking across several modules. In the Code Quality domain, the model generates an average of 9,626 tokens against a fleet median of 3,104 — a factor of 3.1 compared to the average across all tested models. In the CLI Benchmark, it produces 1,448 tokens against a median of 303, a factor of 4.78. In the Cultural Intelligence module, it reaches 1,639 tokens against 257, a factor of 6.38. UX Writing also stands out at 4,781 versus 1,689 tokens, a factor of 2.83.

Importantly, the benchmark does not penalize this verbosity directly. But an API user will, at the end of the month. When two models solve the same task with similar quality and one produces three to six times as much text to do so, that is no longer a character trait. That is a cost factor with literary ambitions.

Hallucinations and Content Reliability

The good news: DeepSeek V4 Pro’s weaknesses lie more in compliance, latency, and verbosity than in outright fabrication. In the available logs, the model does not stand out for wildly inventing nonsense. It fails more often by not cleanly scoping requirements or by inflating responses unnecessarily. That is the more tolerable way to lose. Incorrectly formatted truth is more repairable than fluently formulated nonsense.

Data Privacy and Data Sovereignty

Here things become uncomfortably clear for European organizations. According to the vendor card, DeepSeek is subject to the law of the People’s Republic of China — specifically PIPL/CSL/DSL. The stated data location is China + EU/US cloud partners. For users in Germany and Europe, this represents a relevant third-country transfer risk without an EU adequacy decision. A publicly documented GDPR DPA is not available. For organizations required to operate in GDPR compliance, this is not a detail — it is a potential exclusion criterion.

The data retention period is listed as -1 days, meaning it is effectively unclear. That is precisely the worst possible answer for compliance teams. Added to this is the calculated Sovereign Risk: HIGH. The rationale is explicitly not abstract: DeepSeek is a Chinese company, subject to Chinese national security law, and the BSI issued a warning on 04.02.2025 against using the DeepSeek cloud service, citing that user data is stored on Chinese servers and that use for official or sensitive data is not recommended. The weights provenance risk is also HIGH. The fact that the weights are openly available under the MIT License improves the licensing situation, not the cloud jurisdiction.

Conclusion

DeepSeek V4 Pro is an interesting, in parts impressive Frontier model with open weights, a large context window, and a clear reasoning ambition. It can solve complex tasks substantively, identifies implicit vulnerabilities in security audits, and occasionally delivers creative transformation output that is immediately usable. But it is also a model with manners problems: too slow in the tail, too unstable at the endpoint, too verbose for efficient API usage, and not reliable enough under hard language and length constraints.

For its architecture class in particular, the verdict is therefore mixed. As a Thinking and Agentic Orchestrator model, DeepSeek V4 Pro is entitled to think longer and work more strategically than a simple instruct system. That is its nature, not a flaw. But a Frontier model must translate its depth into controlled outputs. That is precisely where DeepSeek V4 Pro leaves points on the table. For batch analyses, longer editorial drafts, security scans with human review, and context-heavy tasks, it is usable to good. For unsupervised agents, time-critical processes, strictly formatted workflows, and sensitive enterprise data, it is the wrong choice in this cloud configuration. No notable hallucinations across all tests — the model prefers to invent little rather than embarrass itself with fabrication.

This evaluation was generated automatically based on the benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.