Skip to content
CrucibleMark

DeepSeek V4 Pro

DeepSeek V4 Pro is the flagship of the V4 line, designed for reasoning, coding, and agentic workflows. The hybrid attention MoE architecture combines 1.6 trillion total parameters with 49 billion active parameters per token and a context window of one million tokens. The model is available as an Open Weights model under the MIT license, though Chinese jurisdiction in cloud deployments requires a separate privacy assessment.

DeepSeek Version 4 Commercial use permitted MoE 1600 B (49 B active) 1000 K Context 05/2025 $0.435 / $0.87 per 1M

  • Open Weights
  • Frontier
  • OR
  • Text
  • Agentic Orchestrator
  • Long Context
  • Interactive

Sovereign Risk: HIGH DeepSeek is a Chinese company subject to China’s National Security Law (NSL), which may allow state access to data and models. On 04.02.2025, Germany’s BSI explicitly warned against using the DeepSeek cloud service: user data is stored on Chinese servers; use for official or sensitive data is not recommended. This warning applies without restriction to cloud API deployments.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 80 60 70
Tool Failure Handling (404) 80 80 80
Web Search & Tool Selection 100 60 82
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 60 80

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
4.2
First Request
MCP
0.8
Protocol Latency
Synthesis
16.99
Response Generation
Total
131.93
Sum of All Phases
Token
5397
Input + Output
Cost
$0.0032
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

· Agentic Orchestrator · Long Context

Deployment Verdict

Conditional deploy, because DeepSeek V4 Pro produces valid tool calls, showed no hallucination during the run, and with a Combined score of 76.00 clearly appears production-ready — however, synthesis quality remains too uneven for verification-critical pipelines.

Tool Execution Profile

Tool execution is strong. The model operates MCP-compliant, the tool call was valid, and no retry was needed. The decisive factor is tool selection: on the Web Search & Tool Selection test — which checks without explicit hints whether the model searches first rather than fetching directly when current information is required — it makes the right call confidently. This argues against mere schema-following and in favor of usable tool intelligence in dynamic workflows.

Less clean is the precision on the URL Construction test, which checks whether the model can derive the target URL from its own knowledge and then fetch it correctly. P1 80 is good, but not strong enough for strictly deterministic pipelines with hard URL schemas. For search, research, and routing steps the model is reliable. For direct, silent URL derivation without validation, the pipeline should put guardrails in place.

Synthesis Fidelity

How well does it consolidate tool results? Solid, but not consistently precise enough. P2 63.33 shows that the model merges results in a usable way most of the time, but with a noticeable loss of sharpness. This is especially visible on EU License Research with P2 40 and on several search assets with only middling consolidation. For analyst workflows this is acceptable. For compliance, legal, or policy summaries, post-review is required.

Does it stay within the tool result or fall back on training data? On the EU License Research honeypot — which tests exactly this temptation — the model stays formally on the safe side: Content Verification State A, no detected hallucination. The trust signal is therefore better than the P2 score would suggest. The issue here is not fabrication, but imprecise consolidation of current sources.

Error Resilience

Acceptable for production. On the 404 test — which checks whether a failed tool call is openly acknowledged or papered over with fabricated content — the model communicates transparently. It does not hallucinate page content despite the error. This behavior is precisely what keeps a tool pipeline trustworthy, even when the response flow is interrupted.

Operational Profile

Call 1 4.20s. MCP latency 0.80s. Call 2 16.99s. Total 131.93s. Clearly slow. Cost per run 0.003245. Cheap to very cheap for a Frontier reasoning model. The price works; runtime is the real trade-off.

Conclusion & Recommendation

Suitable for MCP-assisted research, analysis, and multi-step reasoning pipelines where correct tooling matters more than low latency. Not the first choice for high-frequency user interaction, strictly deterministic fetch flows, or any pipeline where the final synthesis itself must hold up as reliable evidence. Those deploying it should trust tool use directly, but for sensitive domains should back the final consolidation with citation binding, field extraction, or a second verification step.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.