Skip to content
CrucibleMark

Upstage Solar Pro4

Solar Pro 4 is Upstage’s agentic flagship model, released on August 6/10, 2026, with an undisclosed parameter count. It is available as a proprietary cloud service (additionally as dedicated/on-premises deployment for enterprise customers) and is designed for multi-step agentic workflows across documents, terminals, and tool calls. The model offers a context window of 524,288 tokens with up to 131,072 output tokens, supports English, Korean, and Japanese for input and output, and allows a configurable ‘Reasoning Effort’ (high for deep analysis, low for real-time chat speed).

Upstage Version pro4 Commercial use permitted Dense 524 K Context 02/2026 $0.03 / $0.12 per 1M

  • Proprietary
  • Frontier
  • OpenRouter
  • Text
  • Long Context
  • Agentic Orchestrator
  • Interactive

Sovereign Risk: MEDIUM Upstage is a South Korean company. The risk is rated as medium, as the exact hosting terms and the potential applicability of foreign laws (e.g., the US CLOUD Act, if hosted via US cloud infrastructure) are not publicly documented. Enterprise customers can contractually arrange dedicated or on-premises deployments, which can significantly reduce the risk for this user group.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 60 76
HTTP Fetch & Extract 80 60 70
Tool Failure Handling (404) 40 15 28.3
Web Search & Tool Selection 100 100 100
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 15 57.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
2.4
First Request
MCP
1.31
Protocol Latency
Synthesis
25.71
Response Generation
Total
176.5
Sum of All Phases
Token
13772
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Updated · Long Context · Agentic Orchestrator

Deployment Verdict

Conditional deploy, because tool execution is strong, but detected hallucinations on invalid tool behavior and a Combined Score of 68.62 make the model currently unsafe for unattended MCP pipelines.

Tool Execution Profile

Upstage Solar Pro4 demonstrates genuine tool intelligence, not just rigid procedural behavior. On the Web Search and Tool Selection test — which checks the choice between search and direct retrieval without an explicit hint — it reliably selects the correct tool. This points to usable planning logic in dynamic pipelines. Performance on the URL Construction test, which measures the autonomous derivation of a target URL and the subsequent fetch, is solid but not deterministic enough for every production chain. The P1 score of 90 supports this picture: high operational capability, but no consistently protocol-clean execution. Critically, the tool call was not valid overall. This is not a mere cosmetic flaw but an MCP risk, because an orchestrator must be able to rely on formal correctness. The only positive note is that no retry was required; the problem therefore lay in execution rather than in an unstable formatting loop.

Synthesis Fidelity

How well does it condense tool results? Only limitedly reliable. The P2 score of 66.67 looks workable at first glance, but asset variance is high. Web Search & Tool Selection also performs very well in condensation, while Multilingual Search & Synthesis drops off sharply when merging cross-lingual research in German. For pipelines that need to produce reliable summaries from tool outputs, this is too inconsistent.

Does it stay within the tool result or fall back on training data? In the Honeypot EU License Research test — designed to force current license restrictions from web sources rather than training knowledge — it stays within the retrieval chain and does not hallucinate. This is the strongest trust signal in the review. At the same time, the overall hallucination finding remains a security risk: once a model outputs fabricated facts as a tool result, the entire tool infrastructure loses its verification value.

Error Resilience

The model clearly fails here. On the 404 test, which checks for transparent behavior when a tool call fails, Solar Pro4 fabricates page content instead of cleanly surfacing the error. This is production-critical without exception. A pipeline can work with explicit errors. It cannot safely work with plausible-sounding substitute content.

Operational Profile

Total 176.50s per run. Call 1: 2.40s. MCP latency: 1.31s. Call 2: 25.71s. Slow for this level of output quality. Cost/run: local. Price per model profile is very low, but runtime depresses practical efficiency.

Conclusion & Recommendation

Suitable for supervised agent pipelines with a strong guardrail layer, explicit tool output validation, and hard error handling before every synthesis step. Not suitable for autonomous MCP workflows, compliance pipelines, or retrieval chains where a tool failure must propagate transparently. If you deploy it, use it as a planning-capable orchestrator with downstream verification — not as a trusted final authority.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.