Mistral Small 4

With an overall score of 73.3 percent, Mistral Small 4 emerges as a surprisingly affordable cloud all-rounder from the Mistral API, carrying the speed profile badge “Real-Time DevOps Expert.” The label fits: the model responds quickly, stays concise, and comes across in many tasks like a colleague who delivers promptly but doesn’t always put in the final polish. Its metadata promises a lot at once: generalist ambition, vision capability, a long context window, instruct character, and agentic suitability. In the text-only benchmark, three things stand out clearly: disciplined output, serviceable logic, and a noticeable weakness the moment precision in tool use or strict format requirements matter more than fluent prose. Sovereign Risk: LOW — Mistral AI is a French provider under EU jurisdiction with EU data residency, and no CLOUD Act exposure under the current provider structure.

Header Scores: Stability and Reliability

It matters that this run was conducted under the endpoint’s default out-of-the-box behavior. There is no switchable thinking mode here. Mistral Small 4 was therefore evaluated exactly as a regular API user would actually receive it. For a commercial cloud model, the combination of zero timeouts and short tail latency is more than mere hygiene. It is a genuine practical advantage, especially in agentic workflows where a single dropout can block entire chains.

Architecture and Classification

The pre-assigned category captures the model’s character surprisingly well. As a General model, Mistral Small 4 must function broadly, not merely shine in a niche. As an Instruct model, one can expect direct, fairly compact responses. That is exactly what it delivers: few detours, little self-talk, rarely token-hungry prose. The Long-Context label remains mostly background noise in the present text benchmark. The 256K context window is impressive on paper, but most tasks here fall well below that threshold. What you see instead is whether the model thinks in a structured way — not whether it can elegantly digest entire filing cabinets.

The two most interesting labels are Vision-Capable and Agentic. The former inevitably relativizes the benchmark, because a text-only course measures only part of the model’s competence. Evaluating a multimodal model without giving it images tests only the language leg. That is legitimate, but incomplete. Agentic, in turn, does not automatically mean the model excels at tool tasks. It means, rather, that the model is designed for multi-step workflows and tool use. That is precisely why the weak tool-use scores here are all the more uncomfortable. An agentic model may stumble on standalone direct responses. But it should not start hallucinating when handling tool results.

There is, however, a hard contradiction in the supplied metadata that must be named. Mistral Small 4 was curated as use_case_primary = generalist, size_class = Frontier, parameter_architecture = moe. The model info simultaneously states: 119 billion total parameters, 6.5 billion active, MoE — making this, in practice, a model whose performance benchmark should be calibrated against the active portion, not the imposing total count. That is precisely what explains the character of the results. Mistral Small 4 does not perform like a massive heavyweight thinker, but like an efficient specialist on call. Lots of shell, comparatively lean active core. That is not a flaw. It is the architectural idea.

Performance and Cost Profile

The speed profile badge “Real-Time DevOps Expert” is not decorative — it is a precise shorthand. Mistral Small 4 is, in practical terms, fast enough for interactive use: chat, assistance, revisions, and quick technical queries. Add to that a very aggressive price of $0.15 per 1 million input tokens and $0.60 per 1 million output tokens. For a commercial cloud model, that is a statement. Mistral is not trying to be the smartest model in the room here. It is trying to be the model you call frequently without a guilty conscience.

The token economy is also noteworthy. No module exceeds the expected verbosity range. On the contrary: Reasoning and Metacognition average 770 output tokens, well below the fleet median of 1,412; CLI at 150 versus 303; Code Quality at 1,897 versus 3,104. Even where it marginally exceeds the median — Documentation Quality at 3,132 versus 3,110 tokens — that is not an outlier but rounding noise. Put differently: Mistral Small 4 rarely writes more than necessary. For an API model, that translates directly to money.

This efficiency has its content-side cost, however. Brevity often reads as a virtue here, but sometimes as a cost-cutting measure in the wrong place. The model rarely produces visible filler. But it occasionally omits the depth that makes the difference between serviceable and robust.

Code Quality: Solid Craftsmanship, but No Uncompromising Auditor

In the Code Quality module, Mistral Small 4 reaches 70.92 percent. That is not a collapse, but it is not the kind of security instinct that lets you sleep soundly at night either. The qualitative record shows a fairly typical picture: the table is clean, the language is clean, the structure is sound, and the most important classes of vulnerabilities are recognized. In a PHP security analysis, the model found 15 of 19 vulnerabilities — roughly 79 percent coverage. What it identified was largely correctly classified and usefully explained. That is the good news.

The bad news is sharper. Precisely among security-critical gaps, things were missing that you cannot afford to overlook in a real audit: an SQL injection in a DELETE query, hardcoded database credentials, a reset token without an expiry time, insecure cookie flags, and a header injection issue after output had already been sent. There were also severity miscalibrations: an IDOR vulnerability was rated too benign, as was path traversal. These are not cosmetic slip-ups. These are prioritization errors. Anyone serious about security needs not just hits, but the right hits in the right order.

At the same time, the instruct nature of the model shows here. Mistral Small 4 delivers clean, prompt tables with compact fixes. It writes like a conscientious reviewer, not like a paranoid red-teamer. That can be pleasant in everyday development. For deep security review, it is not enough. The Achilles’ heel is not nonsense — it is insufficient depth.

Reasoning and Logic: Correct, Compact, Rarely Brilliant

In the Logical Reasoning area, Mistral Small 4 scores 72.47 percent. That is respectable, especially when the MoE architecture is fairly measured against the active portion. The qualitative probe with the two-guards puzzle shows: the solution is correct, the logic is correct, the response stays in German, and even the required <thought> format was cleanly fulfilled in this example. The model correctly works through the classic double-inversion trick and reliably arrives at the right result.

What is missing is the second layer. The model abstracts less elegantly than stronger reasoning models, does not always name principles explicitly, and builds less visual or didactic structure. It explains correctly, but without the intellectual generosity that suddenly makes difficult connections feel easy. You get the solution. You do not necessarily get the best explanation for a third party.

For the assigned category, this is nonetheless consistent. Mistral Small 4 is not a dedicated thinking model but a generalist with an instruct temperament. In that light, the performance is good. The model thinks clearly enough, remains economical in doing so, and avoids the kind of sprawling self-absorption that occasionally suffocates other systems. Those seeking deep derivations, alternative proof paths, and didactic elegance will need to shop elsewhere. Those wanting correct, compact logic responses are decently served here.

Content Transformation and UX: Useful, but Not Always Obedient

In Content Transformation, Mistral Small 4 sits at 72.61 percent. On one hand, this signals that the model handles structured rewriting. On the other, the logs reveal exactly the friction that keeps a good assistant from becoming a reliable production writer. On an elaborately staged YouTube script task, the model delivered almost everything: analysis, timestamps, spoken-word tone, screen directions, production cues, hook, CTA, Easter egg. Content-wise it was complete and technically clean. What was missing was the fine-tuning. Dramaturgical precision stayed below the reference level, visual direction remained coarser, and — most importantly — the specified length window was exceeded.

In one Content Transformation task, the model exceeded the explicit word limit of 250 words by 25 percent. The system applied an automatic deduction of 16.80 points, or 20 percent of the achievable sub-score. The content quality of the response is therefore irrelevant — the penalty applies regardless. Hard constraint violations like this are more than mere sloppiness. They reveal which rule the model drops first under load. Here, it is the word limit.

This is not an isolated outlier. Across multiple tasks in the Content Transformation area, the model shows a consistent pattern: when simultaneous constraints on language, length, and format are imposed, the word limit is the first condition to go. The video script task ran too long; another task incurred an automatic penalty score for a clear overage. Mistral Small 4 can rewrite. It can even rewrite quite smoothly. But when multiple reins are applied at once, it tends to drift past precision rather than past meaning. Pleasant for the reader, risky for production pipelines.

In UX-adjacent writing, this behavior fits the pattern. The model often sounds reasonable, sometimes professional, occasionally a bit too eager to explain. Instruct models in particular should be obedient when told to deliver “only the final text.” Mistral Small 4, however, indulges in conspicuously human habits here: it knows what is meant, and comments anyway.

Cultural Intelligence: Content-Sensitive, Not Always Formally Disciplined

In the Cultural Intelligence area, Mistral Small 4 reaches 71.44 percent. The qualitative example is revealing. When rewriting a toxic job posting into correct, inclusive German, the model got much of the content right. It neutralized aggressive language, eliminated gender bias, and formulated the result in a professional manner overall. The cultural instinct is present. The model knows how modern, German-language HR communication should sound.

The point deduction came not from the content but from discipline. The task required only the rewritten text. Mistral Small 4 additionally delivered several paragraphs of justification. That is not a stylistic footnote — it is a straightforward violation of the task format. The LLM Judge correctly flagged this as a critical structural error. To put it less politely: the model talks when it should be silent. For culturally sensitive reformulations, that is frustrating, because in precisely those contexts format requirements are often not decoration but compliance.

What is interesting is that the linguistic quality itself was not the problem. The German is clean, natural, and contextually grounded. Mistral Small 4 does not fail here on language sense — it fails on self-discipline. That is the better way to lose, but losing is still losing.

Documentation and Everyday Assistance: Solid Middle Ground with Serviceable Pragmatism

With 76.4 percent in Documentation Quality, the model shows one of its more appealing sides. It apparently writes with enough structure to be useful without getting lost in endless digressions. The fact that token usage in this module tracks almost exactly with the fleet median fits accordingly. No unnecessary foam, no stingy telegram prose. For knowledge synthesis, factual explanatory texts, and standard documentation, this is a sensible working tool.

This is precisely where the combination of instruct temperament and an actively lean MoE capacity pays off. Mistral Small 4 rarely tries to impress the reader with brilliance. It tries to deliver a usable, organized response. That sounds like faint praise. In everyday use, it is often the more valuable quality.

Tool Use, Hallucinations, and Security Risk: This Is Where It Gets Serious

The real warning sign is not in the general language performance but in the tool domain. The ToolUse score is only 20.33 percent, and the synthetic tool execution score is 39.17 percent. For a model we also classify as agentic, that is clearly too weak. The problem is not merely a lack of elegance in tool calls — it is a concrete hallucination finding.

In one tool-use task, the model hallucinated content that did not originate from the retrieved tool result but was freely invented. As a result, the P2 score was capped by a hallucination cap. For content-critical tasks such as research or fact-bound reporting, this is a disqualifying signal. What fails here is not just format compliance. What breaks down is the foundational rule of agentic systems: use the tool, trust the tool, do not invent beyond it.

This is the decisive flaw of this model. Mistral Small 4 is fast, cheap, and often solid as a cloud assistant. But the moment external facts or tool returns form the ground truth, it requires supervision. An agentic model that hallucinates while reading tool output is like a courier who repackages parcels in transit. You cannot send it out unattended.

Data Privacy and Data Sovereignty

For a commercial cloud model, the data privacy situation here is comparatively favorable. Mistral AI SAS is headquartered in Paris, is subject to EU law according to the vendor card, and holds data within the EU. The calculated Sovereign Risk is LOW. For users in Germany and Europe, that is a real advantage, as neither US CLOUD Act structures nor an unclear non-European jurisdiction are in play.

What remains important: usage occurs via the vendor’s cloud, not on your own infrastructure. According to the vendor card, standard data retention is 30 days and a GDPR DPA is available. For organizations with GDPR obligations, that is the relevant information. It does not make deployment automatically worry-free, but it is considerably more manageable than with many US providers. The weights provenance risk is also LOW, which fits the deployment situation here and introduces no additional sovereignty concern.

Conclusion

Mistral Small 4 is a model with a clearly recognizable character. Fast, affordable, cloud-stable, token-efficient, and in many standard tasks refreshingly sensible. For a MoE system with only 6.5 billion active parameters, it performs convincingly in text work, documentation, and general assistance. The model has no star attitude. It works.

But that is precisely why its weakness in tool use carries weight. Anyone who needs to reliably carry facts forward from tools will not find dependable assurance here. Add to that recurring problems with hard format constraints — especially word limits and instructions to deliver no additional commentary. That is not catastrophic, but in production it is exactly the kind of error that generates tickets, retries, and post-hoc checks.

On balance, Mistral Small 4 is a strong price-to-performance offering from the Mistral API for general productivity tasks, quick technical assistance, documentation work, and robust interactive use. For security audits, the depth falls somewhat short. For autonomous research or tool pipelines, the reliability falls shorter still. Those looking for an affordable, alert cloud generalist will find considerable value here. Those who want to hand an agent the keys to the tool cabinet should hold off for now.

This evaluation was generated automatically based on the benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.