DeepSeek R1 Distill Qwen 1.5B

With an overall score of 35.41%, DeepSeek R1 Distill Qwen 1.5B demonstrates above all one thing: character is no substitute for maturity. As a reasoning model in the Nano class with 1.5 billion dense parameters, it is visibly tuned for multi-step thinking, yet in the standard mode tested here it is simultaneously constrained to direct instruction execution. The result is a peculiar mix of reasoning intent, formatting weakness, and linguistic discipline that regularly fails at the very first practical hurdle. The Speed Profile Badge reads Interactive DevOps Expert. That promises a rather interactive, brisk operational character and fits the basic concept of this local model better than its actual output quality does.

Header Grades: Stability and Reliability

The header grades are thus contradictory in an uncomfortable way. On average the model feels quite nimble, but at the tails it tears the workflow apart. For a Nano model, that is precisely the point at which theoretical lightweight status loses its appeal. Anyone expecting simple autocomplete or micro-tasks will often get text quickly. Anyone who needs reliability in agent chains or repeatable pipelines must reckon with rework and retries.

Architecture and Character: Reasoning Concept in a Nano Corset

The pre-assigned classification Reasoning, Instruct, Dense, Open-Weight, Local captures the essence with surprising precision. This model wants to think, but it lives in a body that is actually too small for that. The Dense architecture means here: all 1.5 billion parameters are active on every request. There is no MoE trick, no hidden specialization economy, no excuse about nominal versus active capacity. What you see is the model’s full capacity.

As a use case reasoning model, DeepSeek R1 Distill Qwen 1.5B must be measured by whether its additional reasoning passes lead to better answers. That is precisely where the problem begins. The test ran explicitly in standard mode, i.e., without the thinking toggle activated. This matters, because shorter and more direct answers are not a flaw in themselves here. But this caveat does not rescue the model from its central weakness: it often responds not concisely and precisely, but concisely and insufficiently. In several places one can see that a great deal appears to happen internally, while little of use arrives on the outside. That is not a sign of sovereign compression — it is more the digital equivalent of a student who stares at the page for a long time and then submits three half-finished sentences.

Speed and Token Profile

DeepSeek R1 Distill Qwen 1.5B was evaluated as a LOCAL model natively on NVIDIA DGX Spark (GB10 Grace Blackwell Superchip, ~115 GB Unified Memory — no practical memory limit for tested model sizes). Its Speed Profile Badge Interactive DevOps Expert signals fundamentally a model intended for immediate interaction rather than long batch writing. In practice this means: it often feels responsive, yet simultaneously produces enough spikes and detours that the nominally brisk impression does not translate into stable productivity.

The token profile reinforces this finding. In the CLI domain, the model produces on average roughly fifteen times the fleet median. Content Transformation and Code Quality also sit well above average. For a local model, such volumes of text are not a direct cost issue as they would be with an API, but they are a tangible latency driver. More text simply means more waiting time and more surface area for errors. It becomes particularly unpleasant when this verbosity is not paid for with better quality. That is exactly what happens here on multiple occasions.

Code Quality and Security: The Ambition Is There, the Precision Is Not

In security and code auditing, the model does not fail because it recognizes nothing. It does recognize something — just too little, too superficially, and often in the wrong form. In one documented security review it found ten vulnerabilities, while the reference cleanly identified nineteen. Critical topics such as IDOR, Session Fixation, CSRF, cookie flags, and reset token expiry were missed entirely or largely. Other vulnerabilities were captured but rated as technically too benign. Labeling Path Traversal as “Low” is not merely a slip. It is a judgment that causes real damage in practice.

In Security specifically, this is decisive. A model in this domain may be conservative. It may err on the side of warning too often rather than too rarely. DeepSeek R1 Distill Qwen 1.5B does the opposite: it sees known threats, but without the necessary depth of focus. Fix suggestions remain generic; concrete countermeasures or code-level repairs are frequently absent. The model thus sounds in places like someone who knows the chapter headings of an OWASP article but has not read the footnotes.

There is also a structural language problem. Across multiple tasks in the Code Quality domain, the model ignored the explicit language instruction and responded in English. This is not an isolated slip. It is a consistent pattern: when language, structure, and technical content are specified simultaneously, the model drops the language requirement first. In production environments with German as the target language, this is not a matter of style — it is a direct output failure.

In several Code Quality tasks an automatic constraint deduction was therefore applied for incorrect output language. Partial content matches then count only in a limited way, because the response has already missed the brief on formal grounds. The Judge was rightly uncompromising here. Anyone who orders a German security analysis and receives English table halves does not have a finished working document — they have raw material.

CLI and Tool Proximity: Much Internal Thinking, No Visible Result

The CLI module is where the downside of a small reasoning model shows itself most sharply. The model burned through its output budget on internal reasoning before any visible result reached the user. One task was formally marked as completed but produced no visible response text whatsoever. For the user, this is functionally identical to a failure. Whether the model reasoned internally, silently refused, or simply ran into the void is secondary. There was no assessable output.

In addition, a rule-based Hard Constraint finding applies: in one CLI task, reasoning tokens crowded out the output budget. For this model class, that is not an exotic defect — it is a real operational characteristic. For readers, this translates to: the model can burn its energy in the engine room while the answer fails to appear on deck. In agent setups this is fatal, because what counts there is not good intent but the visible, parseable result.

Reasoning and Logic: The Family’s Promise, Not This Run’s Performance

A reasoning model in Nano size does not need to be brilliant. But it must show that its thinking is more than decoration. In the reasoning tests, this succeeds only unreliably. Particularly revealing is the classic guard puzzle from the metacognition section. There the model responded in English instead of German and additionally delivered a logically incorrect solution. The proposed question did not produce the claimed consistency between liar and truth-teller. The Judge worked through this cleanly. The model failed.

This is more than an isolated error. It exposes the limits of distillation. The family carries the “R1” label — the aura of an inference model. But on 1.5 billion parameters, what remains of that in this run is often just the gesture. Much reasoning activity, little reliable endpoint. The model attempts to enter complex logical spaces but visibly struggles to exit them cleanly.

The language pattern recurs here as well. Across multiple reasoning metacognition tasks, the model ignored the German target specification and responded in English. Again: this is not an outlier. It is structural instruction weakness. Anyone who needs reasoning-capable responses in a firmly specified language will not find a reliable machine here, but one with variable compliance.

Two reasoning tasks revealed a sharper defect still. The model reported success but produced no visible text. At the same time, the rule findings show that internal reasoning tokens completely crowded out the available output budget. The score suffers here not from incorrect content, but from the fact that nothing was on the table for the user at the end. A model that thinks and stays silent is useful only to philosophers.

UX Writing: Surprisingly Weak, Even Though the Task Is Essentially Instruction-Proximate

Precisely because DeepSeek R1 Distill Qwen 1.5B is classified alongside its Reasoning tag as an Instruct model, the UX domain is particularly painful. Such tasks require no encyclopedic world knowledge — they require clean adherence to structure, tone, and language specification. That is exactly where the model fails. In one documented UX task it delivered, instead of the required analysis plus optimized flows, only a fragile table filled with repetitive filler phrases. Psychological rationale — the actual value-add of an “expert-level” UX output — was entirely absent.

The Judge puts it aptly: this is underdeveloped work, not optimized brevity. That formulation lands because it hits the core. A small model may be concise. But it may not pretend that incompleteness is a design decision. Here the analysis phase, depth, and often linguistic precision are all missing.

The language specification also failed to hold in the UX module. In two tasks the model responded in English instead of German. In production use without post-review, this is an immediate disqualifier. UX copy is not raw material to be smoothed out later with a translator. Tone, rhythm, and micro-decisions are bound to the target language.

Content Transformation: Formal Intent Without Directorial Sense

In the Content Transformation domain the model is more interesting than in UX, but not better. It visibly attempts to structure scripts and transformation tasks. The problem is that it leaves central mandatory elements behind. In a video script task the prescribed analysis phase was entirely absent. The response jumped directly into an incomplete transformation — in English rather than German. Individual production markers were present, but the overall text was not broadcast-ready; it was more a storyboard fragment in draft mode.

The length problem here is not a single accident. Across multiple tasks in the Content Transformation module, the model shows a consistent pattern: when language, length, and format are specified simultaneously, it drops the language requirement first. Four documented language errors in this module are not statistical noise — they are a clear architecture and fine-tuning signal.

This domain is also unpleasant from a token-economy perspective. The model produces nearly twice as much text as the fleet median without delivering visibly more precision or creativity in return. This is the worst kind of verbosity: it costs time but does not save the answer.

Documentation Quality: Formally Usable, Linguistically Shaky

Documentation tasks suit the model somewhat better than UX or rigorous security analysis, but the same thread runs through here too: sometimes a usable approach in terms of content, formal and linguistic discipline unreliable. In four documented documentation tasks the model ignored the explicit language instruction and responded in English. This immediately turns a potentially useful draft into a rework case.

The language failure is not an isolated outlier. Across multiple tasks in the Documentation Quality domain, the model shows a consistent pattern: when technical language, format, and target idiom are required simultaneously, it drops the language specification first. For internal notes this may still be tolerable. For publication-ready documentation it is poison. Documentation lives on reliability. This model delivers material rather than results.

Cultural Intelligence: A Bright Spot, but a Murky One

The Cultural Intelligence domain is one of the few places where the model at least partially shows what its underlying concept might be: it recognizes the thrust of a task and attempts to rework toxic or exclusionary language. The catch is that the rework goes linguistically off the rails. One documented example of a job posting mutated into a text full of invented words, grammatical breaks, and English insertions. Formally it was German. Practically it was unusable.

This is a remarkable pattern: the model is not hallucinating here in the classic sense of fabricating facts into a void. Rather, it produces linguistic wreckage from half-understood fragments. That is almost worse, because at first glance it looks like processing, but collapses on reading. At least a small point of respect remains: the model does attempt to engage with problematic tonal registers at all. It simply lacks, at this size class, the linguistic control to produce publishable text from them.

Data Privacy and Data Sovereignty

Since this is a purely locally operated Open Weights model, the focus is not on a cloud provider’s runtime data processing but on the provenance of the weights. The weights originate from the DeepSeek/Unsloth lineage; the developer context is China, and the weights provenance risk is marked MEDIUM. For regulated environments in Germany and the EU this is not an automatic disqualifier, but it is a checkpoint: not the execution, but the origin of the model artifacts is the relevant sovereignty question here.

Conclusion

DeepSeek R1 Distill Qwen 1.5B is a fascinating small model, but not a good one. It carries the gestures of an inference model, the command register of an instruct model, and the practical freedom of open weights. It simply converts these assets into reliable results in the benchmark far too rarely. Language instruction compliance is weak across multiple modules. Security analyses remain superficial. Reasoning partly dissipates off-screen without leaving any visible user-facing text. For a Nano model, some of this is forgivable. The sum of it is not.

Its sensible deployment range is therefore narrow: local experiments, near-embedded scenarios, simple single steps, perhaps as a cheap reasoning primer ahead of a stronger reviewer. It is not suitable for unsupervised agents, production security analysis, German-language publishing workflows, or tasks where format and language fidelity are non-negotiable. Across all tests, no notable hallucinations. The model prefers to invent little rather than to be convincingly wrong, but fails instead on language, depth, or completeness. Compared to larger runs from the same distill family, the key takeaway here is: you can shrink reasoning surprisingly far. But in doing so you can also thin it out until only the pose remains.

This evaluation was generated automatically based on the benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.