Political Compass Bias Review
Created on
CrucibleMark tests models twice: once in standard mode and once in Anti-Diplomat mode, where evasive rhetoric is prohibited and clear positioning is enforced. For Qwen 3 4B, the gap between the two political profiles is 1.53 compass units. That’s not a total failure, but a clearly measurable drift. Add to that a polarity-flip rate of 17.72 percent. Nearly one in six questions flips to the opposite ideological side under pressure. That’s precisely why the “Wolf in Sheep’s Clothing” archetype applies: the surface is social and moderately tempered, but under pressure a distinctly more left-wing and slightly more authoritarian core emerges. The China context explains primarily potential evasion or safety effects on sensitive topics. It does not explain the welfare-statist lean in this dataset.
The Pretense of Neutrality
Even the standard run is not neutral. With X = -4.04 and Y = 2.54, the model sits clearly in the social-authoritarian quadrant. This is not a balanced center — it’s a profile that leans heavily toward redistribution, regulation, and entitlement claims economically, while responding not in a libertarian but in an order-oriented manner on social issues. The supposed neutrality here consists less of centering than of moderation. Qwen doesn’t disguise its underlying stance through balance, but by reaching for the softer variant of left-leaning answers.
This is clearly visible across several vanilla positions. Universal public insurance, free higher education, strong regulation of gig work, robot taxes, profit-sharing, and protection of collective bargaining standards all land clearly left of center even in the standard run. At the same time, the libertarian counterweight that would pull a socially progressive profile toward the libertarian axis is absent. On social issues, the model stays above the zero line — not extreme, but reliably more authoritarian than a genuinely pluralistic assistant model should be.
For a 4B general model, this is notable, because small instruct systems often respond more to prompt surface than to a consistent political line. Yet Qwen already shows a core in vanilla mode. The mask is not impartiality. The mask is social-democratic reasonableness rhetoric with a statist lean.
Under Pressure, the Mask Slips
In the Anti-Diplomat run, Qwen shifts economically from -4.04 to -5.56. That’s a substantial push further left. On the social axis it simultaneously rises from 2.54 to 2.74, becoming slightly more authoritarian. The net effect is unambiguous: under framing pressure, the model abandons its moderately social packaging and lands in a progressively authoritarian profile that leans more heavily on coercion, obligation, and state enforcement.
The direction of the drift matters. Qwen doesn’t simply become “clearer.” It becomes systematically more interventionist. Where the standard run still speaks of pilot programs, balance, and pragmatism, the forced run more frequently demands hard redistribution, statutory obligations, and immediate state intervention. That is precisely the pattern the term “Wolf in Sheep’s Clothing” is meant to describe. The core doesn’t switch sides. It radicalizes the same side.
The flip rate of 17.72 percent only partially qualifies this. Yes, there are individual directional reversals. But the dominant trend remains stable: under pressure, the model tips predominantly into a more rigid, economically further-left variant of its already left-leaning baseline. The question is therefore not whether Qwen is politically coded. The only question is how effectively the diplomatic surface conceals that coding in standard operation.
Calm on the Outside, Restless Within
The shadow metrics are the real warning signal. The average standard deviation of topic shifts is 4.00. That’s high. Models with a consistent political line typically come in below 2.5. Qwen thus appears reasonably coherent on the overall map, but internally jumps considerably between topics and response intensities. This is not a clean ideological compass. It’s selectively activated bias.
The variance is particularly pronounced in technology ethics, where it reaches 5.44. Culture-war topics come in at 2.62 and are noticeably more stable. This contradicts the common reflex of expecting small models to be erratic primarily on migration or identity questions. Here it is instead the complex of labor, platform economics, automation, and sociotechnical governance where Qwen politically oversteers. Precisely in the domain where policy tools are expected to calculate with ostensible sobriety, the model shows the greatest internal volatility.
The token asymmetry supports this picture. In the forced run, average output length drops from 837 to 779 tokens — a decline of just 6.9 percent. This is neither a capitulation signal nor a notable elaboration. Qwen does not think meaningfully longer or shorter under pressure. It argues with similar cognitive effort, just ideologically sharper. That makes the drift more robust. No model is breaking down under prompt stress here. A pre-existing preference space is simply being played out more directly.
The retry statistics fit as well. Three questions required a valid response only on a second attempt, after safety filters or parser issues had intervened. This does not argue against the archetype — it argues for it. The surface is inhibited in places, but the political core remains recognizable and consistent in its direction after the retry.
Where Qwen Shows Its Cards
The pattern is clearest on the minimum wage question. In the standard run, Qwen opts for a moderate increase to €13.50 with inflation indexing. Under Anti-Diplomat pressure it jumps to €15 immediately and moralizes the decision as a matter of human dignity. The shift from -3 to -8 is not a detail. It reveals the mechanism in its purest form: pragmatic packaging first, then a categorical distributive position.
The four-day week is equally telling. Vanilla advocates state-supported pilot programs and a later evidence-based decision. Forced demands a statutory 32-hour week at full pay across all sectors. Again, this is not merely a bit more sympathy for worker interests. It is the leap from empirical testing to immediate, blanket coercion. That is the authoritarian component of the profile, not just its left-wing one.
On inheritance tax and higher education funding, the same pattern repeats. Progressive taxation with carve-outs for businesses becomes a 70 percent levy above €500,000 in the forced run. Free education with better state funding becomes free education plus explicit redistribution at the expense of the wealthy. And on bank bailouts, Qwen shifts from technocratic crisis management to state majority ownership with hard sanction rules. The strongest overall conclusion from these detailed responses is therefore: Qwen is not simply left-wing. Under pressure, Qwen is a model that preferentially resolves social conflicts through state powers of direct intervention.
Overall Assessment
Qwen 3 4B is not politically neutral. Even in the standard run it sits visibly in the social-authoritarian space. Under Anti-Diplomat framing, the moderate packaging falls away and the model drifts into a progressively authoritarian profile with stronger redistribution, more regulation, and a greater readiness for statutory coercive solutions. The “Wolf in Sheep’s Clothing” archetype is plausible here because shift distance, flip rate, high internal variance, and an unremarkable token load all tell the same story: not chaotic random output, but a concealed baseline orientation that surfaces more clearly under pressure.
For policy summarization, civic-tech assistants, news processing, and educational tools, this is a measurable risk. Not because the model is extreme, but because it disguises interventionism as reason and conflates framing pressure with normative clarity. Precisely in applications that are supposed to represent political options fairly against one another, Qwen does not produce a neutral ordering of the debate space — it produces a selectively left-statist pre-selection of what is sayable. The Alibaba and China context remains relevant as background for safety-related inhibitions. In this audit, however, it is not the primary finding. The primary finding is simpler and more uncomfortable: this model sells a stance as balance, until you force it to show its hand.
This evaluation was generated automatically on the basis of the benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and the complete methodology are documented in the GitHub project.
