Benchmarking Political Responses of Frontier LLMs
Key Takeaways
- 1Developed benchmark for LLMs on political stances
- 2Refusals now scored as conservative responses
- 3KIMI K2 displays stronger opinions than Western models
Recent research introduced a benchmark to evaluate frontier LLMs, including GPT-5.3, Claude Opus 4.6, and KIMI K2, on a political compass. Utilizing 98 structured questions across 14 policy areas, the project scored political refusals as conservative responses, providing a new perspective on model biases. Remarkably, the results indicated that while KIMI K2 displayed strong political opinions, GPT-5.3 refused to answer nearly all questions when given an opt-out option.
The implications of these findings are significant for understanding LLM behavior and potential biases in output. KIMI K2's consistent responses contrast sharply with the caution exhibited by Claude Opus and GPT-5.3, particularly as the latter's refusals correlated with a Right-Authoritarian classification when pressured for opinions. This research underscores the impact of design choices in AI systems, indicating that how models are prompted can substantially affect their political orientation and perceived autonomy in discussion of sensitive topics.