A Framework for Labelling Complex Political Concepts Using Open-Weight LLMs: Evidence from Illiberalism in Party Communication

Political scientists struggle to measure complex, contested concepts at scale, especially when these are rare, multidimensional, and expressed in nuanced language. This paper proposes and evaluates an end-to-end framework that uses open-weight Large Language Models (LLMs) to label such concepts in text, taking illiberalism and its related dimensions as a demanding test case. We analyse party communication on Twitter/X, focusing on illiberal rhetoric and neighbouring issue domains such as immigration, gender politics, populism, and power concentration. A zero-shot LLM first labels a tweet pool enriched for illiberal content; human coders then validate and correct these outputs to obtain multi-label annotations for seven illiberalism dimensions. On this basis, we compare three approaches: a fine-tuned BERT-style encoder, a fine-tuned open-weight Mixture-of-Experts (MoE) LLM, and a prompt-only GPT-oss model guided by a detailed codebook. The BERT encoder and the MoE LLM achieve macro-F1 scores comparable to large proprietary models, with the MoE LLM performing particularly well on context-dependent labels. The framework scales to large corpora, substantially reduces manual annotation needs, and yields continuous illiberalism indices that capture temporal and cross-party variation in rhetoric. We conclude that open-weight LLMs, embedded in a transparent multi-label pipeline, can deliver reliable large-scale measurement of complex political concepts. All code and data are available at \url{https://github.com/mv96/detecting_liberalism}.