‘Does this really work?’ Introducing an LLM-based workflow as a conceptual response to validity concerns in political institutionalism.

The advent of LLMs in political content analysis has been accompanied by significant concerns regarding measurement validity and transparency. Around the same time, debates started about the ability of established political institutional research to implement observer-invariant and transparent measurement methods. This article argues that scepticism towards AI-based measurements is warranted but, in the spirit of scientific rigour, should extend to human coding procedures, as suggested by these debates in the institutional field. This paper introduces a novel LLM-based workflow designed precisely to address these concerns by systematically documenting coding decision. By explicitly linking interpretive choices to identifiable textual evidence, the workflow enhances transparency and enables justification and open discussion of decisions within the research community. This capacity to trace, explain, and debate the logic of institutional coding makes LLM-assisted analysis not merely a technical innovation but a conceptual response to enduring questions of validity and intersubjectivity. Demonstrated through an evaluation of data from the Inter-Parliamentary Union, the workflow shows that transparency and interpretability can coexist with scalability, marking this automated workflow as a promising avenue for contemporary institutional research.