Measuring Party Similarity: A Comparative Evaluation of Metrics

The literature on party positioning exhibits a rich diversity of methods, ranging from expert surveys, through human-coded and automated analysis of party programs, as well as statistical analysis of roll call voting records and electoral patterns, to social network analysis. Advances in machine learning and big data availability open further new vistas. But do all those methods really reflect the same concept of party proximity? Is there even a single such concept? And -- assuming there is -- can we quantitatively assess the accuracy of different methods in measuring that concept? In our paper, we seek to answer those questions by computing -- for several EU countries, including Germany, France, Italy, Spain, Poland, and Belgium -- a large variety of party similarity measures and exploring their correlation matrix using both parametric and non-parametric statistical methods (including PCA and factor analysis). The measures to be tested include not only classical ones, such as similarity of MARPOR topic distributions, NOMINATE scores, or CHES position scores, or cosine similarities of manifesto word distributions, but also those based on manifesto embeddings, transformer-based language models (BERT, GPT), program and parliamentary speech styles, coalition formation patterns, legislative cosponsorship networks, candidate transfer graphs, and social network patterns.