A Generalizing Approach to Protest Event Detection in German Local News

There are by now numerous automated or semi-automated approaches to the extraction of protest event data from news reports. Most, however, work with standard bag-of-words techniques and are based on national or international reporting. With this paper, we discuss the first German language resource of protest event related article excerpts published in local news outlets. We use this dataset to train and evaluate transformer-based text classifiers to automatically detect relevant newspaper articles. Our best approach reaches an F1-score of 81.6 %, which is a promising result. However, in a second experiment, we show that our model does not generalize well when applied to data from time periods and localities other than our training sample. To make protest event detection more robust, we test two ways of alternative article preprocessing. First, we find that letting the classifier concentrate on sentences around protest keywords improves the performance for in-sample test data up to +4 percentage points (pp), and for out-of-sample data even up to +8 pp. Second, against our initial intuition, masking of named entities does not improve the generalization of protest event detection models.