Governing the use of AI in the enforcement of social security

Administrative bodies frequently use AI applications to assist them in enforcing social security legislation. AI in the form of algorithms are used to generate risk profiles. Where a welfare benefits recipient receives a high score on the basis of a risk profile, they are selected for more intrusive surveillance. Algorithms are not immune to risks and their use can lead to undesirable societal outcomes. Such undesirable consequences occurred in the case of SyRI, where the use of AI in social security enforcement contributed to discrimination and infringements on the right to privacy. The District Court of The Hague decided that the SyRI legislation and system used on its basis violated Article 8(2) of the European Convention on Human Rights. SyRI has since been discontinued, however, the legislature recently drafted the Data Processing by Cooperation Act. This drafted legislation is before the senate and if adopted would enable information sharing and the use of algorithmic profiling systems similar to those provided for by the SyRI legislation. This raises questions about how to effectively govern algorithmic systems in social security enforcement. In answering this, we first examine what the object to be governed is, namely decision-making through the use of algorithms. Subsequently, we consider how algorithmic decision-making is regulated. Here we draw inspiration from European Union law, and in particular soft law and ‘guidelines’ such as the Ethical Guidelines from the European Commission. On the Dutch level, similar instruments such as the Ethical Data Assistant can also be observed. Then we will compare the SyRI legislation and case with the proposed Data Processing by Cooperation Act in order to examine what legal safeguards are used to regulate and govern the use of algorithmic systems. Based on our analysis, we will generate recommendations on how to effectively govern the use of algorithmic systems.