Mind the Gap: The Role of Open Data and Big Data in Public Management Reform

This paper discusses the relative merits of 'open data' and 'big data' initiatives, exploring the synergies and tensions between them. Over the last five years, many governments have embarked on 'open government' initiatives, freeing up hundreds of data sets for public and commercial use and forcing departments and agencies to make detailed expenditure data publicly available. Such initiatives have obvious potential to go beyond freedom of information legislation to enhance transparency, making 'open book' governance possible for the first time. However, some aspects have been criticized for being developer-led with commercial ventures gaining most while sidelining the public value benefits of open data. Meanwhile, across the private sector a focus on 'big data' seems to offer a cornucopia of benefits in terms of efficiency and enhanced openness and understanding of policy and operations in public and private sectors alike. Journalists and civil society groups have been innovative in developing the skills and expertise to use big data to open up policy and administration to the public. Governments have been slower to reap the possible benefits from big data, such as the matching of public policy and services to citizens' actual behaviour, preferences and needs which could enhance both efficiency and effectiveness. This paper articulates the distinction between open data and big data and considers how the merits of each might be better exploited for public policy and service design.