Businesses, not-for-profit organisations and governments are increasingly recognising the power of data to improve their decision-making and service to their customers or clients. Data sets held by governments and businesses can now be used for multiple purposes, not just the initial transactional record.
They have the potential to transform policy development and evaluation and economic analysis.
Payment systems for government benefits and services lead to the accumulation of large data sets, containing millions of data items. The primary purpose of the collection of the data is for payment and accountability. But an important secondary purpose is to use the data to understand patterns and trends.
Secondary analysis of data sets collected for ‘routine’ or ‘administrative’ purposes is now a well-accepted type of research.
This secondary analysis of government datasets has a number of additional benefits:
- Because the analysis is conducted on government data, the research is almost always policy-relevant. It can shed insight into patterns of spending or service use that would otherwise not come to the attention of policy makers; and
- Secondary analysis leverages investments that have already been made in data collection and so is generally a less expensive form of research. Secondary analysis of Australian data sets has addressed a range of issues including quality and efficiency of, and access to, health care.
Government data holdings, derived from claims processing, should be seen as an important public resource to assist in policy-relevant research which will benefit the Australian community.
Failure to harness fully the potential of these data sets represents a significant lost opportunity both for policy development and research.
Risks to Privacy
Big data comes with risks, including risks to privacy. Privacy risks can be to individual consumers and to providers, but risks to the two different types of stakeholders are quite different.
But custodians should not unnecessarily regard all data as private. For instance, currently institutions with the best survey data impose strict confidentiality requirements on researchers. We feel that these requirements, as well as being onerous, are largely self-imposed.
Privacy risks can be mitigated by controlling data outputs and/or controlling the data released. The New Zealand approach to controlling data outputs should be explored as a potential additional data release strategy in Australia.
Data custodians also limit release to approved data users. This is appropriate and should be continued.
Data release should be expanded and facilitated by
- Releasing metadata
- Developing streamlined and standardised release approval processes; and
- Developing common-use data sets.
Surveys by institutions like the ABS are clearly intended to be used for research.
Survey respondents volunteering this information do so with full knowledge of this purpose. It is therefore unlikely that the assurance of confidentiality has any effect on the information offered.
Yet once this assurance is given, it must be honoured.
Stephen Duckett and Pete Goss are directors at the Grattan Institute. This is an edited version of their submission to the Productivity Commission’s Inquiry into Data Availability and Use. The opinions in this report are those of the authors and do not necessarily represent the views of the Grattan Institute.