Sheet 1: An Overview of the Commonwealth Arrangements

What is data integration for statistical and research purposes?

Data integration involves combining data about an individual person, household, family or business from different administrative and/or survey sources to produce new datasets for statistical and research purposes. This approach leverages more information from the combination of datasets than is available from the individual datasets separately. Data integration for statistical purposes means that the information is not used to identify an individual person, household, family or business for regulatory purposes, compliance monitoring or service delivery.

Why conduct data integration using Commonwealth data for statistical and research purposes?

There is an increasing demand for integration of datasets at a unit record level in Australia. This demand is being driven by the potential data integration has to produce rich and dynamic datasets. Integrated datasets can assist research into complex problems that have multiple contributing and confounding causes, such as climate change, unemployment, homelessness and obesity. Data integration allows better use of data that is already available, it can be a cost effective and timely way of gathering more information for statistical and research purposes.

While there are clear benefits to data integration, there is also a responsibility for stakeholders of data integration projects to preserve privacy and confidentiality through policies and procedures.

Preservation of privacy and confidentiality is needed to prevent harm to data providers (including individuals, families, households or organisations who have contributed data) or a loss of public trust in the Australian Government or its institutions.

Arrangements for the integration of Commonwealth data for statistical and research purposes

The demand for data integration, coupled with the responsibility to maintain the privacy and confidentiality of those providing the data, have led to the development of the arrangements for data integration involving Commonwealth data for statistical and research purposes (the Commonwealth arrangements). To establish the Commonwealth arrangements seven high level principles were developed. These principles outline when data integration should occur, who is responsible for the data and the data integration process, how the data should be treated and why transparency is important in the process of data integration, see below for a description of each principle.

The High Level Principles are supported by a set of Governance and Institutional Arrangements for the integration of Commonwealth data. These arrangements provide a framework for how data integration projects using Commonwealth data will be managed and governed, providing a foundation for the increased use of Commonwealth data as a strategic resource.

The main components of the Governance and Institutional Arrangements are:

• a Cross Portfolio Data Integration Oversight Board (the Board) to oversee the development and ongoing administration of the Commonwealth arrangements for data integration. There is also a small Secretariat, to provide support to the Board and its ongoing activities.
• nomination of an integrating authority for each project involving Commonwealth data. The integrating authority is responsible for the end to end management of the project and the ongoing management of the integrated dataset throughout its lifecycle.
• an accreditation process to enable the Board to endorse integrating authorities with the demonstrated capacity to deal with projects that present a high risk.
• a guide for stakeholders of data integration projects involving Commonwealth data. The guide contains information describing practices and procedures that can be used to ensure compliance with the Commonwealth arrangements.
• a public register of data integration projects and a public feedback mechanism. The public register builds public trust through transparency and ensures that data integration involving Commonwealth data is conducted in an open and accountable way.
• an Engagement and Education strategy to support the Commonwealth arrangements.

High Level Principles for Data Integration

Principle 1 - Strategic Resource

Agencies should treat data as as strategic resource and design and manage administrative data to support their wider statistical and research use

Principle 2 - Custodian’s Accountability

Agencies responsible for source data used in statistical data integration remain individually accountable for their security and confidentiality.

Principle 3 - Integrator’s Accountability

A responsible ‘integrating authority’ will be nominated for each statistical data integration proposal

Principle 4 - Public Benefit

Statistical integration should only occur where it provides significant overall benefit to the public

Principle 5 - Statistical and Research Purposes

Statistical data integration must be used for statistical and research purposes only.

Principle 6 - Preserving Privacy and Confidentiality

Policies and procedures used in data integration must minimise any potential impact on privacy and confidentiality.

Principle 7 - Transparency

Statistical data integration will be conducted in an open and accountable way.