Data Integration Involving Commonwealth Data for Statistical and Research Purposes

Building a safe and effective environment for data integration

Statistical data integration involves integrating data from different administrative and/or survey sources to provide new datasets for statistical and research purposes. The approach leverages more information from combining datasets than could be obtained by examining individual datasets separately. Data integration aims to maximise the statistical value of existing and new datasets, to improve community health as well as social, economic and environmental wellbeing, by integrating data across multiple sources, working with governments, the community and researchers to build a safe and effective environment for data integration activities.

Integrated datasets provide public benefits in terms of improved research, supporting good government policy making, program management and service delivery. Integrated datasets also create important opportunities to expand the range of official statistics to better inform Australian society.

High level principles for data integration involving Commonwealth data for statistical and research purposes

On 3 February 2010, the Portfolio Secretaries Meeting (now Secretaries Board) endorsed a set of high level principles for the integration of Commonwealth data for statistical and research purposes. At the same time, a set of governance and institutional arrangements to support these principles was requested.

A complete description of the high level principles is available at www.nss.gov.au and a summarised version is provided below.

Strategic resource - Principle 1

Responsible agencies should treat data as a strategic resource and design and manage administrative data to support their wider statistical and research use.

Custodian’s accountability - Principle 2

Agencies responsible for source data used in statistical data integration remain individually accountable for their security and confidentiality.

Integrator’s accountability - Principle 3

A responsible ‘integrating authority’ will be nominated for each statistical data integration proposal.

Public benefit - Principle 4

Statistical integration should only occur where it provides significant overall benefit to the public.

Statistical and research purposes - Principle 5

Statistical data integration must be used for statistical and research purposes only.

Preserving privacy and confidentiality - Principle 6

Policies and procedures used in data integration must minimise any potential impact on privacy and confidentiality.

Transparency - Principle 7

Statistical data integration will be conducted in an open and accountable way.

Cross Portfolio Data Integration Oversight Board

A high level Cross Portfolio Data Integration Oversight Board (the Board) will be established by early 2011 to oversee the development of a cross government environment for data integration involving Commonwealth data for statistical and research purposes that is safe and effective. The Board will be chaired by the Australian Statistician and membership will initially include the heads of the Department of Health and Ageing; the Department of Families, Housing, Community Services and Indigenous Affairs; and the Department of Human Services.

The role of the Board will be to:

• Provide strategic and collaborative leadership, support effective governance and help manage the risks of particular data integration projects;
• Help manage the systemic risk associated with conducting multiple data integration projects involving Commonwealth data through assessment of proposed risk mitigation strategies, and the provision of advice; and
• Endorse any changes or additions to the overall environment, including amendments to the principles or guidelines, or the development of new general tools to support integration or safe access to integrated data for statistical and research purposes.

Activities of the Board will include:

• Advising on data integration projects assessed as being of high risk and with the potential to significantly impact information related activities across government. Where the Board advises on amendments to, or discontinuation of, a particular project, the relevant custodians and integrating authority will need to consider these views in deciding whether to proceed with the proposed project.
• Working with agencies to help ensure the systemic risk associated with high risk projects is adequately managed. It will do this by reviewing data integration projects involving Commonwealth data deemed by agencies to pose a significant level of systemic risk, and advising on the sufficiency of the intended risk mitigation strategies.
• Review of any adverse incidents of high public concern relating to data integration involving Commonwealth data for statistical and research purposes, and seen as having a likely systemic impact on public trust in government use of data.
• Consultation with the Office of the Australian Information Commissioner, as required. The Board is also free to consult representatives from the research community or other parties as required.

Integrating Authorities

An essential pillar of establishing a safe and effective environment for data integration involving Commonwealth data is the nomination of an agency as the authorised integrating authority for each statistical data integration proposal. The integrating authority will be responsible for the sound conduct of the data integration project and may work with other agencies to achieve components of the project, for example it might use another agency to undertake linkage or to support dissemination. The integrating authority has overall responsibility to ensure that risks have been assessed, managed and mitigated throughout the duration of the project, including regular reviews of ongoing projects.

As described by Principle 3, the integrating authority will ensure appropriate governance is in place for the data integration project including: using an open approval process; documenting the proposal; considering the privacy impacts, examining the expected costs and benefits of the proposal and considering the access arrangements and dissemination plans. The integrating authority will be responsible for the ongoing management of the integrated data, ensuring it is kept secure, confidential and fit for the purposes for which it was approved.

Integrating authorities are responsible for the implementation of the data integration project, and the management of the integrated datasets throughout their life cycle, ensuring full compliance with commitments made as part of the project approval, and in line with a set of guidelines to be developed as a priority by 2012.

A key requirement of integrating authorities is that, to the extent that the data they deal with involves identifiable information, they be in a position to comply with the requirements of the Privacy Act 1988 (in regards to information about individuals) and secrecy provisions generally (in regards to information with respect to the affairs of any third party, corporate or individual). This may require either the consent of the individual to the particular use or disclosure for Privacy Act 1988 purposes, or an overriding public interest test certified in accordance with the relevant secrecy provision.

Integrating authorities will only be established at the initiative of an interested agency. Such agencies may wish to involve their respective Ministers in the course of preparing a proposal for accreditation to the Board. An integrating authority could also be established administratively within a Department or other agency, that is, it would be part of an agency subject to the provisions of the Privacy Act 1988.

Accreditation Process

For data integration proposals considered by custodians to pose a high systemic risk, nomination of an authorised and accredited integrating authority is required. An accreditation process will be established through the Board to enable the endorsement of authorised and accredited integrating authorities with the capacity to deal with high risk data integration projects or families of projects involving Commonwealth data.

Data integration projects involving Commonwealth data for statistical and research purposes judged to pose a high systemic risk will need to be undertaken with particular care to help mitigate this risk. This will require a high level of relevant expertise, a strong understanding of, and capability for, maintaining security, as well as a consistently high standard of behaviour by all employees based on a strong culture, and set of values. To ensure effective use of specialist skills and infrastructure, the number of accredited integrating authorities is expected to be relatively limited.

An accreditation process including interim arrangements will be proposed through cross government consultation. It is expected that interim arrangements will be proposed for discussion and endorsement by the Board in early 2011 with final arrangements agreed early 2012.

Best Practice Guidelines

A comprehensive set of guidelines describing best practice for data integration projects involving Commonwealth data for statistical and research purposes will be jointly developed by a cross government working group and approved by the Board. These guidelines will build upon already available guidelines (e.g. National Statement on Ethical Conduct in Human Research, related guidelines of the Office of the Privacy Commissioner, and agency specific guidelines and proformas).

The guidelines will be considered in conjunction with legislation relevant to the data custodians, data providers and the integrating authority.

The guidelines will cover such areas as:

• Privacy including Privacy Impact Assessments and the protection of privacy in medical research (Endnote 1)
• Approval
• Registration
• Agreements – between data custodians and the integrating authority; or with end users
• Responsibilities of custodians and integrating authorities including accreditation criteria and accreditation process
• Handling of identifiers and application of the separation principle, for example, the separation of identifiers and key demographic data used for linking (e.g. date of birth) from the individual content or event information (e.g. clinical or benefit information)
• Minimum standards for the secure management of data during and after integration, including the exchange of sensitive data across agencies
• Minimum standards for consent to access Commonwealth data
• Confidentialising of integrated data sets and research outputs
• Managing access and use of integrated data sets
• Minimum standards for data destruction or review.

The guidelines will be developed progressively, commencing in 2011 with guidelines relating to integrating authorities. They will be made publicly available on an internet site, and maintained by the Secretariat to the Board.

Education and Training

An education and training strategy will be developed and implemented to support the guidelines for data integration projects involving Commonwealth data for statistical and research purposes.

The guidelines and the education and training strategy will be targeted to key stakeholders including government agencies (acting variously as custodians, integrating authorities, researchers, Privacy Commissioners), the private sector (as custodians or researchers), the academic/research community and the public.

The purpose of the education and training strategy will be to build an understanding of:

• The underpinning values of a safe and effective environment for data integration projects.
• Key issues in managing privacy and confidentiality in data integration projects.
• Key legislative requirements within the management of data integration projects including maintaining privacy and confidentiality.
• Sanctions that apply for non compliance with legislative requirements or other institutional requirements.
• Support available in terms of specialist advice, knowledge and tools.

Following its development and finalisation by January 2013, the education and training strategy will be publicly available on an internet site and maintained by the Secretariat to the Board.

Network of Survey Liaison Officers

Survey Liaison Officers (SLOs) have been established in many Commonwealth agencies to provide a central contact of information and expertise in statistical collections involving businesses. Survey Liaison Officers are part of an Australian Government initiative established by the Prime Minister in 1997 to reduce duplication, minimise the burden on business, and ensure that statistical data collections involving businesses are fit-for-purpose.

The role of the existing network of Survey Liaison Officers across Commonwealth agencies will be expanded to provide a central contact of information and expertise in data integration projects involving Commonwealth data. This network will be able to share information on good practice and help minimise duplication.

Survey Liaison Officers as at November 2010:

• Attorney-General’s Department (AGs)
• Australian Bureau of Agricultural and Resource Economics (ABARE)
• Australian Communications and Media Authority (ACMA)
• Australian Competition and Consumer Commission (ACCC)
• Australian Crime Commission (ACC)
• Australian Customs and Border Protection Service
• Australian Institute of Criminology (AIC)
• Australian Institute of Health and Welfare (AIHW)
• Australian Prudential Regulation Authority (APRA)
• Australian Taxation Office (ATO)
• Australian Trade Commission (Austrade)
• Department of Agriculture, Fisheries and Forestry (DAFF)
• Department of Broadband, Communications and the Digital Economy (DBCDE)
• Department of Defence
• Department of Education, Employment and Workplace Relations (DEEWR)
• Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA)
• Department of Foreign Affairs and Trade (DFAT)
• Department of Health and Ageing (DoHA)
• Department of Human Services (DHS)
• Department of Immigration and Citizenship (DIaC)
• Department of Infrastructure and Transport (DIT)
• Department of Innovation, Industry, Science and Research (DIISR)
• Department of Resources, Energy and Tourism (DRET)
• Department of the Prime Minister and Cabinet (PM&C)
• Food Standards Australia New Zealand (FSANZ)
• Medicare Australia
• National Centre for Vocational Education Research (NCVER)
• Private Health Insurance Administration Council (PHIAC)
• Private Health Insurance Ombudsman (PHIO)
• Productivity Commission (PC)
• Royal Australian Mint (RAM)
• Screen Australia (SA)

Register of Data Integration Projects

Consistent with a more open government, and to build trust through transparency, a web-based register will be established to include a description of data integration projects involving Commonwealth data for statistical and research purposes. A public feedback mechanism will enable members of the public to register support or concerns and make suggestions about particular projects or families of projects.

The register entry will describe the project purpose and data sources, recognising that specific details on individual use will generally not be made publicly available for in-confidence government business. For similar reasons, actual data will not be included. A format for such submissions will be developed across government and made available through self-help mechanisms (email, SLO network, internet site).

The integrating authority nominated for each project will be responsible for submitting an entry to the register once the project is finalised.

The register will be available through an appropriate whole-of-government web site, along with related information, including the high level principles, the guidelines and access to the public feedback mechanism. It is expected that development of the register will commence in 2011 with trialling through 2012 and final release by 2013.

The Secretariat will maintain the register and associated internet site.

Secretariat

A small Secretariat will be established by early 2011 and based in the Australian Bureau of Statistics, to support the Board and its ongoing activities.

The Secretariat will support the Board by:

• Providing administrative support as needed to facilitate meetings, maintain centralised documentation and undertake investigation or development work as directed by the Board.
• Assisting the cross government development of the guidelines, the development and, in part, the delivery of the education and training strategy, and expansion of the network of survey liaison officers.
• Developing and maintaining a web-based public register of the description of projects and the provision of a public feedback mechanism, in conjunction with other agencies.
• Monitoring the risk assessment of projects, including legal and systemic risks, advising project managers where further advice has been requested, and identifying instances where apparently high risk projects have not been assessed as such, initiating discussion with the agency involved in the first instance to resolve concerns, and as required, escalating the assessed level of risk, for Board consideration.
• Providing a contact point in government for issues relating to data integration involving Commonwealth data for statistical and research purposes, whether coming from data custodians, integrating authorities, researchers or the public.
• Following development of the public register, releasing an annual report on key aspects of data integration activity involving Commonwealth data for statistical and research purposes, including a summarised report on public feedback.

Transition Arrangements

Transition arrangements will apply during the early stages of the implementation of the high level principles and the governance and institutional arrangements for data integration involving Commonwealth data for statistical and research purposes. Transition arrangements include the following:

• The Board and Secretariat will be established in early 2011 to commission cross government work to develop and begin communicating the guidelines, the education and training strategy, the use of survey liaison officers, and accreditation criteria for integrating authorities.
• In early 2011, it is expected that a small number of agencies will be accredited as authorised integrating authorities for high risk projects. This will be an interim accreditation, subject to review and finalisation of the formal accreditation process in early 2012.
• The education and training strategy will be developed by January 2013 to provide a range of self help tools to inform stakeholders. This will be expanded to include seminars, workshops, and conferences. The strategy will also be designed to develop the skills of integrating authorities.
• New data integration projects which commence prior to the development of the guidelines are expected to be set up to conform to the high level principles.
• High risk projects should be notified to the Secretariat for review by the Board. The project should be fully developed before consideration by the Board. However, in the initial stages, it may be useful for those developing the proposal to liaise with the Secretariat. Where a new, potentially high risk project has been substantially developed, and committed timeframes do not permit review by the Board, the Board should be notified of the project and it should be treated as a project currently underway.
• Those projects, including families of projects, which are underway as at October 2010, and which are expected to be completed by October 2012, will not be subject to the new arrangements, although project managers are encouraged to consider the conformity of the project to the principles and to seek advice from the Secretariat if there are aspects of the projects that may be of concern.
• Those projects, including families of projects, which are underway, and are expected to continue beyond October 2012, will need to become aligned with the principles, and with the proposed governance and institutional arrangements. It is proposed that such projects are brought into alignment by October 2012.