Open data

The concept of “open data” has recently gained significant attention, as public institutions and stakeholders look for ways to make better use of data, particularly in the public sector, as well as improve government services and decision-making. Open Data are generally understood as:

  1. Being easily available to anyone, typically via the internet.
  2. Being accessible in both human- and machine-readable formats that allow data to be combined and utilized in different ways using computer programs.
  3. Free from legal restrictions, usable and re-usable for any commercial or non-commercial purpose without registration requirements, or the need to obtain advance permission.
  4. Available at no cost.

“Open Government Data” is a subset of Open Data that specifically pertains to data originating from government ministries and agencies. Open Government Data can include (among many others):

  • Micro-data from statistical surveys and censuses.
  • Macroeconomic data and aggregate socioeconomic indicators.
  • Budgets, revenues, and expenditures.
  • Property records and company registries.
  • Contract and procurement records.
  • Performance monitoring and evaluation.
  • Administrative records in health, education, transport, environment, gender, financial sector and many others types of data produced by public sector agencies in the conduct of their regular duties.
  • Local data on public services, such as public transport schedules, weather data, crime data and similar.
  • Declarations of assets and conflict of interests of civil servants or politically appointed authorities.
  • Digital maps of statistical and administrative boundaries and geo-referenced information.

Open Government Data is closely aligned with “good government” efforts, in which high-quality, publicly available information is seen as essential to transparency, public trust (See A.MANAGING), and improving public services. The Open Government Partnership, founded in 2011 and currently consisting of 59 countries, commits in its Declaration to “provide high-value information, including raw data, in a timely manner, in formats that the public can easily locate, understand and use, and in formats that facilitate reuse.” (*)  Hence, Open Government Data initiatives typically involve multiple agencies that coordinate their efforts. While NSOs may or may not be the lead agency in an Open Data program, as curators of official statistics, they are well-positioned to play a strong role in a government-wide initiative.

 

OPEN DATA AND NATIONAL STATISTICAL SYSTEMS

Benefits and Complementarities

  1. Alignment of objectives.  The basic principles of open data imply that government data should be considered a public good for the benefit of all. These principles are strongly aligned with those of statistical systems. Accordingly, aligning the national statistical system with an open data initiative can enhance the prominence and relevance of the NSO in the context of a public agenda.
  2. Public trust. Governments and agencies that are committed to openness and transparency stand to gain in terms of public trust and support. By making essential data easy to obtain and use, as well as providing metadata and documentation so that people can understand the data and how they are produced, NSOs stand to gain credibility.
  3. Greater efficiency. It is not uncommon for public ministries to expend significant resources managing data and information systems for the benefit of users either within the agency itself or within other agencies. Open data initiatives streamline the process of obtaining public data by simplifying and clarifying the terms under which it can be used, and removing technical barriers to accessing the data. These benefits confer not only to the public at large, but to other public ministries. As such, there are often cost savings that benefit the NSO and the government as a whole.
  4. Public & economic innovation. There is now a growing body of evidence of how open data are being used to improve public services, create new economic opportunities and jobs, and produce new insights and understanding about public policies. All of these are opportunities for NSOs and other public agencies that provide data to develop stronger and more collaborative relationships with the public at large as well as specific constituencies, advocacy groups and the private sector (since open data can be used commercially).

 

Risks & Challenges

  1. Confidentiality. Confidentiality issues do not generally arise from the publication of official statistics, which are typically intended for public use even without an open data policy. However, confidentiality is often a risk with respect to micro-data or primary data the NSO uses in production, which often contain personal or private information.  NSOs should establish policies and data management practices to differentiate “open data,” which may be disseminated under an open data policy, from “private data” for which there are legitimate reasons to restrict access. Most likely, it is neither practical nor advisable for an NSO to release all of its micro-data under an open data policy such as the one described here.
  2. Reputation.  NSOs often worry that broad data dissemination will result in criticism or reputational risk if the data are perceived to be poor quality or are used inappropriately. While this concerns are understandable, they can be addressed by clearly and publicly documenting the established scientific methods used to produce the data, and by publishing metadata (the more extensive the better) to accompany the data. Cases where other parties misinterpret the data are opportunities for data providers to provide an authoritative voice on proper data techniques, which often works to the agency’s benefit. Furthermore, open data can be an important strategy to improve data quality, when accompanied by appropriate interactions with data users (See 4. ASSESSING). By opening data, NSOs can de facto use the free review services of users and encourage them to submit detected inaccuracies, omissions or inconsistencies, so that these can be addressed. Similarly, computer applications that use open data are more likely to highlight some data deficiencies than are difficult to detect by internal or external human reviewers.
  3. Capacity.  Open data best practices set a high standard for data publication, requiring coordination across government agencies with respect to data review, work flow, publication schedules, standards, and metadata. Most NSOs are familiar with the processes of compiling data from other ministries, departments and agencies, as well as curating and presenting these data through user-friendly statistical abstracts yearbooks or, in some cases, central databases containing the most important socioeconomic indicators and time-series. This experience suggests that NSOs may be well-positioned to play a strong technical role in an open data program. However, these roles may impose capacity issues on the NSO, necessitating the need for additional staff and/or funding. Without high-level support (See B. COMMITTING) to ensure capacity issues are addressed, the NSO may find it difficult to meet new obligations. Possibly for this reason, experience so far has shown that NSOs do not typically assume a strong inter-agency coordinating role.
  4. Revenues and Budget. If an NSO receives significant budget support through the sale of data or data products, an open data program will likely reduce or possibly eliminate these revenues. Accordingly, it is important that an open data program provide direct budget support for NSOs, particularly when this is the case. High-level political support is therefore essential to address the budget impacts, as discussed below.
  5. User engagement and expectations.  One of the basic principles of open data is that anyone can access the data, and in fact they are encouraged to use the data in unanticipated ways. This can place unexpected demands on public agencies that are not used to dealing with a general audience.  Furthermore, users may come to expect and rely on open data, and request that other datasets be made public as well (in fact, many open data policies explicitly encourage this). This “virtuous cycle” can place additional and uncertain demands on public agencies.

 

INTEGRATING OPEN DATA INTO AN NSDS

  1. Political commitment.  High-level political commitment, both within the NSO and the government as a whole, is essential to ensure the success of an open data initiative. As with an NSDS, multiple agencies may be involved in providing open data, and a high-level directive will facilitate alignment of their efforts. NSOs leaders should understand and communicate the advantages of open data both to the public and to key ministries to ensure cooperation and public engagement.
  2. Legal framework.  It is important to place open data in the proper legal context to eliminate barriers to the release of data and provide clarity for how the data can be used. In an Open Data program, it is important to publish a clear license or Terms of Use so that users understand how the data can be used (Creative Commons licenses, specifically CC-By  and CC-0 , accomplish this purpose in a clear, uniform manner).
  3. Standards and safeguards.  Open data should conform to statistical standards consistent with the best statistical practice, as well as standards suggested by open data best practices, such as those described in the definition above. Safeguards should be established to ensure that public data is reliable, and that confidential, private or sensitive data –such as for national security reasons- is not improperly disclosed. To this end, the national statistical system should have an established process to determine what data poses disclosure risks.
  4. Leading by example. NSOs will often be the lead agency in charge of formulating and implementing a country’s NSDS. NSOs can play an important role by setting a good example for how an open government data policy could be implemented by striving to make as much of their own data open with due consideration of privacy and quality concerns.
Good Practices: