On the Benefits and Costs of Public Access to Data Used to Support Federal Policy Making

Over the past few decades, the quality of published scientific research has increasingly come into question. Top academic journals and research organizations have realized that empirical research that cannot be replicated has little credibility. The federal government has recognized the need to ensure both the reliability of the scientific underpinnings of its policy decisions and public confidence in that reliability, and it has taken steps to improve the quality and accessibility of information it uses in policymaking.

In a new paper for the Mercatus Center at George Mason University, economists Randall Lutter and David Zorn review current federal policies and procedures intended to ensure that scientific and technical research meets appropriate quality standards, comparing them with similar practices and procedures already in use at nonfederal institutions. Contrary to estimates by the Congressional Budget Office (CBO), federal agencies can provide access to data and computer code at a reasonable cost—a cost that is likely exceeded by the benefits of transparency and greater reproducibility.

BACKGROUND

A 1999 directive to federal agencies provides for public access under the Freedom of Information Act to federally funded research data related to published research findings used in developing federal regulations. However, in 2015 some members of Congress introduced two bills (H.R. 1030 and S. 544) that would require the Environmental Protection Agency (EPA) to make data publicly available for any studies that it relies on in its policy making, including any research that is not federally funded.

However, the Obama administration has stated that these bills would likely be vetoed because they “would undermine the EPA’s ability to protect the health of Americans [and] would impose expensive new mandates on the EPA. . . . [They] also could impede EPA’s reliance on the best available science.” Instead, the administration claims that an Office of Science and Technology Policy directive is sufficient to promote transparency. This directive stipulates that agencies spending more than $100 million a year on research should issue plans to maximize public access to federally funded data. However, the EPA has not followed the directive. This suggests that H.R. 1030 and S. 544 may be more effective, especially since they would require public access to data used by the EPA that is not federally funded.

DATA AND METHODOLOGY

CBO “estimates that [the EPA] would spend, on average, $10,000 per scientific study for activities to meet [H.R. 1030 and S. 544]’s requirements.” Following CBO’s cost-estimation process and using available official information on wages and research and EPA cost estimates, this study develops a more transparent estimate of the costs of complying with these bills.

KEY FINDINGS

Costs of Greater Access to Data Relevant to Federal Rulemaking

Based on the EPA’s methodology, the total cost to the EPA for data collection and data accessibility would be $2,558 per study—or about 26 percent of the $10,000-per-study cost estimated by CBO.

• To the extent that the agency uses the same scientific research in multiple rules, the cost of making the data publicly accessible would be less than $2,000 per study in those cases, or less than 20 percent of the cost estimated by CBO.
• The estimate of the annual cost for the EPA to obtain and post the data would be about $46 million if the EPA were successful at obtaining data for the amount of scientific research that the EPA has traditionally cited in its regulations.
• Because of inconsistencies in and unavailability of research, it is likely that the EPA would receive data for only 20 percent of the studies for which it sought data. This reduces the estimate of the total cost of the data collection and posting to less than $18 million per year.

Benefits of Greater Data Access

Public access to the data underlying the studies agencies use to make significant public policies will help ensure that the policies are based on valid science, leading to increases in the true net benefits of federal policies.

• According to the EPA’s own estimates of the net benefits of its major rules, if providing public access to research that the EPA uses increases net benefits by only four-tenths of 1 percent, then the benefits of public access to data will exceed the costs.
• These estimates are conservative because they ignore increased net benefits related to the 287 nonmajor final rules that the EPA issued if data for these rules were posted, as well as other important benefits of transparency, public participation, and collaboration.

POLICY RECOMMENDATIONS

Without public access to data, federal agencies risk making policy decisions based on flawed information that can misdirect public and private resources. A policy prescribing public access to data in studies that federal agencies, such as the EPA, rely on in their rulemaking would likely offer net benefits—with costs much smaller than those estimated by CBO for H.R. 1030 and S. 544.

Several improvements could be made to the current legislative proposals:

• The legislation could be broadened to cover all regulatory agencies, because all agencies are vulnerable to invalid research.
• Legislation could target only regulations that are economically significant (as defined by Executive Order 12866) to limit the cost of obtaining data for all rulemaking.
• The definition of “relied upon” could be amended to clarify that the law affects only research that an agency uses to support or define key dimensions of the policy under consideration.