On Objective Risk

Over the last several decades, criticism of regulatory agency estimates of risk has come from a variety of institutions, including the National Research Council and the president’s Office of Management and Budget. As with other matters related to regulation, Congress has delegated the task of estimating risks to federal agencies, with the expectation that they will apply the necessary level of expertise. Debate over agency risk assessments has often focused almost exclusively on the accuracy of the assessment, with little attention to whether the assessment followed objective scientific processes.

In a new paper published by the Mercatus Center at George Mason University, economist Dima Yazji Shamoun and toxicologist Edward J. Calabrese show that shifting the debate to process objectivity would allow better evaluation of risk assessments. In doing so, the paper draws from the government’s own guidance for best practices for performing risk assessments. This paper provides a crucial first step toward enabling those who monitor regulatory agencies to hold them to those practices.

To read the paper in its entirety and learn more about its authors, see “On Objective Risk.”

BACKGROUND

Health and safety questions concern every American household: How much seafood should I consume given the possibility of ingesting methyl mercury? Is air pollution causing my kids’ asthma attacks or my mother’s cardiovascular disease? What chemicals are responsible for increasing my cancer risk? Government regulatory agencies are charged with determining these risks and acting on them where appropriate, but the challenge is always in the details:

• Environmental Protection Agency rules alone are responsible for 63%–82% of monetized benefits reported by all regulatory agencies. Indeed, the vast majority of the federal government’s regulation of the economy depends on a process that determines how risky an activity or a chemical is and how much of that risk will be reduced by some regulatory action.
• Too much focus on outcomes can introduce bias, which can increase costs and misallocate resources. Americans depend on federal agencies to strike the right balance and regulate risks to health and safety in a way that neither under- nor overregulates risk. When an agency’s performance is evaluated based on the accuracy of its risk evaluations, bias toward certain outcomes may be introduced, which in turn may cause agencies to overregulate risks relative to the costs Americans pay—or, worse, may cause other risks to increase, making Americans less safe.
• Process objectivity is a scientific means to ensure consistency across risk assessments. Rather than debating the true risk involved with any particular case, those concerned about regulatory risk assessments should focus on whether the government followed an objective, standard process based on the scientific method. This paper provides the framework for evaluating all risk assessments performed by government agencies.

KEY PRINCIPLES

Risk estimates and benefit estimates of health and safety regulations derive their objectivity from the process that brings them about. Consistent adherence to a process meant to produce objectivity yields objective results. Risk estimates derived from an ad hoc application of the objective process are biased and may be responsible for vast resource misallocation. A routine application of the objective process outlined in the paper will reduce error and achieve consistency across assessments.

The paper proposes a novel methodology for testing the objectivity of the risk/benefit estimates of federal health and safety regulations:

• In order for the process to be objective, two factors are necessary: (1) adherence to a body of principles, applied consistently and in their entirety; and (2) an independent reassessment (by a third party outside the regulatory agencies), according to the body of principles, of the risk and benefit estimates of major health and safety regulations.
• There are four main categories of objective risk assessment: analysis, robustness, openness and transparency, and review. By following an objective process along the lines introduced in the paper—based on well-established principles already supposed to be in use by the federal government—risk assessments can be independently verified by a third party, such as a university-based research center.

CONCLUSION

Taxpayers spend billions of dollars on regulatory agencies that promise to protect their health and safety by reducing their risk from exposure to myriad alleged hazards. Such regulatory decisions are based on highly technical and scientific documents, which leave both Congress and the majority of the public in the dark. Using the framework provided in the paper, all people interested in health and safety regulation can systematically review risk assessments and begin the process of holding agencies accountable.