Anne Smith, Clean Air Standards, Feb. 5, 1997

Statement of Dr. Anne E. Smith
Subcommittee on Clean Air, Wetlands, Private Property and Nuclear Safety regarding Scientific Issues Surrounding the
Proposed PM-2.5 Standard
February 5, 1997
Washington, DC

My name is Dr. Anne E. Smith. I am a Vice President and Principal of Decision Focus Incorporated, a consulting firm with offices in Mountain View, CA, Washington, DC, and London, UK. I have 20 years of experience in environmental risk assessment and risk management, founded on a Ph.D. in economics from Stanford University. I started my professional career in the U.S. EPA's Office of Policy, Planning and Evaluation in 1977, where I was involved in air quality issues such as airborne arsenic regulations and EPA's air cancer policy. Over the 18 years since, I have contributed to a wide range of major environmental science/policy assessments for the U.S. Environmental Protection Agency, the National Acid Precipitation Assessment Program, the Grand Canyon Visibility Transport Commission, the Electric Power Research Institute, the Gas Research Institute, and many others.

In 1980, I was one of the experts selected by the U.S. Environmental Protection Agency's Office of Air Quality Planning and Standards to develop methods for assessing risks from criteria air pollutants, with a demonstration assessing risks from ambient carbon monoxide. I also served the United Nations Economic Commission for Europe in preparing a plan for analyzing acid rain control strategies. In the late 1980s, I worked closely with the Director of the U.S. National Acid Precipitation Assessment Program, advising on methods for integrating the scientific research into a comprehensive assessment. Recently, I developed the system used by the Grand Canyon Visibility Transport Commission to assess alternative policies for managing the particle precursors that contribute to impaired visibility in the Southwest. I also performed the economic analyses of the Commission's recommended visibility management alternatives. In addition to my consulting engagements, I have served on a number of expert panels on risk assessment, including two committees of the National Academy of Sciences, a Keystone Foundation dialogue, and two committees of the United Nations Environment Programme.

I am honored to have this opportunity to speak with you today about the science supporting the proposed new standards for fine particulate matter, PM2.5. My statement reflects my personal opinions, and not those of my company or any other group.

In previous statements on this issue, I have likened the current situation for PM to the classic "Shell Game" -- the one where you try to guess which of several walnut shells is covering a pea. The proposed fine particle standard would force the expenditure of a great deal of money to reduce PM from a variety of sources, yet it is far from clear that the proposed standard would successfully target the true culprit that is causing adverse health impacts. We could turn over many empty shells, at great expense, but with little benefit to public health. I will explain why.

What is the evidence of a PM-related health effect?

There are a number of statistical, or "epidemiology", studies that seem to indicate that as ambient PM goes up and down, so too do the levels of health effects. However, when we observe two types of data going up and down together, we should not necessarily conclude that there is a causal relationship between the two phenomena. For example, if we were to observe such an association between heat stress mortality and ice cream cone sales, few people would suggest that one is caused by the other. The error in this example is so obvious to us because we all have a good understanding of the biological processes that result in heat stress. So, when we have statistical evidence of the sort that seems to suggest that ambient PM and mortality go up and down together, we also want to have scientific data about biological processes associated with PM to help us explain why we should believe this is a causal relationship and not just a statistical association.

What does the science tell us about a biological explanation?

If you review EPA's Criteria Document for PM, you will find that EPA concludes that "no credible supporting toxicologic data are yet available." That is, when very high levels of various types of PM constituents have been inhaled or otherwise placed in the lungs of humans or animals, no one has observed a consistent response of the tissues that could be clearly linked to the health effects observed in the statistical studies. This inability to elicit significant and consistent biological responses to high levels of PM exposure is troubling, since you might expect adverse changes to be readily observable in laboratory experiments if the health effects were as large as the statistics seem to suggest. Toxicological evidence suggesting adverse health effects is present for other criteria pollutants (e.g., ozone, carbon monoxide, sulfur dioxide, nitrogen oxides, etc.).

The inconsistency between the statistics and the toxicology findings give us a strong motivation to try to develop a line of physiological or medical reasoning to explain whether or not these statistical relationships are biologically plausible. Attempts to provide such reasoning have been at best speculative. In the peer-reviewed Criteria Document, EPA suggests that such reasoning is not compelling: "There is...a paucity of information...that argues for the biologic plausibility of the epidemiologic results." Those attempts that have been made to construct an argument for biological plausibility for mortality (which is what is driving the large benefits estimates for the proposed standard) have suggested that the susceptible person is very much on the edge of life: for example, "a triggering of a lethal failing of a critical function, such as ...lung fluid balance...in [people] already approaching the limits of tolerance due to preexisting conditions." Under such circumstances, any of a number of air contaminants could have the same effect on the person. I don't find these plausibility arguments a compelling case for PM alone, because (1) these arguments could be used to explain the effects of many other air pollutants or weather patterns, while also (2) there are some very good reasons why the statistical results could be picking up the effect of one of these other possible contributors, as I will now explain.

What are the statistical reasons to doubt that PM is truly causing the observed mortality?

We are faced with a situation where statistical results have not been corroborated by the rest of the sciences. As every first-year statistics student is taught, it is very easy to make big mistakes with statistics in this situation. This is why many of the researchers, whose findings EPA is using, describe PM as a possible "surrogate for" or "correlate of" a yet-to-be-known specific culprit.

In statistical studies cited in EPA's Criteria Document, researchers looked for patterns of association between PM and mortality. The difficulty is that the data to do this contain many types of random variations, and the relationships we are looking for are probably complex. There are many types of statistical errors that one can commit when analyzing data that contain random variations, and there are many ways of trying to avoid or minimize statistical errors. The Criteria Document describes these statistical errors and the potential for misinterpreting statistical results. Due to these potential errors, the Criteria Document states that "confident assignment of...variations in health endpoints to specific air pollutants may still require additional study" and also concludes that "much caution is warranted with regard to derivation or extrapolation of quantitative estimates of increased risks ... based on available epidemiology information."

The question for me has been, How much caution is warranted? Recently, I started to explore the likelihood that these errors might be large enough to affect the overall qualitative picture of PM risks that can emerge from statistical studies. As a result of some numerical experiments of my own, I believe that we need to really look much more closely at the potential errors in the statistical results than EPA has done to date. This is because the PM studies exhibit two distinct types of data problems at the same time. It may seem arcane to worry about combinations of problems, but the common statistical methods for detecting these errors individually don't work when both of the following common data problems are present in the same data set:

(1) Several different pollutants in the data tend to rise and fall with similar patterns (i.e., levels of various pollutants are "correlated"); and

(2) There is more difficulty in getting good estimates of people's actual exposures for some of the pollutants than for others (i.e., there are differences in "measurement errors").

These are very common problems for ambient pollution data. They both occur to a certain degree in all of the PM studies; they occur together. My numerical experiments with these two effects (correlations and differential measurement errors) have suggested to me that the epidemiological conclusions on PM may not only be subject to quantitative inaccuracy, but actually may be at odds with the truth in a qualitative sense. In my numerical experiments, a pollutant that was constructed to have a perfect relationship with the mortality data repeatedly appeared to have no statistically significant relationship. A pollutant that was constructed to have no effect on mortality repeatedly appeared to have a strong and statistically significant effect.

It is not surprising that I could generate such results, since the potential for such errors has been proven theoretically. However, I was surprised at how large and consistent the error in the statistical conclusions was when I used realistic values for degree of correlation and measurement error. If these typical data conditions really can be this effective in getting us to draw incorrect conclusions, then it means that we could be finding consistent statistical evidence implicating fine PM across numerous studies in many locations and over different periods of time, even if fine PM were having little or no causal effect on mortality at all. One or more other factors may be the real cause.

If this potential statistical error cannot be addressed satisfactorily, then reduction of uncertainty about the causative role of PM-10 or PM-2.5 should depend very heavily on obtaining corroborating scientific evidence of a biological mechanism.

But what if we decide to believe there is a fine particle effect anyway?

I have given you my reasons for skepticism about the statistical evidence. But everyone has to draw their own conclusions, and other people may be prepared to believe that there really is a significant fine particle effect. If we were to have confidence there is a fine particle effect, then would we have enough information to set standards that are protective of the public health? I think not. The Shell Game still applies, and at this point, the existing statistical studies do not even pretend to be able to help.

Why? Look at what PM2.5 consists of. Unlike any other criteria pollutant, it is made up of many components, and each component is like another shell that may or may not contain the pea. Particles come from many types of sources, and for each source, the particles consist of very different chemicals and particle sizes. These differences may be highly significant for health. Not one of the available statistical studies on PM has attempted to unravel the roles of all the key types of PM constituents, simply because there are no statistically-usable data about how these constituents vary in different places and at different points in time. As a result, consider the effect on these policy-relevant questions:

-- Are some specific PM constituents creating a toxic effect, while other parts of the PM mix are non-potent?...No one yet knows.

-- Have we deduced the likely importance of the various constituents from biological data?...Not yet.

-- If we require reductions of fine particles generically, can we be confident that the true culprit or culprits will end up being controlled?...No.

The true culprit is not known, and better statistical analysis will not resolve this uncertainty; only better exposure data will. Better laboratory and clinical-level information on health effects will also help. Until we have data that can start to reveal the roles of the constituents in the PM mix, and the role of PM versus other pollutants, we cannot expect to have better answers to these important policy questions. Thus, use of current scientific information to set public policy amounts to playing a classic "Shell Game," even if you believe fine particles cause adverse health effects.

Let me try to illustrate the dilemma by reviewing some of the hypotheses described in the Criteria Document:

-- Some toxicological evidence points not to the fine particles, but the ultrafine particles (e.g., less than 0.1 um in diameter). This would suggest that regulations should target combustion sources that are very close to people, such as automobiles.

-- Another hypothesis relates to how acid the particles are. Acid particles mostly come from sources of SOx and NOx, such as power plants.

-- Yet another hypothesis points to long-term accumulation of particles in the lungs. This would suggest controls on those particles that are not soluble, such as road dusts, and soot from diesel combustion.

The list of hypotheses and potential culprits goes on. It seems unlikely that all of the hypothesized physiological effects will turn out to be equally important. Until we know which of the hypotheses to believe, we run the risk of controlling particles that don't significantly harm the public health. And, we run the risk of not controlling particles that do create a public health hazard. I do not have confidence that we will end up controlling the right constituent if we set a generic fine particle standard as proposed.

How has EPA communicated about these uncertainties in its risk and benefits assessments?

EPA's peer-reviewed Criteria Document for PM describes the pitfalls that need to be considered in the use of the statistical findings, and issues warnings about using the statistical results as an actual dose-response:

"There remains much uncertainty...regarding the shapes of PM exposure-response relationships, the magnitudes and variabilities of risk estimates for PM, the ability to attribute observed health effects to specific PM constituents...and the nature and magnitude of the overall public health risk imposed by ambient PM exposure."

Despite these warnings in the peer-reviewed Criteria Document, EPA's Staff Paper and its Regulatory Impact Analysis have all used the statistically-derived estimates as if they give us a reasonable approximation of a causal relationship, with no uncertainty other than the error bars reported in the single study used for each health endpoint. As I have explained above, those statistically-derived error bars may themselves be unreliable. And, in the case of the benefits ranges in the Regulatory Impact Analysis, even the statistical error bars have been dropped; uncertainty analysis has devolved to two point estimates from two individual studies, and EPA seems to imply that this is the major source of uncertainty in these benefits estimates:

"The uncertainty associated with the benefits estimates are substantial. In particular, benefit estimates vary greatly depending [whether the long-term or short-term mortality study is used to estimate mortality benefits]." (emphasis added).

Thus, EPA has made several very important presumptions in the risk analyses and benefits estimates that it is using to support its proposed PM2.5 standards:

-- EPA's risk analysis presumes that if the statistical indicator or surrogate is controlled, that the actual culprit also will be controlled.

Until we are confident that the statistical association is evidence of causation, this is like the ancient Greek practice of killing the messenger who delivers bad news. For example, ambient levels of PM might simply be correlated with another factor that is the true culprit, such as carbon monoxide or weather. Reducing PM would not produce any health benefits--at the moment it is still only a kind of "statistical messenger", telling us that some kind of health effect exists in our environment.

-- Even if PM2.5 is a problem, EPA's risk analysis also presumes that any action taken to reduce PM2.5 will certainly control the specific culprit.

For example, if organic carbon particles are the culprit, controls on SOx and soot are still assumed to provide health benefits. This is like assuming that we can win the Shell Game no matter what shell we look under.

There are many other types of uncertainties in the risk analysis that EPA's staff also have not incorporated, and which I have described in earlier formal written comments to EPA. Overall, EPA's estimates of the benefits of the proposed PM2.5 standards do not reflect the real uncertainties that statisticians openly acknowledge in their publications, and which EPA describes in its own Criteria Document. The $58 to $119 billion per year of benefits that EPA estimates we will obtain from the proposed PM2.5 standard is actually like a lottery that we might win -- if all of these presumptions are correct. At the same time, there is a substantial probability that the benefits could be very small, even zero.

Does the nation want to play this Shell Game?

This is a valid policy question. Given the large cost of the proposed regulation, it deserves an open public debate tempered with a willingness to acknowledge the true state of scientific understanding. Since the costs of any additional regulation would be undertaken with a degree of uncertainty that has the quality of a Shell Game, it is essential to good public policy that this decision be informed by estimates of risks and benefits that properly reflect the true extent of uncertainty that we are facing.

The state of science leaves a reasonable chance that the proposed PM2.5 standard would not generate any significant benefits at all. In such a situation, it is also reasonable to consider whether there are more effective ways of protecting the public health. I have seen no serious discussion from EPA of the merits of regulatory options other than a generic PM2.5 standard. The proposed PM2.5 standard has not been designed to try to manage the uncertainties I have described. It does not account for or suggest the relevance of trying to maximize the chances that the most likely culprits will be controlled. Why should anyone expect this standard to accidentally hit the right target?

We should try to aim more carefully, with a more thorough consideration of the uncertainties, and of alternatives that can improve our likelihood of achieving the desired public health benefits. I am not suggesting years of delay...I am suggesting better risk management through a more complete assessment of the uncertainties, and a more complete assessment of alternative regulatory approaches.