The Impact of Exposure Measurement Error on the Estimated Concentration–Response Relationship between Long-Term Exposure to $\mathbf{P}{\mathbf{M}}_{2.5}$ and Mortality

Exposure measurement error is a central concern in air pollution epidemiology.¹ The problem of exposure measurement error arises from using mismeasured exposure in the analysis given that error in exposure assessment is unavoidable. For decades, researchers have studied and taken steps to control the consequences of exposure error through careful research design and data collection, by evaluating the sensitivity of health effect estimates with respect to some degree of exposure error,^2–4 and by adjusting for exposure error with any number of methods, including regression calibration,⁵ Bayesian approaches,⁶ and simulation extrapolation estimation (SIMEX).^7,8 In the context of air pollution, ideally, monitoring an individual’s personal exposure (e.g., the amount of air pollution inhaled into the lungs) would minimize the exposure error and provide the most accurate measurement of exposure. In practice, however, it is usually not feasible to monitor personal exposure for a long period of time, especially for a large population.⁹

In the past, researchers have used air pollutant concentrations measured at monitoring stations, primarily located in urban areas, as population-averaged exposure measurements.¹⁰ Recently, significant advances in satellite remote sensing technology have enabled exposure assessment at a finer spatial resolution, and many spatiotemporal prediction models (e.g., hierarchical models, fusion models, machine learning models) have been developed to obtain high-resolution predictions of ambient pollutant concentrations.^11–13 Epidemiological studies then have relied on these spatiotemporal predictions at an individual’s residence as surrogate exposures.^14,15 The use of spatiotemporal predictions, although more precise, is still subject to measurement error regarding the extent to which the surrogate is representative of the personal exposure.⁹

Zeger et al.¹⁶ divided the difference between the spatiotemporal ambient air pollution prediction and the error-free personal exposure into three components: the differences between personal and population-averaged exposures, between population-averaged exposure and true ambient level, and between true ambient level and the predicted level. This partition suggests the existence of both classical and Berkson errors. Classical error is assumed to be independent of the true exposure, which occurs when the error-prone exposure is randomly distributed around the truth. Classical error arises when the predicted level is accurate but not precise to the true exposure.¹⁷ Conversely, Berkson error is assumed to be independent of the error-prone exposure and occurs when the true exposure is randomly scattered around the error-prone exposure. Berkson error may result from the difference between individual and population-averaged exposures (i.e., the individual exposure may randomly vary around the population-averaged exposure) and also from the spatial smoothing in the modeling of ambient predictions.¹⁸ In addition to classical and Berkson errors, which are random by themselves, systematic error may also occur owing to the spatial correlation between neighboring areas, particularly for studies that have ambient predictions with coarse spatial resolution.¹⁹ Previous studies have proposed several methodologies to correct for spatially correlated errors, including asymptotic approximation or parametric bootstrap for optimizing exposure model parameters,^20,21 but the performance has been mixed. One challenging part of evaluation and correction for spatially correlated errors is to separate the systematic and random components, whose structures are different from each other.²² In a measurement error setting, it is important to characterize both random and systematic errors, especially the type and degree of errors, as well as their spatial correlations, to comprehensively address measurement error problems.

In reality, the specific impact of exposure error on epidemiological findings varies with the error type, and it is therefore informative to discuss each of them in turn.¹⁶ Classic error, in which the measured exposure is expected to have more variation than the truth, tends to bias the effect estimate toward the null; whereas under Berkson error, the measurement is less variable than the truth, which results in an unbiased effect estimate but greater variance, provided that the true relationship between exposure and outcome is linear.¹⁹ In the presence of confounders, the direction of bias becomes unclear in principle and depends on the correlation of exposure with the confounders.¹⁹ Often, such findings are observed under the condition that the concentration–response relationship is correctly specified in epidemiological analysis: for example, investigators simulate the study outcome based on the error-free exposure and a prespecified concentration–response relationship (mostly linear) and then assume the true relationship when estimating the health effect of error-prone exposure.^23,24 However, there is the possibility that investigators fail to specify the true concentration–response relationship, and the practice of assuming a linear relationship restricts discussion of the impact of exposure error to only that case.

In this work, we focus on long-term exposure to fine particulate matter [PM $\le 2.5\mathrm{\mu }\mathrm{m}$ in aerodynamic diameter (${\mathrm{PM}}_{2.5}$)] because it is one of the most deadly air pollutants based on current research findings.^25,26 We conduct an extensive simulation study to assess the impact of exposure error on the estimated concentration–response relationship between long-term exposure to ${\mathrm{PM}}_{2.5}$ and all-cause mortality across the contiguous United States using high-resolution ambient ${\mathrm{PM}}_{2.5}$ predictions as proxy exposure measures. We incorporate a variety of confounders to ensure our simulations are close to reality. The true exposure level is defined as the ambient concentration at an individual’s ZIP Code of residence. The goal is to generate various plausible scenarios of classical and Berkson errors to assess the possible direction and magnitude of bias in the estimated concentration–response relationship between long-term ${\mathrm{PM}}_{2.5}$ exposure and mortality.

We include a total of 39,401 ZIP Codes of residence without missing values in each variable, covering 95% of all the ZIP Codes used by the U.S. Postal Service in the contiguous United States.³⁸ Among these, 3,871 (9.8%) have monitoring sites and 14,961 (38.0%) have annual ${\mathrm{PM}}_{2.5}$ predictions always $<12{\mathrm{\mu }\mathrm{g}/\mathrm{m}}^3$ during the study period. For general ZIP Codes that have polygon representations, the average distance from the ZIP Code centroid to boundary is $7.3\mathrm{km}$; the 5th and 95th percentiles of the distance from the centroid to boundary are 1.2 and $17.5\mathrm{km}$, respectively. In each replicate, the data set consists of 649,910 observations (i.e., 649,910 unique ZIP Code–year combinations). The annual ${\mathrm{PM}}_{2.5}$ predictions over all the ZIP Codes from 2000 to 2016 range from 0.1 to $30.9{\mathrm{\mu }\mathrm{g}/\mathrm{m}}^3$ and approximately follow a normal distribution with the mean of $9.6{\mathrm{\mu }\mathrm{g}/\mathrm{m}}^3$ and standard deviation of $3.2{\mathrm{\mu }\mathrm{g}/\mathrm{m}}^3$ (Figure 2). The spatially correlated errors have slightly greater variability than independent errors (Figure 3).

Table 2 presents the relative bias between the assumed true effect of error-free exposure and the estimated coefficient of error-prone exposure obtained from epidemiological analysis under three representative specifications of the linear concentration–response model. Simulation results are consistent for each error type and magnitude. Complete results are available at https://doi.org/10.7910/DVN/QVNKPP. Specifically, the biases caused by classical errors are all negative (i.e., toward the null), ranging from $-0.56\%$ to $-5.21\%$ and becoming larger as the magnitude of error increases, whereas Berkson errors result in much smaller biases that range from $-0.09\%$ to 0.12% and are either positive or negative. The size of Berkson error–induced biases does not tend to change as we increase the magnitude of errors. The results remain consistent for errors incorporating spatial correlation among neighboring errors up to $100\mathrm{km}$ (Table S3). The direction of classical error–induced biases remains negative in low- and single-pollutant analyses; compared with those in the main analysis, the size of classical error–induced bias increase in the low-level analysis and decrease in the single-pollutant analysis (Table S4). Consistent with the main results, the Berkson error–induced biases are either positive or negative and remain much smaller than the classical error–induced biases in low- and single-pollutant analyses. For the scenario in which we assume that ${\mathrm{PM}}_{2.5}$ has no effect on mortality ($b_1=0$), the bias occurs in either positive or negative direction and is so small as to be negligible (Tables S5 and S6). The size of error does not affect the inference of fitted effects (${\widehat{b}}_1$) through comparing the width of the confidence intervals (Tables S7 and S8). Varying baseline mortality risk ($b_0$) does not have any impact on size or direction of bias.

Figure 4 illustrates discrepancies between the true and fitted concentration–response curves under representative specifications of the quadratic concentration–response model, taking either a concave ($b_2<0$) or a convex ($b_2>0$) upward shape. Complete simulation results are available at https://doi.org/10.7910/DVN/QVNKPP. The fitted curves are obtained by averaging 1,000 replicated responses at each exposure level. Overall, biases are small in size and become relatively larger as the exposure level increases. When error-prone exposure is modeled with a quadratic polynomial in epidemiological analysis (i.e., the concentration–response relationship is correctly specified), the classical error introduces negative bias, whereas Berkson error does not induce any bias. When error-prone exposure is modeled with a penalized cubic spline (i.e., the concentration–response relationship is misspecified), both types of errors induce negative biases. In each model specification, classical error produces larger bias than Berkson error. Again, varying baseline mortality ($b_0$) does not affect size or direction of bias.

Figure 5 compares the true and fitted concentration–response curves under representative specifications of the soft-threshold concentration–response model with different levels of curvature, where the error-prone exposure is modeled with a penalized cubic spline in epidemiological analysis (i.e., the concentration–response relationship is misspecified). Complete simulation results are available at https://doi.org/10.7910/DVN/QVNKPP. Both types of errors induce little bias at levels $<25{\mathrm{\mu }\mathrm{g}/\mathrm{m}}^3$, where over 99.99% observations fall within this range; at extremely high exposure levels, both types of errors induce negative bias. Biases gets relatively smaller as the curvature increases. In each model specification, classical error induces larger bias than Berkson error.

We also tested biases caused by independent errors without accommodating the spatial correlations among neighboring errors. The biases show the same patterns as for the spatially correlated errors under each scenario but are smaller in size. (Tables S9 and S10, Figures S3 and S4).

In this simulation study, we use ambient ${\mathrm{PM}}_{2.5}$ predictions as proxy exposure measurements at individual residence to evaluate the direction and magnitude of bias caused by exposure measurement error in various plausible concentration–response relationships between long-term exposure to ${\mathrm{PM}}_{2.5}$ and mortality. We find that both classical and Berkson errors can bias the concentration–response curve toward the null, suggesting that the causal determination for the long-term ${\mathrm{PM}}_{2.5}$ exposure and mortality drawn from studies using high-resolution ambient predictions is unlikely to be undermined given that the estimated effect is generally smaller than the truth. Further, all the biases are small in size even when using three times the predicted uncertainty to generate exposure errors, suggesting that epidemiological findings are relatively robust against the exposure measurement error from high-resolution ambient predictions. In our settings, because all residents are assigned to an average exposure level in each ZIP Code each year, the Berkson error may be considered as the dominant error source. Therefore in practice, even smaller bias may be expected. In the low-level analysis, the finding of a greater downward bias in models restricted to people never exposed above the national standard provides assurance that health effects below the standard are not being overestimated. Finally, from our finding that a larger error is associated with larger bias, it is inferred that the use of ambient predictions with a finer spatial resolution will result in smaller bias, and that previous studies that use less accurate predictions are more likely biased downward. Indeed, a recent meta-regression found that studies that have cruder exposure estimates tended to have lower estimated effect sizes.⁴²

The observed small magnitude of biases is consistent with Gryparis et al.,²² who found relatively little bias when using spatiotemporal predictions in health effect estimate with an average exposure metric of 9 months. The underlying reason is that the long-term average concentrations are relatively stable, which is consistent with Pinto et al.⁴³ and Wang et al.⁴⁴ who observed low levels of variability of annual ${\mathrm{PM}}_{2.5}$ from monitoring sites. Given the reasonably good performance of our ${\mathrm{PM}}_{2.5}$ prediction model, the annual level predicted and monitored ${\mathrm{PM}}_{2.5}$ are highly consistent, which leads to generated errors being small. As a result, we obtain slight differences between error-prone and error-free exposures, as well as between their variability, with a variance ratio of the error-prone and error-free exposures ranging from 0.97 to 1.03. By allowing the variance ratio to take values between 0.5 and 2, Butland et al.² observed up to 65% of relative bias caused from exposure error. However, their settings were at the daily level and from ambient predictions with lower accuracy, which may not reflect annual exposures and the higher accuracy of modern spatiotemporal models.^45,46

Despite the small magnitude of bias, our findings confirm previously published results suggesting the negative direction of bias caused by classical error, including those by Butland et al.,²⁴ Samoli et al.,⁴⁷ Kioumourtzoglou et al.,²³ and Goldman et al.¹⁸ Assuming a linear concentration–response relationship between ${\mathrm{PM}}_{2.5}$ and mortality, we find either positive or negative bias caused by Berkson error, which is consistent with the results of Bateson et al.,⁵ although the magnitude was even smaller or trivial. Further, we simultaneously consider the two important issues in air pollution epidemiology: exposure measurement error and misspecification of concentration–response relationship between exposure and outcome.⁴⁸ When the nonlinear concentration–response relationship is misspecified, we find that both classical and Berkson errors produce negative bias, given that the nonlinearity has been adequately captured by the penalized cubic spline. In particular, for the soft-threshold models, the fitted penalized splines using error-prone exposures detect the thresholds with good accuracy and no upward bias, suggesting that exposure errors are unlikely to impede detection of the thresholds, if present. In line with Richmond-Bryant and Long,¹⁹ our findings suggest that the specific impact of exposure error is complex and depends on multiple factors, such as error type and magnitude, the concentration–response relationship between exposure and outcome, and the degree of misspecification. Because nature is rarely so simple as assumed in a quadratic or soft-threshold relationship, further investigation is clearly warranted.

The spatial correlation of the exposure error is an important concern in driving the unreliability of health effect estimates.¹⁹ In our study, the advanced ensemble approach used to predict air pollution concentrations demonstrate high accuracy compared with the observed monitored data, and the lack of bias and high level of cross-validated $R^2$ suggests that the prediction model is not overfitted. In this scenario, the potential for very strong correlations among neighboring errors is likely to be low. To accommodate spatial correlation, we incorporated spatial correlation estimated from neighboring monitoring sites, resulting in slightly greater variability of exposure error and larger bias in the health effect estimates, which is consistent with Goldman et al.,⁴⁹ who identified negative bias related to the spatial correlation. In epidemiological studies, although systematic error may exist when spatial correlation is not adequately adjusted, the health effect estimates are underestimated and are thus conservative.

We avoid the temporal correlation of daily differences between predicted and monitored levels by sampling them every other day and considering them independent. Given the low levels of serial correlation, this simplified sampling approach is at least approximately correct. By comparison, the root mean square error (RMSE) of daily predictions for each year reported by Di et al.,²⁷ on which the simulations are based, ranges between 2.404 and 3.385, which is relatively consistent with the average standard deviation of our sampled daily differences of 2.101. The consistency brings value through validity of our uncertainty estimates and so the reliability of conclusions of simulations. The RMSEs are slightly larger than our estimate because they measured the unconditional (overall) uncertainty across grids, whereas our estimate measures the conditional uncertainty within each grid.

This study includes neighborhood-level covariates in its simulations, including control for other air pollutants that may be correlated with ${\mathrm{PM}}_{2.5}$ and its error. It does not include any individual characteristics such as sex, race/ethnicity, age, or socioeconomic status because we believe they are unlikely to be confounders in the analysis. In principle, ambient air pollution in neighborhoods would not be influenced by individual-level covariates. Rather, it is neighborhoods with lower levels of household income, higher smoking and obesity rates, and so on, that may also be neighborhoods with higher air pollution levels.⁵⁰ This is because, for example, if a person with high income were to move to a low-income neighborhood, they would receive an exposure level associated with characteristics at the neighborhood level and not at the individual level. Hence, only neighborhood-level covariates are the confounders, which we have controlled for. Similar arguments apply to personal behavioral stressors—such as the use of tobacco, alcohol, or drugs; exercising; or transportation: We think that air pollution is spatially patterned in neighborhoods with different neighborhood-level behavioral characteristics, not by whether individuals have higher or lower exposures.⁵¹ Since the exposures are air pollution levels in the neighborhoods, it is the neighborhood characteristics that are the confounders.

The large geographical area covered and comprehensive set of scenarios tested are the major strengths of this study. Although relatively new in air pollution epidemiology, the soft-threshold concentration–response model introduces more flexibility by assuming that even if there is a threshold for a health effect, its level will have some distribution, making the soft-threshold model closer to reality than the hard-threshold model.⁵² In addition, although this study has a cohort design for the long-term exposure, the Poisson regression we posited is widely used for time-series analysis to investigate short-term exposures²³; hence our findings may be generalized to other settings, although the magnitude and type of error may differ. Further, our ambient ${\mathrm{PM}}_{2.5}$ predictions have been used for a large number of studies, and thus our results may be directly applicable to them.^14,26,53 Finally, although this study is motivated by ${\mathrm{PM}}_{2.5}$, our proposed approach is applicable to other pollutants and other areas dealing with contextual data, such as neighborhood socioeconomic status, walkability, noise, or nature environments. However, the results may differ because of the differences in exposure errors.

Our simulation framework and the scenarios discussed makes room for new and different questions to be raised and opportunities for further research. First, the impact of exposure measurement error is influenced by many other factors, such as the measurement error of confounders, and whether and how the errors of exposure and confounders are related.²⁴ Such issues are very specific to the nature of confounders and may be considered in future studies. Second, because air pollution and ${\mathrm{PM}}_{2.5}$ itself are complex mixtures of various components, it will be informative to consider multipollutant scenarios to identify which ones most bias the health effect estimates. However, given that the components are highly positively related,⁵⁴ it is challenging to characterize the error size and structure of each component and to separate their independent impacts. It is also important for subsequent analyses to address the problem of effect transfer from a pollutant measured with larger error, such as ${\mathrm{NO}}_2$, to that measured with less error, such as ${\mathrm{PM}}_{2.5}$.⁵⁵ Third, although we consider many different scenarios for the concentration–response relationship between ${\mathrm{PM}}_{2.5}$ exposure and mortality, we take only two scenarios for the error type: 100% classical or 100% Berkson. It would be interesting for future studies to explore the error decomposition and evaluate the impact of mixture error, which is generally accepted to be more realistic.⁵⁶ Finally, assessing the impacts of exposure measurement error by region or urbanicity is an important unanswered question given that less monitored areas likely have larger errors. In this work, our findings are likely conservative, in that we have given equal weighting to those more error-prone, but less populous, locations.

Exposure measurement error–induced bias is pervasive but commonly ignored in air pollution epidemiology. In practice, because hybrid spatiotemporal predictions are being used more often in recent years, our findings offer an opportunity to interpret whether the bias might or might not plausibly account for the relations observed in epidemiological studies, in particular whether the bias is upward. When drawing conclusions about general causations and policy implications, we hope more epidemiological studies will include a quantitative estimate of the effect of measurement error on the results,⁵⁷ along with other bias correction techniques of, for example, Alexeeff et al.⁸ and Hart et al.,⁵⁸ to treat measurement error more seriously. Now that several high-resolution, well-validated prediction models of ambient pollutant concentrations are publicly accessible,^28,30,31 we hope these data sets will be more widely used.

Our simulations investigate the direction and magnitude of bias caused by exposure measurement error in the estimated concentration–response relationship between long-term exposure to ${\mathrm{PM}}_{2.5}$ and all-cause mortality across the contiguous United States, under an extensive range of scenarios. Using high-resolution ambient ${\mathrm{PM}}_{2.5}$ predictions as proxy exposure measurements, our results demonstrate that both classical and Berkson errors can bias the concentration–response curve toward the null, suggesting that the causal determination for the long-term ${\mathrm{PM}}_{2.5}$ exposure and mortality drawn from studies using high-resolution ambient predictions is unlikely to be undermined given that the estimated effect is generally smaller than the truth. Further, the small magnitude of biases suggests that epidemiological findings are relatively robust against exposure measurement error from high-resolution ambient predictions. Finally, from our finding that a larger error is associated with larger bias, it is inferred that the use of ambient predictions with a finer spatial resolution will result in smaller bias.

This publication was made possible by the U.S. Environmental Protection Agency (EPA) grants RD-8358720 and RD-83587201-0 (both to J.D.S.). Its contents are solely the responsibility of the grantee and do not necessarily represent the official views of the U.S. EPA. Further, the U.S. EPA does not endorse the purchase of any commercial products or services mentioned in the publication. This publication was also made possible by National Institutes of Health grants R01ES032418-01 (to J.D.S.), R01AG074357-01 (to L.S.), and ES-000002 (to J.D.S.).

The grid-level fine particulate matter (${\mathrm{PM}}_{2.5}$) data are available at https://doi.org/10.7927/0rvr-4538. The grid-level ozone (${\mathrm{O}}_3$) data are available at https://doi.org/10.7927/a4mb-4t86. The grid-level nitrogen dioxide (${\mathrm{NO}}_2$) data are available at https://doi.org/10.7927/f8eh-5864. The ZIP Code-level ${\mathrm{PM}}_{2.5}$, ${\mathrm{O}}_3$, and ${\mathrm{NO}}_2$, and ${\mathrm{PM}}_{2.5}$ uncertainty data are available from the corresponding author on reasonable request. Other covariate data are publicly available with sources described in the manuscript. Data availability that relates to predictors in the ${\mathrm{PM}}_{2.5}$ prediction model can be referred to Di et al.²⁷

All authors declare they have no actual or potential competing financial interest.