Correspondence Volume 122 | Issue 1 | January 2014
Environ Health Perspect; DOI:10.1289/ehp.1307701
A Valuable Contribution toward Adopting Systematic Review in Environmental Health
Jennifer McPartland,1 Juleen Lam,1,2 and Colleen Lanier-Christensen3
1Environmental Defense Fund, Washington, DC, USA; 2Department of Health, Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA; 3Department of Sociomedical Sciences, Columbia University Mailman School of Public Health, New York, New York, USA
Citation: McPartland J, Lam J, Lanier-Christensen C. 2014. A valuable contribution toward adopting systematic review in environmental health. Environ Health
Perspect 122:A10; http://dx.doi.org/10.1289/ehp.1307701
J.M. is employed by the Environmental Defense Fund. C.L.-C. was an Environmental Defense Fund summer fellow (summer 2013). J.L. participated in the development of the Navigation Guide (NavGuide), a systematic review methodology designed for application in environmental health. The NavGuide was a collaborative effort between University of California, San Francisco (UCSF) Program on Reproductive Health and the Environment, the U.S. Environmental Protection Agency (EPA), and a variety of other academic, government, and nonprofit organizations. J.L. was an ORISE (Oak Ridge Institute for Science and Education) postdoctoral fellow at the U.S. EPA at the time and did not receive any financial compensation from UCSF for her contributions. The authors declare they have no actual or potential competing financial interests.
Editor’s note: In accordance with journal policy, Krauth et al. were asked whether they wanted to respond to this letter, but they chose not to do so.
Published: 1 January 2014
Scientific and regulatory disagreements and debates routinely arise during the determination of which specific environmental agents are of concern and at what exposure levels. Such debates are common during the development of human health assessments by the U.S. Environmental Protection Agency (EPA) Integrated Risk Information System (IRIS). Indeed, IRIS has been the subject of several congressional hearings and National Academy reviews (U.S. EPA 2012; U.S. House of Representatives Committee on Science, Space, and Technology, Subcommittee on Oversight 2011) precisely because it is in the contentious position of assessing chemical hazard. Recently, IRIS has put forth significant effort to enhance and increase the efficiency of its reviews, which includes refining the process by which it selects, evaluates, and integrates scientific evidence (U.S. EPA 2013)—all central elements of systematic review that require transparent and objective criteria or protocols.
In “Instruments for Assessing Risk of Bias and Other Methodological Criteria of Published Animal Studies: A Systematic Review,” Krauth et al. (2013) provided a much needed initial overview of various instruments proposed to evaluate animal study quality. In our view, four particularly valuable contributions of the paper will be useful to IRIS and others seeking to adopt systematic review approaches for environmental health. First, the authors systematically identified instruments currently available to assess animal study quality. This is the first review of its kind and is invaluable for the further development of such instruments. Second, the authors highlighted the considerable variability found between instruments with regard to origin, number, and type (e.g., risk of bias, reporting) of evaluation criteria. Clearly, application of these different instruments will lead to different conclusions given this degree of variability. The authors’ discussion of differences between criteria related to risk of bias, reporting, and imprecision provides important insight on how different criteria can affect study estimates and, consequently, how each should be considered in the evaluation of evidence. Third, the authors described serious uncertainties regarding the performance of these instruments that must be considered in their application. Two of the findings by Krauth et al. (2013) were particularly striking: First, of the 30 instruments evaluated, only 1 had been tested for validity; and second, only 6 contained at least one criterion empirically supported to systematically bias effect sizes in animal studies. The possibility that these instruments would be used to assert definitive conclusions of study quality is especially worrisome given the current lack of empirical support for their design. Last, the authors pointed to specific criteria that need additional research to determine whether they introduce systematic bias, such as timing of exposure, sex, and funding sources and financial ties of investigators. Building the evidence base around these criteria would improve the development of future instruments intended to evaluate experimental animal studies.
In the clinical field, objective and transparent evidence-based systematic review methods have been used for several decades. These methods have been empirically tested and refined over time for evaluating scientific evidence to assess the effectiveness and potential risks of medical interventions (Guyatt et al. 2011; Higgins and Green 2011). Such standardized review methods are desperately needed for the field of environmental health in order to assess potential human health or environmental impacts of chemical exposures.
In their paper, Krauth et al. (2013) provided a critical overview of instruments available for assessing animal study quality and they indicated where additional research is needed to assess and improve these instruments. We believe this review is extremely valuable to entities such as IRIS and the National Toxicology Program Office of Health Assessment and Translation in their current efforts to establish formal systematic review processes when making authoritative determinations of chemical hazard. In particular, it is evident that the differences and uncertainties identified between available instruments assessed by Krauth et al. (2013) must be addressed if public health protective decisions are to be ensured. We strongly believe that the findings of Krauth et al. make apparent the need for a similar review to be conducted on study evaluation instruments and systematic review approaches currently being developed specifically for application to environmental health.
Higgins JPT, Green S, eds. 2011. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 (updated March 2011). Available: http://handbook.cochrane.org/ [accessed 19 September 2013].
Krauth D, Woodruff TJ, Bero L. 2013. Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. Environ Health Perspect 121:985–992 (2013); doi: 10.1289/ehp.1206389.
U.S. EPA (U.S. Environmental Protection Agency). 2012. EPA Announces NAS’ Review of IRIS Assessment Development Process. Available: http://yosemite.epa.gov/opa/admpress.nsf/0/1ce2a7875daf093485257a000054df54?OpenDocument [accessed 11 December 2013].
U.S. EPA (U.S. Environmental Protection Agency). 2013. Process for Developing IRIS Health Assessments. Available: http://www.epa.gov/IRIS/process.htm [accessed 19 September 2013].
U.S. House of Representatives Committee on Science, Space, and Technology, Subcommittee on Oversight. 2011. EPA’s IRIS Program: Evaluating the Science and Process Behind Chemical Risk Assessment. Available: http://science.house.gov/hearing/investigations-and-oversight-hearing-epas-iris-program [accessed 11 December 2013].
ISEE 2015 Abstracts Now AvailableEHP is pleased to present the abstracts for the 2015 annual conference of the International Society for Environmental Epidemiology (ISEE), “Addressing Environmental Health Inequalities,” held 30 August–3 September 2015 in São Paulo, Brazil.
CEHN September 2015 Article of the Month“Relation of Prenatal Methylmercury Exposure from Environmental Sources to Childhood IQ” (DOI:10.1289/ehp.1408554) has been selected by the Children’s Environmental Health Network (CEHN) as its September 2015 Article of the Month. These CEHN summaries discuss the potential policy implications of current children’s environmental health research.
New Editor-in-ChiefWe are pleased to announce that Sally Perreault Darney has been selected as the new Editor-in-Chief of EHP. Sally comes to EHP from the U.S. Environmental Protection Agency, where she most recently co-led a large research project focused on assessing health disparities in vulnerable groups and providing healthy environments for children. Learn more about Sally and her vision for the journal in the September issue of EHP.
Sign Up to Receive E-mail Alerts
Recent Advance Publications
Alternatives Assessment Frameworks: Research Needs for the Informed Substitution of Hazardous Chemicals
Occupation and Risk of Non-Hodgkin Lymphoma and Its Subtypes: A Pooled Analysis from the InterLymph Consortium
Blood Biomarkers of Late Pregnancy Exposure to Trihalomethanes in Drinking Water and Fetal Growth Measures and Gestational Age in a Chinese Cohort
Birth Weight, Ethnicity, and Exposure to Trihalomethanes and Haloacetic Acids in Drinking Water during Pregnancy in the Born in Bradford Cohort
Prenatal Ambient Air Pollution, Placental Mitochondrial DNA Content, and Birth Weight in the INMA (Spain) and ENVIRONAGE (Belgium) Birth Cohorts