Correspondence Volume 122 | Issue 1 | January 2014
Environ Health Perspect; DOI:10.1289/ehp.1307701
A Valuable Contribution toward Adopting Systematic Review in Environmental Health
Jennifer McPartland,1 Juleen Lam,1,2 and Colleen Lanier-Christensen3
1Environmental Defense Fund, Washington, DC, USA; 2Department of Health, Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA; 3Department of Sociomedical Sciences, Columbia University Mailman School of Public Health, New York, New York, USA
Citation: McPartland J, Lam J, Lanier-Christensen C. 2014. A valuable contribution toward adopting systematic review in environmental health. Environ Health
Perspect 122:A10; http://dx.doi.org/10.1289/ehp.1307701
J.M. is employed by the Environmental Defense Fund. C.L.-C. was an Environmental Defense Fund summer fellow (summer 2013). J.L. participated in the development of the Navigation Guide (NavGuide), a systematic review methodology designed for application in environmental health. The NavGuide was a collaborative effort between University of California, San Francisco (UCSF) Program on Reproductive Health and the Environment, the U.S. Environmental Protection Agency (EPA), and a variety of other academic, government, and nonprofit organizations. J.L. was an ORISE (Oak Ridge Institute for Science and Education) postdoctoral fellow at the U.S. EPA at the time and did not receive any financial compensation from UCSF for her contributions. The authors declare they have no actual or potential competing financial interests.
Editor’s note: In accordance with journal policy, Krauth et al. were asked whether they wanted to respond to this letter, but they chose not to do so.
Published: 1 January 2014
Scientific and regulatory disagreements and debates routinely arise during the determination of which specific environmental agents are of concern and at what exposure levels. Such debates are common during the development of human health assessments by the U.S. Environmental Protection Agency (EPA) Integrated Risk Information System (IRIS). Indeed, IRIS has been the subject of several congressional hearings and National Academy reviews (U.S. EPA 2012; U.S. House of Representatives Committee on Science, Space, and Technology, Subcommittee on Oversight 2011) precisely because it is in the contentious position of assessing chemical hazard. Recently, IRIS has put forth significant effort to enhance and increase the efficiency of its reviews, which includes refining the process by which it selects, evaluates, and integrates scientific evidence (U.S. EPA 2013)—all central elements of systematic review that require transparent and objective criteria or protocols.
In “Instruments for Assessing Risk of Bias and Other Methodological Criteria of Published Animal Studies: A Systematic Review,” Krauth et al. (2013) provided a much needed initial overview of various instruments proposed to evaluate animal study quality. In our view, four particularly valuable contributions of the paper will be useful to IRIS and others seeking to adopt systematic review approaches for environmental health. First, the authors systematically identified instruments currently available to assess animal study quality. This is the first review of its kind and is invaluable for the further development of such instruments. Second, the authors highlighted the considerable variability found between instruments with regard to origin, number, and type (e.g., risk of bias, reporting) of evaluation criteria. Clearly, application of these different instruments will lead to different conclusions given this degree of variability. The authors’ discussion of differences between criteria related to risk of bias, reporting, and imprecision provides important insight on how different criteria can affect study estimates and, consequently, how each should be considered in the evaluation of evidence. Third, the authors described serious uncertainties regarding the performance of these instruments that must be considered in their application. Two of the findings by Krauth et al. (2013) were particularly striking: First, of the 30 instruments evaluated, only 1 had been tested for validity; and second, only 6 contained at least one criterion empirically supported to systematically bias effect sizes in animal studies. The possibility that these instruments would be used to assert definitive conclusions of study quality is especially worrisome given the current lack of empirical support for their design. Last, the authors pointed to specific criteria that need additional research to determine whether they introduce systematic bias, such as timing of exposure, sex, and funding sources and financial ties of investigators. Building the evidence base around these criteria would improve the development of future instruments intended to evaluate experimental animal studies.
In the clinical field, objective and transparent evidence-based systematic review methods have been used for several decades. These methods have been empirically tested and refined over time for evaluating scientific evidence to assess the effectiveness and potential risks of medical interventions (Guyatt et al. 2011; Higgins and Green 2011). Such standardized review methods are desperately needed for the field of environmental health in order to assess potential human health or environmental impacts of chemical exposures.
In their paper, Krauth et al. (2013) provided a critical overview of instruments available for assessing animal study quality and they indicated where additional research is needed to assess and improve these instruments. We believe this review is extremely valuable to entities such as IRIS and the National Toxicology Program Office of Health Assessment and Translation in their current efforts to establish formal systematic review processes when making authoritative determinations of chemical hazard. In particular, it is evident that the differences and uncertainties identified between available instruments assessed by Krauth et al. (2013) must be addressed if public health protective decisions are to be ensured. We strongly believe that the findings of Krauth et al. make apparent the need for a similar review to be conducted on study evaluation instruments and systematic review approaches currently being developed specifically for application to environmental health.
Higgins JPT, Green S, eds. 2011. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 (updated March 2011). Available: http://handbook.cochrane.org/ [accessed 19 September 2013].
Krauth D, Woodruff TJ, Bero L. 2013. Instruments for assessing risk of bias and other methodological criteria of published animal studies: a systematic review. Environ Health Perspect 121:985–992 (2013); doi: 10.1289/ehp.1206389.
U.S. EPA (U.S. Environmental Protection Agency). 2012. EPA Announces NAS’ Review of IRIS Assessment Development Process. Available: http://yosemite.epa.gov/opa/admpress.nsf/0/1ce2a7875daf093485257a000054df54?OpenDocument [accessed 11 December 2013].
U.S. EPA (U.S. Environmental Protection Agency). 2013. Process for Developing IRIS Health Assessments. Available: http://www.epa.gov/IRIS/process.htm [accessed 19 September 2013].
U.S. House of Representatives Committee on Science, Space, and Technology, Subcommittee on Oversight. 2011. EPA’s IRIS Program: Evaluating the Science and Process Behind Chemical Risk Assessment. Available: http://science.house.gov/hearing/investigations-and-oversight-hearing-epas-iris-program [accessed 11 December 2013].
New Impact Factor
We are pleased to announce our new impact factor of 7.03 and—for the first time—our inclusion in the category of Toxicology. Thank you to our authors and readers, who continue to make EHP a leading journal in Environmental Sciences, in Public, Environmental, and Occupational Health, and now in Toxicology.
EHP News Feature Wins Award
Our March 2013 cover story, “A Tale of Two Forests: Addressing Postnuclear Radiation at Chernobyl and Fukushima,” has won first place in the Outstanding Feature Story category of the Society of Environmental Journalists Awards for Reporting on the Environment. The judges called the story “an exceedingly vital and original piece of environmental feature writing.” Congratulations to authors Winifred A. Bird and Jane Braxton Little on their exciting achievement!
CEHN August 2014 Article of the Month
“Gestational Exposure to Endocrine-Disrupting Chemicals and Reciprocal Social, Repetitive and Stereotypic Behaviors in 4- and 5-Year-Old Children: The HOME Study” (Environ Health Perspect; DOI:10.1289/ehp.1307261) has been selected by the Children’s Environmental Health Network (CEHN) as its August 2014 Article of the Month. These CEHN summaries discuss the potential policy implications of current children’s environmental health research.
Register Today for PPTOX IVEnvironmental Stressors in Disease and Implications for Human Health
October 26–29, 2014
Boston Marriott Long Wharf Hotel
Join the Endocrine Society for the fourth international summit of Prenatal Programming and Toxicity (PPTOX), a premiere conference series dedicated to cutting-edge discussion of environmental hazards during early life and long-term consequences.Key deadlines:
* Abstract submission: Tuesday, September 2
* Housing deadline: Monday, October 6
For more information, visit the Endocrine Society website.
Sign Up to Receive E-mail Alerts
Recent Advance Publications
- Risks and Benefits of Green Spaces for Children: A Cross-Sectional Study of Associations with Sedentary Behavior, Obesity, Asthma, and Allergy
- Genistein Disrupts Glucocorticoid Receptor Signaling in Human Uterine Endometrial Ishikawa Cells
- Modification of the Association between PM10 and Lung Function Decline by Cadherin 13 Polymorphisms in the SAPALDIA Cohort: A Genome-Wide Interaction Analysis
- Mitochondria, Energetics, Epigenetics, and Cellular Responses to Stress
- Incense Use and Cardiovascular Mortality among Chinese in Singapore: The Singapore Chinese Health Study
- Perfluorochemicals and Human Semen Quality: The LIFE Study
- Molecular Signaling Network Motifs Provide a Mechanistic Basis for Cellular Threshold Responses
- CareerOpp: Asst/assoc prof of food for health and wellness, Univ Georgia, Athens, GA http://t.co/I6vfAlMBLt
- EHPNoonNews: NIH issues finalized policy on genomic data sharing http://t.co/pcZNltOdGD @nih
- EHPNoonNews: Global trash burning produces more pollution than expected http://t.co/PQ95YANueh @ap
- EHPNoonNews: Environmental emergencies put science advisers to the test http://t.co/YwrpJNisEH @nature
- EHPNoonNews: Consumer Reports summarizes fish mercury guidance http://t.co/IgQvaD7x8F @consumerreports