Skip to content
EHP Banner Ad

EHP

Facebook Page EHP Twitter Feed Open Access icon  

Reviews November 2016 | Volume 124 | Issue 11

Email this to someoneShare on FacebookTweet about this on TwitterShare on LinkedInShare on Google+Share on StumbleUpon
Environ Health Perspect; DOI:10.1289/EHP233

The Next Generation of Risk Assessment Multi-Year Study—Highlights of Findings, Applications to Risk Assessment, and Future Directions

Ila Cote,1 Melvin E. Andersen,2 Gerald T. Ankley,3 Stanley Barone,4 Linda S. Birnbaum,5,6 Kim Boekelheide,7 Frederic Y. Bois,8 Lyle D. Burgoon,9 Weihsueh A. Chiu,10 Douglas Crawford-Brown,11 Kevin M. Crofton,12 Michael DeVito,5,6 Robert B. Devlin,13 Stephen W. Edwards,13 Kathryn Z. Guyton,14 Dale Hattis,15 Richard S. Judson,12 Derek Knight,16 Daniel Krewski,17 Jason Lambert,18 Elizabeth Anne Maull,5,6 Donna Mendrick,19 Gregory M. Paoli,20 Chirag Jagdish Patel,21 Edward J. Perkins,22 Gerald Poje,23 Christopher J. Portier,24 Ivan Rusyn,10 Paul A. Schulte,25 Anton Simeonov,26 Martyn T. Smith,27 Kristina A. Thayer,5,6 Russell S. Thomas,12 Reuben Thomas,28 Raymond R. Tice,5,6 John J. Vandenberg,1 Daniel L. Villeneuve,3 Scott Wesselkamper,18 Maurice Whelan,29 Christine Whittaker,25 Ronald White,30 Menghang Xia,26 Carole Yauk,31 Lauren Zeise,32 Jay Zhao,18 and Robert S. DeWoskin1

Author Affiliations open
1National Center for Environmental Assessment, U.S. Environmental Protection Agency (EPA), Washington, District of Columbia, USA; 2ScitoVation, Research Triangle Park, North Carolina, USA; 3National Health and Environmental Effects Research Laboratory, U.S. EPA, Duluth, Minnesota, USA; 4Office of Chemical Safety and Pollution Prevention, U.S. EPA, Washington, District of Columbia, USA; 5National Institute of Environmental Health Sciences, and 6National Toxicology Program, National Institutes of Health (NIH), Department of Health and Human Services (DHHS), Research Triangle Park, North Carolina, USA; 7Department of Pathology and Laboratory Medicine, Brown University, Providence, Rhode Island, USA; 8Unité Modèles pour l’Écotoxicologie et la Toxicologie, Institut National de l’Environnement Industriel et des Risques, Verneuil en Halatte, France; 9U.S. Army Engineer Research and Development Center, Research Triangle Park, North Carolina, USA; 10Department of Veterinary Integrative Biosciences, College of Veterinary Medicine and Biomedical Sciences, Texas A&M University, College Station, Texas, USA; 11Department of Land Economy, University of Cambridge, Cambridge, England; 12National Center for Computational Toxicology, and 13National Health and Environmental Effects Research Laboratory, U.S. EPA, Research Triangle Park, North Carolina, USA; 14International Agency for Cancer Research, Lyon, France; 15George Perkins Marsh Institute, Clark University, Worcester, Massachusetts, USA; 16European Chemicals Agency, Annankatu, Helsinki, Finland; 17McLaughlin Centre for Population Health Risk Assessment, University of Ottawa, Ottawa, Ontario, Canada; 18National Center for Environmental Assessment, U.S. EPA, Cincinnati, Ohio, USA; 19National Center for Toxicological Research, Food and Drug Administration, Jefferson, Arkansas, USA; 20Risk Sciences International, Ottawa, Ontario, Canada; 21Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA; 22U.S. Army Engineer Research and Development Center, Vicksburg, Mississippi, USA; 23Grant Consulting Group, Washington, District of Columbia, USA; 24Environmental Defense Fund, Washington, District of Columbia, USA; 25Education and Information Division, National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention, Cincinnati, Ohio, USA; 26National Center for Advancing Translational Sciences, NIH, DHHS, Bethesda, Maryland, USA; 27Division of Environmental Health Sciences, School of Public Health, University of California, Berkeley, Berkeley, California, USA; 28Gladstone Institutes, University of California, San Francisco, San Francisco, California, USA; 29Systems Toxicology Unit, European Commission Joint Research Centre, Ispra, Italy; 30Center for Effective Government, Washington, District of Columbia, USA; 31Environmental Health Science and Research Bureau, Health Canada, Ottawa, Ontario, Canada; 32Office of Environmental Health Hazard Assessment, California EPA, Oakland, California, USA

PDF icon PDF Version (349 KB)

  • Background: The Next Generation (NexGen) of Risk Assessment effort is a multi-year collaboration among several organizations evaluating new, potentially more efficient molecular, computational, and systems biology approaches to risk assessment. This article summarizes our findings, suggests applications to risk assessment, and identifies strategic research directions.

    Objective: Our specific objectives were to test whether advanced biological data and methods could better inform our understanding of public health risks posed by environmental exposures.

    Methods: New data and methods were applied and evaluated for use in hazard identification and dose–response assessment. Biomarkers of exposure and effect, and risk characterization were also examined. Consideration was given to various decision contexts with increasing regulatory and public health impacts. Data types included transcriptomics, genomics, and proteomics. Methods included molecular epidemiology and clinical studies, bioinformatic knowledge mining, pathway and network analyses, short-duration in vivo and in vitro bioassays, and quantitative structure activity relationship modeling.

    Discussion: NexGen has advanced our ability to apply new science by more rapidly identifying chemicals and exposures of potential concern, helping characterize mechanisms of action that influence conclusions about causality, exposure–response relationships, susceptibility and cumulative risk, and by elucidating new biomarkers of exposure and effects. Additionally, NexGen has fostered extensive discussion among risk scientists and managers and improved confidence in interpreting and applying new data streams.

    Conclusions: While considerable uncertainties remain, thoughtful application of new knowledge to risk assessment appears reasonable for augmenting major scope assessments, forming the basis for or augmenting limited scope assessments, and for prioritization and screening of very data limited chemicals.

  • Citation: Cote I, Andersen ME, Ankley GT, Barone S, Birnbaum LS, Boekelheide K, Bois FY, Burgoon LD, Chiu WA, Crawford-Brown D, Crofton KM, DeVito M, Devlin RB, Edwards SW, Guyton KZ, Hattis D, Judson RS, Knight D, Krewski D, Lambert J, Maull EA, Mendrick D, Paoli GM, Patel CJ, Perkins EJ, Poje G, Portier CJ, Rusyn I, Schulte PA, Simeonov A, Smith MT, Thayer KA, Thomas RS, Thomas R, Tice RR, Vandenberg JJ, Villeneuve DL, Wesselkamper S, Whelan M, Whittaker C, White R, Xia M, Yauk C, Zeise L, Zhao J, DeWoskin RS. 2016. The Next Generation of Risk Assessment multiyear study—highlights of findings, applications to risk assessment, and future directions. Environ Health Perspect 124:1671–1682; http://dx.doi.org/10.1289/EHP233

    Address correspondence to I. Cote, U.S. Environmental Protection Agency, Region 8, Room 8152, 1595 Wynkoop St., Denver, CO 80202-1129 USA. Telephone: (202) 288-9539. E-mail: cote.ila@epa.gov

    We would like to thank P. Preuss, K. Olden, D. Dix, T. Bahadori, R. Clark, and D. Walsh [U.S. Environmental Protection Agency (EPA)] and K. Osborne (ICF) for their unflagging support of this multiyear project.

    The views in this article are solely those of the authors and do not necessarily reflect the views or policies of their agencies or institutes including (but not limited to) the U.S. EPA, U.S. Food and Drug Administration (FDA), National Institute for Environmental Health Sciences (NIEHS), National Institute for Occupational Safety and Health (NIOSH), California EPA Office of Environmental Health Hazard Assessment (OEHHA), or European Chemicals Agency.

    K.B. is an occasional expert consultant for chemical and pharmaceutical companies, including Boehringer Ingelheim and Sangamo, and owns stock in and is a consultant for CytoSolv, an early stage biotechnology company. D.K. (D. Krewski) and G.M.P. are subcontractors to ICF International, Durham, NC, USA. G.P. is employed by Grant Consulting Group, Washington, DC, USA. D.K. (D. Krewski) is a Principal at Risk Sciences International (RSI), a Canadian company established in partnership with the University of Ottawa in 2006. RSI contributed to the development of the U.S. EPA NexGen framework under a subcontract with ICF International. M.T.S. has received consulting and expert testimony fees from lawyers representing both plaintiffs and defendants in cases involving claims related to exposure to benzene. These activities are unrelated to the current work.

    The remaining authors declare they have no actual or potential competing financial interests.

    Received: 12 June 2015
    Revised: 30 October 2015
    Accepted: 29 March 2016
    Published: 19 April 2016

    Note to readers with disabilities: EHP strives to ensure that all journal content is accessible to all readers. However, some figures and Supplemental Material published in EHP articles may not conform to 508 standards due to the complexity of the information being presented. If you need assistance accessing journal content, please contact ehponline@niehs.nih.gov. Our staff will work with you to assess and meet your accessibility needs within 3 working days.

  • PDF icon Supplemental Material PDF (212 KB)

    Note to readers with disabilities: EHP has provided a 508-conformant table of contents summarizing the Supplemental Material for this article (see below) so readers with disabilities may determine whether they wish to access the full, nonconformant Supplemental Material. If you need assistance accessing journal content, please contact ehponline@niehs.nih.gov. Our staff will work with you to assess and meet your accessibility needs within 3 working days.

    PDF icon Supplemental Table of Contents PDF (114 KB)

Introduction

Background

Advances in molecular and cell biology provide new insights into the etiology of human disease, largely by evaluating molecular events that influence cell function and interactions (Audouze et al. 2013; Hood and Tian 2012; McCullough et al. 2014, 2016; McHale et al. 2012; Thomas R. et al. 2014). High-throughput and high-content (HT/HC) assays and robotic implementation are generating data streams at unprecedented speeds. Computational tools, automated analytical methods (bioinformatics), and systems biology approaches are being developed to organize and interpret the information (Attene-Ramos et al. 2013, 2015; U.S. EPA 2016; Freitas et al. 2014; Hsu et al. 2014; Huang et al. 2014, 2016; Judson et al. 2011, 2012, 2013, 2014, 2015). The National Library of Medicine databases, Tox21 (Toxicity Testing in the 21st Century), and ToxCast™ (Toxicity ForeCaster) are among the efforts to compile, organize, manage, and store these data to better understand determinants of population health (U.S. EPA 2016; Krewski et al. 2014; NRC 2009) and to help answer such questions as: Which chemicals are environmentally better choices in commerce? Why do individuals and specific subpopulations respond differently to chemical exposures? What happens when people are exposed to low levels of multiple chemicals? How do factors like socioeconomic status and pre-existing illness influence public health risk? How might evaluating and applying these data, methods, and models support environmental health decisions?

A revolution in molecular, computational, and systems biology has occurred over the past 25 years, providing dramatic insights into the causation of disease. This new science, however, has not been extensively incorporated into environmental health risk assessment, although much related research is occurring. To evaluate how new data types and approaches can inform environmental health risk assessments, the U.S. Environmental Protection Agency (EPA) collaborated with several U.S. and international agencies and organizations (see Table S1). We considered the state of science and developed illustrative prototypes (case studies) demonstrating various approaches that investigators could apply to different risk management problems. Our goal was to provide examples that would promote discussion in the risk assessment, risk management, and stakeholder communities and that would facilitate the transition from strategy to practical application.

In this article, we summarize the results of more than 40 separate publications resulting from our collaborative efforts, along with a few key papers by other authors; identify potential application to risk assessment; and articulate strategic research directions. A detailed report of our efforts with an extensive review of the general literature (~ 400 references) is also available (U.S. EPA 2014). Toxicity testing and risk assessment are anticipated to benefit from these advances (Krewski et al. 2014; NRC 2007).

Objectives

Our specific objectives were to test whether new data sources and risk assessment methods would help a) identify specific patterns of molecular events that are associated with impacts of chemical exposures (hazard identification); b) characterize exposure–dose within the range of environmental exposures (dose–response); c) inform risk factors such as genomic variants, chemical and nonchemical stressor co-exposures (risk modifiers); and d) improve indicators of adverse health effects and chemical potency (toxicity surrogates). We also considered how new types of assessments might address differing risk management needs or risk context and help develop decision rules for integrating and applying the available data.

Methods

We applied and evaluated diverse types of data and methods to determine if, and how, advanced biological data would better inform risk assessments.

Preparation for Prototype Development

To establish the foundation for this effort, we a) worked with the U.S. Environmental Protection Agency (EPA) risk managers to identify research needs and develop a strategy for the overall approach (Cote et al. 2012); b) consulted with experts on the concepts for the prototypes (U.S. EPA 2011a); c) held a stakeholder conference to inform the public about upcoming activities and to solicit advice (U.S. EPA 2011b); and d) developed a framework articulating the guiding principles for NexGen (Krewski et al. 2014).

Risk Assessments Targeted to Various Decision Contexts

We developed eight prototypes illustrating three decision contexts generally representing environmental challenges risk managers face:

  • Major scope decisions, usually regulatory decision-making, generally aimed at nationwide exposures and associated risks.
  • Limited scope decisions, often non-regulatory decision-making, generally aimed at limited exposure, hazard, or data situations.
  • Chemical screening and prioritization for further testing, research, or assessment or for emergency response (Figure 1). Decision contexts were derived from observation of problems commonly faced by the U.S. EPA (NRC 2009). These generalized decision contexts do not, and are not meant to, capture all decisions or situational nuances risk managers face.

Figure 1 - Conceptual DiagramFigure 1 – Three broad decision-context categories are shown across the top (white type); the eight “fit-for-purpose” prototypes developed for this effort are shown in black type. From left to right in Figure 1, the amount of traditional toxicological data available for assessment (e.g., in vivo rodent toxicity data, epidemiology data) and the confidence in the assessment conclusions decrease, but the number of chemicals that can be evaluated increases markedly.

Note: B[a]P, benzo[a]pyrene; PAHs, polycyclic aromatic hydrocarbons.

View larger image (TIF File)

Study Selection

Establishing systematic review criteria for study selection helps ensure reproducibility, transparency, and scientific acceptability of regulatory actions (McConnell et al. 2014). Our criteria were similar to those used for traditional data (e.g., adequate study design and reporting), augmented with additional criteria specifically applicable to new methodologies (Bourdon-Lacombe et al. 2015; McConnell et al. 2014). Rapidly evolving best practices for advanced biology and certain reporting requirements led many initially considered studies to be deemed inadequate for risk assessment purposes (U.S. EPA 2013b, 2014; McConnell et al. 2014).

The Prototypes

This section provides an overview of the science considered in the prototypes. Table 1 (adapted from Krewski et al. 2014) summarizes tools and techniques evaluated in the prototypes, organized by decision context. While the tools and techniques are categorized here for simplicity, they represent a continuum of methods that can be applied in various combinations to address agency needs. Additional details are provided both in the papers referenced throughout and in U.S. EPA (2014).

Table 1 - Select View Table (HTML Version) for a 508-conformant versionTable 1 – Prototype use of new scientific tools and techniques applied (1) or not applied (0) (adapted from Krewski et al. 2014).

View Table (HTML Version)
View larger image (TIF File)

Major-scope assessment prototypes. Three major-scope prototypes explored how toxicogenomic studies of exposed human populations can inform risk assessment:

  • Characterizing early key events in the biological cascade that results in adverse outcomes.
  • Identifying and characterizing biomarkers of exposure and effects.
  • Identifying factors contributing to population variability and susceptibility.
  • Elucidating lower exposure–response relationship.

These prototypes used chemicals with known outcomes, robust traditional data, and substantial systems biology understanding to determine if new data types could accurately predict known outcomes—essentially proof of concept for use of molecular biology data in risk assessment. In two of the three prototypes, we compared concomitantly collected traditional and new data types. We considered this an important verification step in order to provide us some confidence that new methods could be successfully applied in situations where data are limited. Additionally, we were interested in examples of how new data types could better inform unresolved uncertainties in chemical assessments based on robust traditional data.

We evaluated transcriptomic and epigenomic data (epidemiological and clinical) in the range of environmental exposures for three chemicals: a) benzene and other leukemogens (McHale et al. 2011, 2012; Smith et al. 2011; Thomas R et al. 2012, 2013, 2014); b) ozone (Duncan et al. 2012; U.S. EPA 2013a; Hatch et al. 2014; McCullough et al. 2014, 2016); and c) polycyclic aromatic hydrocarbons (PAHs), including tobacco smoke and benzo[a]pyrene (DHHS 2014; U.S. EPA 2013b; IARC 2010; Mattes et al. 2014). We also considered genomic, proteomic, and epigenomic data as available, and molecular animal and in vitro data for benzene and B[a]P (U.S. EPA 2013b; French et al. 2015). We evaluated exposures for benzene of < 0.1 to 10 parts per million (ppm) and ozone of 0.5 ppm for 2 hr. We used individual measures of the exposure–dose relationship for benzene and ozone (benzene urinary metabolites (Vermeulen et al. 2004) and heavy oxygen-labeled ozone (18O3) (Hatch et al. 2014). For PAH exposures, we used self-reported smoking. The PAH–tobacco smoke prototype focused on pathway mining of existing human microarray data from the ArrayExpress (http://www.ebi.ac.uk/) and Gene Expression Omnibus (http://www.ncbi.nlm.nih.gov/geo/). The toxicogenomics data were compared qualitatively and quantitatively to known health outcomes associated with these chemicals, specifically hematotoxicity and leukemia (benzene and other known leukemogens), lung inflammation and injury (ozone), and lung cancer (PAHs). The results of these data-rich comparisons therefore enabled us to draw on a wealth of chemical- and disease-specific data to help characterize associations among upstream molecular changes, downstream cellular events, and public health outcomes. Thus, the potential role of toxicogenomics in hazard identification and dose–response assessment was explored.

Limited-scope assessment prototypes. These prototypes explored approaches falling between molecular human clinical and epidemiology studies (described in “Major-scope assessment prototypes”) and in vitro, HT screening bioassays (described in “Screening and prioritization prototypes”) in terms of confidence in the data to characterize public health risks, resources expended to collect data, and the number of chemicals that can be evaluated in a given period. We considered three approaches to limited-scope assessment:

Short-duration in vivo exposure rodent studies that correlated transcriptomic alterations with cancer and noncancer outcomes as determined in traditional bioassays (Thomas RS et al. 2012a, 2013a, 2013c).

Advantages of the limited-scope approaches compared to HT in vitro approaches include intact metabolism and intact cell and tissue interactions and the potential to measure adverse health outcomes, including complex outcomes such as altered behavior and development.

Screening and prioritization prototypes. The two screening and prioritization prototypes are a) quantitative structure activity relationship (QSAR) models and use of analogous chemicals to expand available information (also called “read-across”) (Golbraikh et al. 2012; NAFTA Technical Working Group for Pesticides 2012; OECD 2016a; Politi et al. 2014; Wang et al. 2011, 2012a); and b) in vitro cell-based and biochemical-based (including enzymatic and ligand-binding) HT screening assays [focusing on evaluating thyroid hormone disruptors (Cox et al. 2014; Rotroff et al. 2013; Sipes et al. 2011a; Judson et al. 2010)]. Of note, although QSAR and in vitro assays are illustrated separately here, they often are used most effectively in combination. The U.S. EPA’s ToxCast™ program (Judson et al. 2010, 2011, 2012, 2013, 2014; Kavlock et al. 2012) and the multi-agency collaborative Tox21 program (Attene-Ramos et al. 2013, 2015; Freitas et al. 2014; Hsu et al. 2014; Huang et al. 2014; Tice et al. 2013) provide more information on these approaches. Virtual tissue modeling (DeWoskin et al. 2014; Knudsen and DeWoskin 2011; Knudsen et al. 2013, 2015) and toxicokinetic approaches (Wambaugh et al. 2015; Wetmore et al. 2012, 2013) also are discussed.

Examining human variability in responses. The data to evaluate variability and susceptibility are usually scant. We evaluated several data types to inform this issue:

Adverse outcome networks are conceptual mechanistic models that combine key events and adverse outcome pathways (AOP) into networks associated with specific diseases and disorders [see Zeise et al. (2013) and NRC (2011) for further details on examining human variability].

Results and Discussion

The NexGen prototypes help us to better understand and apply emerging science in a transparent and scientifically robust manner to environmental health risk assessment. Additionally, these prototypes help realize the National Research Council’s vision embodied in Toxicity Testing in the 21st Century: A Vision and a Strategy (NRC 2007; Krewski et al. 2011). Since this report was published, toxicity testing and risk assessment has continued shifting from the traditional, almost exclusive, use of animal data to using the new approaches the prototypes demonstrate (Adeleye et al. 2015; Abdo et al. 2015a, 2015b; Attene-Ramos et al. 2015; Bourdon-Lacombe et al. 2015; EC 2016, EC and JRC 2015; ECHA 2016a, 2016b; U.S. EPA 2015a, 2015b; Huang et al. 2016; JRC 2016; Mansouri et al. 2016; OECD 2016a, 2016b; Wambaugh et al. 2015). The new approaches consider a broader data array, foster mechanistic understanding of adverse effects, and move toward replacing uncertainty factors and extrapolations with data-derived probability distributions.

In each decision context category, new methods and data types were identified that could help inform assessment efforts. Methods illustrated in the screening and prioritization (Tier 1) and limited-scope (Tier 2) prototypes originally were designed for qualitative evaluation of chemicals. New and integrated approaches, however, are being developed to estimate relative potencies and more rapid quantitative toxicity values for use in certain decision contexts.

We used AOP and AON extensively to organize and interpret data for most of the prototypes and regard them as critical for linking molecular events to apical outcomes. The AOP–AON concept has gained considerable traction since it was first introduced (Ankley et al. 2010; Davis et al. 2015; Garcia-Reyero 2015; Geer et al. 2010; Tollefsen et al. 2014; Vinken 2013). We use the terms AOP and AONs throughput this article as they are commonly used by many U.S. and European agencies (OECD 2013).

Data quality and reporting are always critically important. Our data searches identified many published studies that we could not use because the data or the reporting was not sufficient for use in health risk assessment (e.g., does not meet minimum standards for study design or reporting) (U.S. EPA 2014; McConnell et al. 2014). This situation derives from the lag between establishing best practice criteria and full implementation in the research community, and inconsistent application of criteria for data quality and reporting (U.S. EPA 2014; McConnell et al. 2014).

Integrating the available data into a coherent analysis is also a challenge. Table S2 presents the evidence integration framework used for the prototypes. The framework focuses on evaluating and integrating evidence and drawing conclusions based on inferences drawn from new data types. To our knowledge this illustrative framework is the most complete illustration of using a new data type in a variety of assessment situations. More limited examples of evidence integration using new approaches include a) the International Agency for Research on Cancer’s determination of a likely causal link between benzene exposures and lymphoma based on molecular mechanisms data (IARC 2012); b) the U.S. EPA’s cumulative risk evaluation of relatively uncharacterized conazole fungicides based on molecular mechanisms data (U.S. EPA 2011d); c) the U.S. EPA’s use of toxicogenomic data in the Endocrine Disruptor Screening Program (EDSP) (Mansouri et al. 2016; U.S. EPA 2011c); d) OECD’s guidance on use of adverse outcome pathways in toxicity evaluations (OECD 2013); and e) OECDs guidance on the use of quantitative structure activity data to evaluate relative toxicity, and other activities on molecular screening and toxicogenomics (OECD 2016a, 2016b).

Major-Scope Assessment Prototypes (Tier 3)

We designed the Tier 3 prototypes to determine whether new data types could provide results comparable to robust traditional data. We also evaluated whether new data types could add to information robust traditional data sets provide. Support for this hypothesis and several sources of variability are given below (U.S. EPA 2013a, 2014; Esposito et al. 2014; Hatch et al. 2014; McCullough et al. 2014; McHale et al. 2011, 2012; Smith 2010; Smith et al. 2011; Thomas R et al. 2014). Highlights from the prototypes include:

  • AONs, once verified for accuracy, are useful in predicting specific hazards [e.g., benzene and other known leukemogens (hematotoxicity) (U.S. EPA 2014; IARC 2010; McHale et al. 2012; Smith 2010; Smith et al. 2011; Thomas R et al. 2012, 2014), ozone (lung inflammation and injury) (U.S. EPA 2013a, 2014; McCullough et al. 2014, 2016; Wu et al. 2015), and PAHs (lung cancer) (U.S. EPA 2013b, 2014; Mattes et al. 2014)].
  • Related chemical and nonchemical stressors (known to cause or exacerbate the same adverse health outcome) were shown to perturb various pathways within the same disease associated network, but do not always affect the same expressed genes or pathway (U.S. EPA 2014). Hence, overly simplistic descriptions of AOPs could miss the potential for network-level interactions. Evidence for a causal relationship between a specific AOP and adverse effects includes pharmacologic intervention to block identified pathway changes, use of knock-in and knock-out models, or identification of pathway polymorphisms and concomitant amelioration of severity or incidence of the specified adverse outcomes (U.S. EPA 2014; French et al. 2015; Hatzimichael and Crook 2013; Kasahara et al. 2015; McCullough et al. 2014; McHale et al. 2012; Smith 2010; Thomas R et al. 2014; Wu et al. 2015).
  • Less well-studied chemicals inducing the same AOP or AON could be of concern for concomitant health outcomes. Conversely, lack of an apparent mechanistic link to an adverse outcome might justify downgrading questionable in vivo data. Thus, network-level knowledge often is highly valuable to understand causal mechanisms, help integrate evidence, assess potential hazards of well-studied chemicals, provide a basis for cumulative assessment by grouping chemical and nonchemical stressors according to their common AOP network, and evaluate mechanisms underlying human susceptibility (e.g., genetic differences) (Bell et al. 2016; Carter et al. 2013; Ideker and Krogan 2012; Kleinstreuer et al. 2016; Schadt and Björkegren 2012; Zhang et al. 2014; Smith et al. 2016).
  • Biomarkers appropriately anchored to in vivo results can help elucidate exposure–dose–response relationships. Thomas R et al. (2014), extending the work of McHale et al. (2011), best illustrates use of molecular biomarkers to potentially predict public health risks. They reported dose-dependent effects of benzene exposure on gene expression and biochemical pathways, using transcriptome profiling of peripheral blood mononuclear cells, in people (< 1 ppm to > 10 ppm). Benzene exposures were estimated by urinary benzene levels. They estimated dose–response of gene expression in acute myeloid leukemia (AML) and related pathways. Responses at or below 0.1 ppm benzene were observed for altered expression of AML pathway genes and CYP2E1. Together, these data show that benzene alters disease-relevant pathways and genes in a dose-dependent manner. It should be noted that while benzene is considered a known hematotoxicant and leukemogen, the benzene exposed population from which the toxicogenomic biomarkers were characterized at this time only show hematotoxicity (U.S. EPA 2014). The leukemia lag time is such that additional follow-up will be required to demonstrate if the toxicogenomic signature is predictive of leukemia in the same individuals. Understanding the quantitative relationship of any biomarker to exposure and effect requires substantial study. A most promising application of biomarkers, however, is the ability to measure events of interest directly in environmentally exposed humans—an application revolutionizing epidemiology.
  • For benzene, ozone, and theoretically for PAHs, we demonstrated that multiple AOPs developed and progressed with increasing exposures (U.S. EPA 2014). With benzene, gene and pathway alterations associated with altered proliferation and differentiation, DNA-repair and immune function, among others, were discussed; impaired immune function was shown to occur at all exposure levels evaluated (from < 0.1 ppm to 10 ppm) (French et al. 2015; Thomas R et al. 2014). At higher concentrations, molecular pathways and effects characteristic of more severe toxicity (apoptosis and cell death) begin to emerge (French et al. 2015; Thomas R et al. 2014). Data collection over a range of concentrations thus remains essential when evaluating new data types. Additionally, limited time-course post-exposure data were available for ozone; various adverse outcomes involved in lung injury progressed after exposure, demonstrating the potential dynamic nature of underlying mechanisms (U.S. EPA 2013a; McCullough et al. 2014, 2016).
  • Chemical exposures resulting in adverse outcomes (e.g., benzene induced leukemia or ozone induced inflammation) appear to share AOP networks with pathologies of unknown origins (e.g., idiopathic or potentially naturally occurring disease) (U.S. EPA 2013a; Hatzimichael and Crook 2013; McCullough et al. 2014, 2016; McHale et al. 2012; Smith 2010; Smith et al. 2011; Thomas R et al. 2014; Wu et al. 2015). This suggests that chemically induced events might add to naturally occurring backgrounds of disease via shared mechanisms (U.S. EPA 2014). As NRC (2009) and Crump et al. (1976) discuss, this observation might have implications for an assumption of low-dose linearity for cancer and noncancer outcomes at the population level.
  • The prototypes helped characterize experimental and organismic factors influencing data interpretation, including experimental variability resulting from differing exposure concentrations, dosimetry, time courses, experimental techniques, experimental paradigms, cell and tissue types, individual genomic profiles, co-exposures, and lifestages (Ankley and Gray 2013; Bell and Edwards 2015; Cho et al. 2013; U.S. EPA 2014; French et al. 2015; Godderis et al. 2012; Hatch et al. 2014; McCullough et al. 2014; McHale et al. 2014; Mendrick 2011; Perkins et al. 2013; Smith 2010; Smith et al. 2011; Thomas R et al. 2014; Thomas RS et al. 2012b; Tice et al. 2013; Zeise et al. 2013). Identifying causal events without tight control of variability can be difficult even knowing the adverse outcome, reinforcing the importance for careful experimentation and interpretation when potential outcomes are unknown (U.S. EPA 2014).

Limited-Scope Assessment Prototypes (Tier 2)

We designed the Tier 2 prototypes to evaluate data from knowledge mining, alternative species bioassays, and short-term in vivo studies for identifying potential hazards, refining mechanistic understanding, and characterizing the relative potencies of thousands of chemicals more rapidly than possible with traditional methods. Confidence in these data generally ranks between Tier 3 and Tier 1 approaches. Highlights from the prototypes include:

  • These approaches are faster and less expensive than the molecular human epidemiology studies noted above and traditional chronic animal bioassays. Furthermore, unlike the QSAR models and HT screening data (discussed below), the data from in vivo studies are from intact organisms with metabolic function, normal architecture (for various cell and tissue types), and normal cell-cell, tissue-tissue interactions. The data also can be used to study more complex system-level adverse outcomes, such as developmental and neurobehavioral outcomes.
  • In the data-mining exercises, specific chemical exposures were associated with altered risks for diabetes or prediabetes (e.g., chlorinated organics, heavy metals, selected nutrients) (Bell and Edwards 2015; U.S. EPA 2014; Patel et al. 2012, 2013a, 2013b). We mined exposure data from NHANES human tissue biomonitored levels and NHANES clinically defined incidence. Additional risk factors—multiple chemical exposures and genetic and lifestyle susceptibility traits—were identified (Bell and Edwards 2015; U.S. EPA 2014; Patel et al. 2012, 2013a, 2013b). In one example, 59% of people with high levels of cadmium, lead, and arsenic also had markers for diabetes (U.S. EPA 2014). The data-mining results are generally most suitable for hypothesis generation because the output only identifies associations among events in very large data sets. The availability of biomonitoring data and clinical diagnoses in the same individuals, or understanding of mechanisms, however, is useful in an evidence analysis. Others also have provided traditional and computational data that report an association between chemical exposure and diabetes (Audouze et al. 2013; Dimas et al. 2014; Inadera 2013; Thayer et al. 2012).
  • Two Tier 2 prototypes demonstrated use of short-duration exposure bioassays in alternative species and mammalian species. We evaluated the results with traditional, molecular, and computational approaches. Collectively, these bioassays successfully identified exposure concentrations associated with transcriptomic changes, AOP–AON alterations and adverse outcomes (Padilla et al. 2012; Perkins et al. 2013; Thomas RS et al. 2011, 2012b, 2013c; Villeneuve et al. 2014). These prototypes provided data on complex mechanistic behaviors, effects of mixtures, and species-to-species similarities and differences, illustrating how these data could be used to evaluate potential hazards and chemical potencies (Ankley and Gray 2013; LaLone et al. 2014; Padilla et al. 2012; Painter et al. 2014; Perkins et al. 2013; Thomas RS et al. 2013b, 2013c).

Screening and Prioritization Prototypes (Tier 1)

For the first time, new approaches are being used that can evaluate vast numbers of chemicals relatively rapidly. For example, the tens of thousands of chemicals covered by the European Regulation on Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) legislation are being evaluated using QSAR and new types of bioassays (EC 2016; ECHA 2016a, 2016b; JRC 2016; OECD 2013, 2016a, 2016b). The U.S. Tox21 program is screening approximately 8,500 chemicals using innovative robotic technology and in vitro bioassays (Tice et al. 2013). Kavlock et al. (2012) note that “These tools can probe chemical-biological interactions at fundamental levels, focusing on the molecular and cellular pathways that are targets of chemical disruption.” The QSAR models (Goldsmith et al. 2012; Venkatapathy and Wang 2013; Wang et al. 2012a, 2012b) and HT in vitro bioassays were used to illustrate the rapid successful screening and prioritization of chemicals (Judson et al. 2013; Kavlock et al. 2012; Kleinstreuer et al. 2014; Rusyn et al. 2012; Sipes et al. 2013; Tice et al. 2013). Additional insights include:

Caveats Pertaining to Applying New Data Types in Risk Assessment

Exposure and adverse outcomes often can be associated with hundreds to thousands of gene changes, not all of which are causal (Mendrick 2011). Associative data can “suggest” a causal relationship between exposure and adverse health outcomes. Criteria to move from “suggestive” to “likely” causal include meta-analyses of multiple, independent studies yielding similar results; experimental evidence of causative relationships between key events in AOP networks and consequent adverse health outcomes; or combinations of consistent, coherent traditional and new data types. The prototypes demonstrated how different types of evidence in each decision support category might be characterized with respect to establishing causality and evidence integration (U.S. EPA 2014; NRC 2014). Additional caveats are described below. Many of these concerns apply to traditional, as well as new data types.

Cell type, tissue, individual, subpopulation, strain, species, and test system can affect how specific alterations in molecular events manifest as adverse outcomes or disease, even when the molecular signature is the same (Ankley and Gray 2013; Bell and Edwards 2015; Cho et al. 2013; U.S. EPA 2014; French et al. 2015; Godderis et al. 2012; Hatch et al. 2014; McCullough et al. 2014; McHale et al. 2014; Mendrick 2011; Perkins et al. 2013; Smith 2010; Smith et al. 2011; Thomas R et al. 2014; Thomas RS et al. 2012b; Tice et al. 2013; Zeise et al. 2013).

  • This phenomenon likely is due, at least in part, to dosimetry, epigenomic differences, and genomic plasticity, which assessments should consider whenever feasible.
  • For many chemicals, metabolism is critical to toxicity. That most HT in vitro test systems have limited or no metabolic competence should be considered. Although researchers are evaluating various approaches to add or enhance metabolic capability, satisfactory solutions that incorporate metabolism for routine screening of larger numbers of chemicals are not yet available. Consequently, although positive results are informative, negative results should not yet be interpreted as a lack of toxicity.
  • Molecular profiles can be both dose and time dependent (Knudsen et al. 2013, 2015; McCullough et al. 2014; Perkins et al. 2013; Thomas R et al. 2014; Thienpont et al. 2011). Predicting adverse outcomes based only on “snapshots” of biological events can therefore be challenging. Focusing on profiles associated with environmentally relevant exposures should improve predictions. Some signatures do appear stable over time, however, and might also serve as reliable indicators of chronic outcomes (Thomas RS et al. 2013c).
  • Gene expression data contain much uncertainty, as messenger RNA expression levels cannot be used to infer protein activity directly. Thus, these data alone could be suitable only for ranking and screening and used in assessments to complement other mechanistic data.
  • Our current ability to monitor multiple molecular processes (genomics, transcriptomics, proteomics, and epigenomics) in a single study is very limited, primarily due to cost. This hampers biological integration and limits our understanding of how chemicals influence complex biological systems.
  • A major challenge in using molecular data in risk assessment is how to use the data to improve predictions of adverse effects in humans. For example, how do changes in molecular events affect cells, changes in cells affect tissues and organs, and changes in organs affect the whole body? Researchers are collecting large amounts of HT/HC screening data on molecular-level effects, and the body of information on diseases and disease outcomes is substantial (http://www.ncbi.nlm.nih.gov/geo/EC 2016; EC and JRC 2015; Huang et al. 2016Tice et al. 2013). Very sparse chemical-specific data are available, however, on intermediate levels of organization and on the sequence of cellular-level disruption of normal biology to effects at higher organizational levels. Even so, tremendous strides are being made in generating disease-specific information.
  • Characterizing population response variability among individuals is a major challenge, given the many sources of inherent biological variability (e.g., genetic differences) and extrinsic influences (e.g., lifestyle, poverty, multiple chemical exposures). Each chemical exposure–health outcome pair involves combinations of these sources, and different decision contexts present distinct needs regarding the identification—and extent of characterization—of interindividual variability in the human population (see Figure 2). New approaches to examining variability in responses include a) computational modeling, in which variability in parameter values is simulated and differences among subpopulations are explored (Diaz Ochoa et al. 2013; Knudsen and DeWoskin 2011; Knudsen et al. 2015; Shah and Wambaugh 2010); b) HT in vitro data analysis of cell lines with different genetic backgrounds from the 1000 Genomes effort (Abdo et al. 2015a, 2015b; Eduati et al. 2015; Lock et al. 2012; O’Shea et al. 2011); c) human clinical and in vivo animal studies in genetically diverse individuals to identify genetic and epigenetic determinants of susceptibility (French et al. 2015; Harrill et al. 2009a, 2009b; McCullough et al. 2016); d) comprehensive scanning of gene coding regions in diverse individuals to examine the relationships among environmental exposures, interindividual sequence variation in human genes, and population disease risks (Mortensen and Euling 2013; NIEHS 2015); e) genome-wide association studies to uncover genomic loci that might contribute to risk of disease (NHGRI 2015; Wright et al. 2012); and f) association studies correlating phenotypic differences among diverse populations with expression patterns for groups of genes based on coexpression (Friend 2013; Patel et al. 2012, 2013a; Weiss et al. 2012). Additionally, understanding of the contribution of epigenomics to disease is the focus of much research (Ghantous et al. 2015).
  • Verifying toxicity-testing schemes and computational models that are more efficient is essential for using these new data and approaches for risk-based decisions. Central to this effort are a framework and criteria for determining whether the new data types are adequate for various types of decisions. While ultimately different methods and models based on their ability to predict human outcomes, they are also evaluated against their intended purpose. For example, high-throughput methods that can relatively rank thousands to tens of thousands of chemicals, with some certainty, based on their potential toxicity would be deemed extremely successful even though they may not be able to predict the specific health outcome anticipated in humans. Alternatively, methods and models relied upon to support regulation must contribute to the understanding of public health risks. The level of certainty needed in the data varies with their intended use because inaccurate results have increasing consequences and costs as decisions progress from screening, to further testing, to what safe chemical levels are, to what regulatory (or mitigation) actions should be taken (Crawford-Brown 2013). Traditional validation approaches that evaluate conventional assay and testing structures do not adequately address the potential uses of these new data and methods and would require years to implement (Judson et al. 2013). Thus, as the technology for rapid, efficient, robust hazard testing advances, the verification process also must advance to ensure confidence in their use. Clear and transparent articulation of these decision considerations are essential to the acceptance of, and support for, assessment results and in the overall evidence integration. Crawford-Brown (2013) discusses these issues relative to NexGen more extensively.

Figure 2 - Four line graphs of hypothetical data. A) Internal dose (y-axis) according to external dose (x-axis) and alternative PK parameters. B) Biological response (y-axis) according to internal dose (x-axis) and alternative PD parameters. C) Internal dose (y-axis) according to total external dose (x-axis). D) Biological response (y-axis) according to total internal dose (x-axis).Figure 2 – Effects of variability in (A) pharmacokinetics (PK), (B) pharmacodynamics (PD), (C) background and exposures, and (D) endogenous concentrations. In (A) and (B), individuals differ in PK or PD parameters. In (C) and (D), individuals have different initial baseline conditions (e.g., exposure to sources outside of the risk management decisions context; endogenously produced compounds) (Zeise et al. 2013). Reproduced with permission from Environmental Health Perspectives.

View larger image (TIF File)

Based on the lessons learned in the NexGen program and elsewhere, several new types of high- and medium-throughput assessments are being advanced (Casey et al. 2015; ECHA 2016b; U.S. EPA 2014, 2015b; Langley et al. 2015; Perkins et al. 2013; Settivari et al. 2015). Table 2 shows how characteristics of “fit-for-purpose” assessments could be tailored to support three illustrative decision-context categories. The table lists potential uses for NexGen assessments, data sources and types in different assessment categories, exposure paradigms used, incorporation of toxicokinetics, use of traditional data, hazard characterization, potency metrics, inferences drawn about the causal associations between exposures and adverse outcomes, the numbers of chemicals that can be assessed, and the time to conduct any given assessment.

Table 2 - Select View Table (HTML Version) for a 508-conformant versionTable 2 – Possible characteristics of fit-for-purpose assessments matched to illustrative decision-context categories.

View Table (HTML Version)
View larger image (TIF File)

Research Needs

Enhancing our understanding of complex chemical and biological interactions at various levels of biological organization requires integrating computational research with strategic laboratory studies to advance available models and accelerate application of new data in risk assessment. We suggest focusing on the following specific areas:

  • Developing reliable, molecular biomarkers and bioindicators that represent a wide variety of chemical exposures and key events of pathogenesis for building confidence in the characterization of key events used to construct an AOP.
  • Identifying and understanding AOP network interactions among different levels of organization for observed key events (genes, proteins, cells, tissues, organs, individuals, populations, and communities), including characterizing compensatory responses and their prognostic value for different adverse outcomes or disease states.
  • Collecting data and developing methods for a) reverse toxicokinetics to extrapolate concentrations used in cellular and cell-free systems to in vivo tissue doses and exposures, b) nonaqueous in vitro exposure methods for chemicals present as gases or as airborne particles, and c) adjusting for intra- and interspecies differences when assessing potential human effects based on nonhuman toxicity data.
  • Developing approaches for grouping chemical and nonchemical stressors based on common key events within AOPs to enable cumulative risk assessment and consideration of source apportionment with respect to exposures for cumulative risk assessment.
  • Evaluating individual human variability due to lifestage vulnerabilities, genetic differences, pre-existing disease and exposure, or adaptive and compensatory capabilities and developing techniques to incorporate this variability into population-level risk assessment.

Conclusions

A revolution in molecular, computational, and systems biology is rapidly advancing our understanding of what causes disease and who becomes affected, and the role of environmental factors on public health. This information is just beginning to result in innovative, more efficient approaches to toxicity testing and risk assessment. This article summarizes recent, multi-organizational efforts to understand and apply emerging science in a transparent and scientifically robust manner to environmental health risk assessment. We anticipate these novel methods will provide a more complete understanding of the biological underpinnings of health risks and, also, methods and data to help evaluate the tens of thousands of unaddressed chemicals in the nation (U.S. EPA 2015c). The overarching challenge to risk assessors is to obtain and interpret sufficient data for quick and efficient assessment to support decisions that protect public health and the environment. The ultimate goal is to develop safer chemicals and to better manage risks to public health and the environment. The prototypes demonstrate how new data can be used to help address these challenges.

The following list presents the ongoing efforts to advance toxicity testing and risk assessment:

  • Thousands of chemicals, previously having no or very limited traditional data, are being assessed based on similarities in physical–chemical structure to known toxicants (QSAR modeling) and on the results of rapid, robotically conducted in vitro bioassays. These evaluations will help prioritize testing, research, and assessment, and responding in emergency response situations.
  • Hundreds of chemicals are being evaluated by using computational analyses of large primary databases held in public repositories and by identifying the most important findings in the burgeoning literature. These efforts are playing a central role in developing knowledge about the potential toxicity of chemicals and the causes of disease. These approaches, in combination with high-throughput approaches, could be used to support limited scope assessments or to augment robust traditional data-based assessments.
  • Developing innovative, targeted testing approaches that combine short-duration in vivo bioassays and HT approaches will provide even more robust information for testing and assessment.
  • Finally, a variety of new methods are addressing the formidable challenges of characterizing cumulative effects from exposure to multiple chemical and nonchemical stressors, susceptible subpopulations, and low-dose responses, primarily based on improving mechanistic understanding of adverse health effects.

Near-term efforts include developing additional prototypes for public input and peer review and providing more opportunities to solicit stakeholder comments and participation. The U.S. EPA is developing a verification process for new methods and data types that focuses on integrating the evidence into various decision contexts for use by risk assessors and considers the external validity of different models in terms of human relevance (U.S. EPA 2014). The goal is to increase confidence for using these new approaches in risk assessment. Significant scientific gaps identified in the completed and ongoing prototypes are helping guide future research plans. An overview of issues being considered is provided by Crawford-Brown (2013).

We anticipate the prototype demonstrations will help overcome the significant logistical and methodological challenges in interpreting and using these new data and methods in risk assessment. For now, major chemical assessments will continue to be driven primarily by traditional data but with increasing augmentation with the new types of data. The U.S. EPA risk managers and the risk assessment community at large will continue to be informed of the new tools and methods being developed with an emphasis on high-quality, human-relevant science and transparency. Historically difficult risk assessment questions that this new and emerging knowledge are likely to inform include a) Why do individual and specific populations respond differently to environmental exposures? b) How are children at greater risk for certain exposures and effects? c) What happens when people are exposed to mixtures that contain very low levels of individual chemicals, such as those commonly found in the environment? d) How do other environmental factors like preexisting health conditions alter the response to chemical exposures? These are just some of the issues that NexGen assessments will help address to improve the identification of safer chemicals and reduce risk from exposures to hazardous chemicals in the environment. A more detailed report is available (U.S. EPA 2014).


References

Abdo N, Wetmore BA, Chappell GA, Shea D, Wright FA, Rusyn I. 2015a. In vitro screening for population variability in toxicity of pesticide-containing mixtures. Environ Int 85:147–155.

Abdo N, Xia M, Brown CC, Kosyk O, Huang R, Sakamuru S, et al. 2015b. Population-based in vitro hazard and concentration–response assessment of chemicals: the 1000 Genomes high throughput screening study. Environ Health Perspect 123:458–466, doi: 10.1289/ehp.1408775.

Adeleye Y, Andersen M, Clewell R, Davies M, Dent M, Edwards S, et al. 2015. Implementing Toxicity Testing in the 21st Century (TT21C): making safety decisions using toxicity pathways, and progress in a prototype risk assessment. Toxicology 332:102–111.

Ankley GT, Bennett RS, Erickson RJ, Hoff DJ, Hornung MW, Johnson RD, et al. 2010. Adverse outcome pathways: a conceptual framework to support ecotoxicology research and risk assessment. Environ Toxicol Chem 29:730–741.

Ankley GT, Gray LE. 2013. Cross-species conservation of endocrine pathways: a critical analysis of Tier 1 fish and rat screening assays with 12 model chemicals. Environ Toxicol Chem 32:1084–1087.

Attene-Ramos MS, Huang R, Michael S, Witt KL, Richard A, Tice RR, et al. 2015. Profiling of the Tox21 chemical collection for mitochondrial function to identify compounds that acutely decrease mitochondrial membrane potential. Environ Health Perspect 123:49–56, doi: 10.1289/ehp.1408642.

Attene-Ramos MS, Miller N, Huang R, Michael S, Itkin M, Kavlock RJ, et al. 2013. The Tox21 robotic platform for the assessment of environmental chemicals – from vision to reality. Drug Discov Today 18:716–723.

Audouze K, Brunak S, Grandjean P. 2013. A computational approach to chemical etiologies of diabetes. Sci Rep 3:2712, doi: 10.1038/srep02712.

Bell SM, Angrish MM, Wood CE, Edwards SW. 2016. Integrating publicly available data to generate computationally predicted adverse outcome pathways for fatty liver. Toxicol Sci 150:510–520.

Bell SM, Edwards SW. 2015. Identification and prioritization of relationships between environmental stressors and adverse human health impacts. Environ Health Perspect 123:1193–1199, doi: 10.1289/ehp.1409138.

Bourdon-Lacombe JA, Moffat ID, Deveau M, Husain M, Auerbach S, Krewski D, et al. 2015. Technical guide for applications of gene expression profiling in human health risk assessment of environmental chemicals. Regul Toxicol Pharmacol 72(2):292–309.

Carter H, Hofree M, Ideker T. 2013. Genotype to phenotype via network analysis. Curr Opin Genet Dev 23:611–621.

Casey W, Jacobs A, Maull E, Matheson J, Clarke C, Lowit A. 2015. A new path forward: the Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM) and National Toxicology Program’s Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM). J Am Assoc Lab Anim Sci 54(2):170–173.

Cho HY, Gladwell W, Yamamoto M, Kleeberger SR. 2013. Exacerbated airway toxicity of environmental oxidant ozone in mice deficient in Nrf2. Oxid Med Cell Longev 2013: 254069, doi: 10.1155/2013/254069.

Cote I, Anastas PT, Birnbaum LS, Clark RM, Dix DJ, Edwards SW, et al. 2012. Advancing the next generation of health risk assessment. Environ Health Perspect 120:1499–1502, doi: 10.1289/ehp.1104870.

Cox LA, Popken D, Marty MS, Rowlands JC, Patlewicz G, Goyak KO, et al. 2014. Developing scientific confidence in HTS-derived prediction models: lessons learned from an endocrine case study. Regul Toxicol Pharmacol 69:443–450.

Crawford-Brown D. 2013. The role of advanced biological data in the rationality of risk-based regulatory decisions. J Environ Prot (Irvine, Calif) 4:238–249.

Crump KS, Hoel DG, Langley CH, Peto R. 1976. Fundamental carcinogenic processes and their implications for low dose risk assessment. Cancer Res 36(9 pt 1):2973–2979.

Davis AP, Grondin CJ, Lennon-Hopkins K, Saraceni-Richards C, Sciaky D, King BL, et al. 2015. The Comparative Toxicogenomics Database’s 10th year anniversary: update 2015. Nucleic Acids Res 43(Database issue):D914–D920.

DeWoskin RS, Knudsen TB, Shah I. 2014. Virtual models (vM). In: Encyclopedia of Toxicology, Vol. 4 (Wexler P, ed). 3rd ed. London, UK:Elsevier, Inc., 948–957.

DHHS (U.S. Department of Health and Human Services). 2014. The Health Consequences of Smoking—50 years of Progress. Atlanta, GA:DHHS, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health. http://www.surgeongeneral.gov/library/reports/50-years-of-progress/exec-summary.pdf [accessed 7 March 2016].

Diaz Ochoa JG, Bucher J, Péry AR, Zaldivar Comenges JM, Niklas J, Mauch K. 2013. A multi-scale modeling framework for individualized, spatiotemporal prediction of drug effects and toxicological risk. Front Pharmacol 3:204, doi: 10.3389/fphar.2012.00204.

Dimas AS, Lagou V, Barker A, Knowles JW, Mägi R, Hivert MF, et al. 2014. Impact of type 2 diabetes susceptibility variants on quantitative glycemic traits reveals mechanistic heterogeneity. Diabetes 63:2158–2171.

Duncan KE, Dailey LA, Carson JL, Hernandez ML, Peden DB, Devlin R. 2012. Cultured basal airway epithelial cells from asthmatics display baseline gene expression profiles that differ from normal healthy cells and exhibit differential responses to ambient air pollution particles. Am J Respir Crit Care Med 185:A4291, doi: 10.1164/ajrccm-conference.2012.185.1_MeetingAbstracts.A4291.

EC (European Commission). 2016. SEURAT-1: Towards the Replacement of in Vivo Repeated Dose Systemic Toxicity Testing. http://www.seurat-1.eu [accessed 7 March 2016].

EC, JRC (European Commission, Joint Research Centre). 2015. EURL ECVAM Status Report on the Development, Validation and Regulatory Acceptance of Alternative Methods and Approaches (2015). http://bookshop.europa.eu/en/eurl-ecvam-status-report-on-the-development-validation-and-regulatory-acceptance-of-alternative-methods-and-approaches-2015–pbLBNA27474/ [accessed 7 March 2016].

ECHA. (European Chemical Agency). 2016a. ECHA News: The QSAR Toolbox. https://echa.eurpoa.eu/support/oecd-qsar-toolbox [accessed 7 September 2016].

ECHA. 2016b. OECD and EU Test Guidelines. http://echa.europa.eu/support/oecd-eu-test-guidelines [accessed 10 March 2016].

Eduati F, Mangravite LM, Wang T, Tang H, Bare JC, Huang R, et al. 2015. Prediction of human population responses to toxic compounds by a collaborative competition. Nat Biotechnol 33(9):933–940.

Esposito S, Tenconi R, Lelii M, Preti V, Nazzari E, Consolo S, et al. 2014. Possible molecular mechanisms linking air pollution and asthma in children. BMC Pulm Med 14:31, doi: 10.1186/1471-2466-14-31.

Freitas J, Miller N, Mengeling BJ, Xia M, Huang R, Houck K, et al. 2014. Identification of thyroid hormone receptor active compounds using a quantitative high-throughput screening platform. Curr Chem Genomics Transl Med 8:36–46.

French JE, Gatti DM, Morgan DL, Kissling GE, Shockley KR, Knudsen GA, et al. 2015. Diversity outbred mice identify population-based exposure thresholds and genetic factors that influence benzene-induced genotoxicity. Environ Health Perspect 123:237–245, doi: 10.1289/ehp.1408202.

Friend S. 2013. Scientific opportunities from heterogeneous biological data analysis: overcoming complexity. Presentation. National Academy of Sciences Meeting: Integrating Environmental Health Data to Advance Discovery, 10–11 January 2013, Washington, DC. https://www.youtube.com/watch?v=qNcATDilfVo&index=3&list=PLzsdEyVNFvgyizsegxlcIbLz1glyOIHxJ [accessed 10 March 2016].

Garcia-Reyero N. 2015. Are adverse outcome pathways here to stay? Environ Sci Technol 49:3–9.

Geer LY, Marchler-Bauer A, Geer RC, Han L, He J, He S, et al. 2010. The NCBI BioSystems database. Nucleic Acids Res 38(Database issue):D492–D496.

Ghantous A, Hernandez-Vargas H, Byrnes G, Dwyer T, Herceg Z. 2015. Characterising the epigenome as a key component of the fetal exposome in evaluating in utero exposures and childhood cancer risk. Mutagenesis 30:733–742.

Godderis L, Thomas R, Hubbard AE, Tabish AM, Hoet P, Zhang L, et al. 2012. Effect of chemical mutagens and carcinogens on gene expression profiles in human TK6 cells. PLoS One 7(6):e39205, doi: 10.1371/journal.pone.0039205.

Golbraikh A, Wang XS, Zhu H, Tropsha A. 2012. Predictive QSAR modeling: methods and applications in drug discovery and chemical risk assessment. In: Handbook of Computational Chemistry (Leszczynski J, ed). New York, NY:Springer, 1309–1342.

Goldsmith MR, Peterson SD, Chang DT, Transue TR, Tornero-Velez R, Tan YM, et al. 2012. Informing mechanistic toxicology with computational molecular models. Methods Mol Biol 929:139–165.

Harrill AH, Ross PK, Gatti DM, Threadgill DW, Rusyn I. 2009a. Population-based discovery of toxicogenomics biomarkers for hepatotoxicity using a laboratory strain diversity panel. Toxicol Sci 110(1):235–243.

Harrill AH, Watkins PB, Su S, Ross PK, Harbourt DE, Stylianou IM, et al. 2009b. Mouse population-guided resequencing reveals that variants in CD44 contribute to acetaminophen-induced liver injury in humans. Genome Res 19(9):1507–1515.

Hatch GE, Duncan KE, Diaz-Sanchez D, Schmitt MT, Ghio AJ, Carraway MS, et al. 2014. Progress in assessing air pollutant risks from in vitro exposures: matching ozone dose and effect in human airway cells. Toxicol Sci 141:198–205.

Hatzimichael E, Crook T. 2013. Cancer epigenetics: new therapies and new challenges. J Drug Deliv 2013:529312, doi: 10.1155/2013/529312.

Hood L, Tian Q. 2012. Systems approaches to biology and disease enable translational systems medicine. Genomics Proteomics Bioinformatics 10:181–185.

Hsu CW, Zhao J, Huang R, Hsieh JH, Hamm J, Chang X, et al. 2014. Quantitative high-throughput profiling of environmental chemicals and drugs that modulate farnesoid X receptor. Sci Rep 4:6437, doi: 10.1038/srep06437.

Huang R, Sakamuru S, Martin MT, Reif DM, Judson RS, Houck KA, et al. 2014. Profiling of the Tox21 10K compound library for agonists and antagonists of the estrogen receptor alpha signaling pathway. Sci Rep 4:5664, doi: 10.1038/srep05664.

Huang R, Xia M, Sakamuru S, Zhao J, Shahane SA, Attene-Ramos M, et al. 2016. Modelling the Tox21 10 K chemical profiles for in vivo toxicity prediction and mechanism characterization. Nat Commun 7:10425, doi: 10.1038/ncomms10425.

Hubal EA. 2009. Biologically relevant exposure science for 21st century toxicity testing. Toxicol Sci 111:226–232.

IARC (International Agency for Research on Cancer). 2010. Some non-heterocyclic polycyclic aromatic hydrocarbons and some related exposures. IARC Monogr Eval Carcinog Risk Hum 92.

IARC. 2012. Benzene. IARC Monogr Eval Carcinog Risk Hum 100F:249–294.

Ideker T, Krogan NJ. 2012. Differential network biology. Mol Syst Biol 8:565, doi: 10.1038/msb.2011.99.

Inadera H. 2013. Developmental origins of obesity and type 2 diabetes: molecular aspects and role of chemicals. Environ Health Prev Med 18:185–197.

JRC (Joint Research Centre). 2016. Alternatives to Animal Testing and Safety Assessment of Chemicals. https://ec.europa.eu/jrc/en/research-topic/alternatives-animal-testing-and-safety-assessment-chemicals [accessed 2 September 2016].

Judson R, Houck K, Martin M, Knudsen TB, Thomas RS, Sipes N, et al. 2014. In vitro and modelling approaches to risk assessment from the U.S. Environmental Protection Agency ToxCast programme. Basic Clin Pharmacol Toxicol 115:69–76.

Judson R, Kavlock R, Martin M, Reif D, Houck K, Knudsen T, et al. 2013. Perspectives on validation of high-throughput assays supporting 21st century toxicity testing. ALTEX 30:51–66.

Judson RS, Kavlock RJ, Setzer RW, Hubal EA, Martin MT, Knudsen TB, et al. 2011. Estimating toxicity-related biological pathway altering doses for high-throughput chemical risk assessment. Chem Res Toxicol 24:451–462.

Judson RS, Magpantay FM, Chickarmane V, Haskell C, Tania N, Taylor J, et al. 2015. Integrated model of chemical perturbations of a biological pathway using 18 in vitro high-throughput screening assays for the estrogen receptor. Toxicol Sci 148(1):137–154.

Judson RS, Martin MT, Egeghy P, Gangwal S, Reif DM, Kothiya P, et al. 2012. Aggregating data for computational toxicology applications: the U.S. Environmental Protection Agency (EPA) Aggregated Computational Toxicology Resource (ACToR) system. Int J Mol Sci 13:1805–1831.

Judson RS, Martin MT, Reif DM, Houck KA, Knudsen TB, Rotroff DM, et al. 2010. Analysis of eight oil spill dispersants using rapid, in vitro tests for endocrine and other biological activity. Environ Sci Technol 44(15):5979–5985.

Kasahara DI, Mathews JA, Park CY, Cho Y, Hunt G, Wurmbrand AP, et al. 2015. ROCK insufficiency attenuates ozone-induced airway hyperresponsiveness in mice. Am J Physiol Lung Cell Mol Physiol 309(7):L736–L746.

Kavlock R, Chandler K, Houck K, Hunter S, Judson R, Kleinstreuer N, et al. 2012. Update on EPA’s ToxCast program: providing high throughput decision support tools for chemical risk management. Chem Res Toxicol 25:1287–1302.

Kleinstreuer NC, Sullivan K, Allen D, Edwards S, Mendrick DL, Embry M, et al. 2016. Adverse outcome pathways: from research to regulation scientific workshop report. Regul Toxicol Pharmacol 76:39–50.

Kleinstreuer NC, Yang J, Berg EL, Knudsen TB, Richard AM, Martin MT, et al. 2014. Phenotypic screening of the ToxCast chemical library to classify toxic and therapeutic mechanisms. Nat Biotechnol 32:583–591.

Knudsen TB, DeWoskin RS. 2011. Systems modeling in developmental toxicity. In: Handbook of Systems Toxicology (Casciano DA, Sahu SC, eds). Chichester, UK:John Wiley & Sons, Ltd., 599–617.

Knudsen TB, Keller DA, Sander M, Carney EW, Doerrer NG, Eaton DL, et al. 2015. FutureTox II: in vitro data and in silico models for predictive toxicology. Toxicol Sci 143(2):256–267.

Knudsen TB, Martin M, Chandler K, Kleinstreuer N, Judson R, Sipes N. 2013. Predictive models and computational toxicology. Methods Mol Biol 947:343–374.

Krewski D, Westphal M, Al-Zoughool M, Croteau MC, Andersen ME. 2011. New directions in toxicity testing. Annu Rev Public Health 32:161–178.

Krewski D, Westphal M, Andersen ME, Paoli GM, Chiu WA, Al-Zoughool M, et al. 2014. A framework for the next generation of risk science. Environ Health Perspect 122:796–805, doi: 10.1289/ehp.1307260.

LaLone CA, Berninger JP, Villeneuve DL, Ankley GT. 2014. Leveraging existing data for prioritization of the ecological risks of human and veterinary pharmaceuticals to aquatic organisms. Philos Trans R Soc Lond B Biol Sci 369:20140022, doi: 10.1098/rstb.2014.0022.

Langley G, Austin CP, Balapure AK, Birnbaum LS, Bucher JR, Fentem J, et al. 2015. Lessons from toxicology: developing a 21st-century paradigm for medical research. Environ Health Perspect 123:A268–A272, doi: 10.1289/ehp.1510345.

Lock EF, Abdo N, Huang R, Xia M, Kosyk O, O’Shea SH, et al. 2012. Quantitative high-throughput screening for chemical toxicity in a population-based in vitro model. Toxicol Sci 126:578–588.

Mansouri K, Abdelaziz A, Rybacka A, Roncaglioni A, Tropsha A, Varnek A, et al. 2016. CERAPP: collaborative estrogen receptor activity prediction project. Environ Health Perspect 124(7):1023–1033, doi: 10.1289/ehp.1510267.

Mattes W, Yang X, Orr MS, Richter P, Mendrick DL. 2014. Biomarkers of tobacco smoke exposure. Adv Clin Chem 67:1–45.

McConnell ER, Bell SM, Cote I, Wang RL, Perkins EJ, Garcia-Reyero N, et al. 2014. Systematic Omics Analysis Review (SOAR) tool to support risk assessment. PloS One 9:e110379, doi: 10.1371/journal.pone.0110379.

McCullough SD, Bowers EC, On DM, Morgan DS, Dailey LA, Hines RN, et al. 2016. Baseline chromatin modification levels may predict interindividual variability in ozone-induced gene expression. Toxicol Sci 150(1):216–224.

McCullough SD, Duncan KE, Swanton SM, Dailey LA, Diaz-Sanchez D, Devlin RB. 2014. Ozone induces a pro-inflammatory response in primary human bronchial epithelial cells through mitogen-activated protein kinase activation without nuclear factor-κB activation. Am J Respir Cell Mol Biol 51:426–435.

McHale CM, Smith MT, Zhang L. 2014. Application of toxicogenomic profiling to evaluate effects of benzene and formaldehyde: from yeast to human. Ann N Y Acad Sci 1310:74–83.

McHale CM, Zhang L, Lan Q, Vermeulen R, Li G, Hubbard AE, et al. 2011. Global gene expression profiling of a population exposed to a range of benzene levels. Environ Health Perspect 119:628–634, doi: 10.1289/ehp.1002546.

McHale CM, Zhang L, Smith MT. 2012. Current understanding of the mechanism of benzene-induced leukemia in humans: implications for risk assessment. Carcinogenesis 33:240–252.

Mendrick DL. 2011. Transcriptional profiling to identify biomarkers of disease and drug response. Pharmacogenomics 12:235–249.

Mortensen HM, Euling SY. 2013. Integrating mechanistic and polymorphism data to characterize human genetic susceptibility for environmental chemical risk assessment in the 21st century. Toxicol Appl Pharmacol 271:395–404.

NAFTA (North American Free Trade Agreement, Technical Working Group on Pesticides). 2012. (Q)uantitative Structure Activity Relationship [(Q)SAR] Guidance Document. https://www.epa.gov/sites/production/files/2016-01/documents/qsarguidance.pdf [accessed 21 September 2016].

NHGRI (National Human Genome Research Institute). 2015. A Catalog of Published Genome-Wide Association Studies. http://www.genome.gov/ gwastudies/ [accessed 7 March 2016].

NIEHS (National Institute of Environmental Health Sciences). 2015. NIEHS Single Nucleotide Polymorphisms (SNPs) Environmental Genome Project. http://www.ncbi.nlm.nih.gov/guide/howto/ view-all-snps/ [accessed 2 September 2016].

Nishihara K, Huang R, Zhao J, Shahane SA, Witt KL, Smith-Roe SL, et al. 2016. Identification of genotoxic compounds using isogenic DNA repair deficient DT40 cell lines on a quantitative high throughput screening platform. Mutagenesis 31:69–81.

NRC (National Research Council). 2007. Toxicity Testing in the 21st Century: A Vision and a Strategy. Washington, DC:National Academies Press.

NRC. 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC:National Academies Press.

NRC. 2011. Predicting Later-Life Outcomes of Early- Life Exposures [news story]. Standing Committee on Use of Emerging Science for Environmental Health Decisions Newsletter. Emerging Science for Environmental Health Decisions, Washington, DC; February 2011. http://nas-sites. org/emergingscience/files/2011/05/inutero_final_ April2011.pdf [accessed 7 March 2016].

NRC. 2014. Review of EPA’s Integrated Risk Information System (IRIS) Process. Washington, DC:National Academies Press.

OECD (Organization for Economic Cooperation and Development). 2013. Guidance Document on Developing and Assessing Adverse Outcome Pathways. Series on Testing and Assessment, No. 184. ENV/JM/MONO(2013)6. Available: http://search.oecd.org/officialdocuments/displaydocumentpdf/?cote=env/jm/mono%282013%296&doclanguage=en [accessed 7 March 2016].

OECD. 2016a. OECD QSAR Toolbox. Version 3.0. http://www.oecd.org/chemicalsafety/risk-assessment/ theoecdqsartoolbox.htm [accessed 2 September 2016].

OECD. 2016b. Other Activities on Molecular Screening and Toxicogenomics. http://www.oecd.org/env/ ehs/testing/toxicogenomics.htm [accessed 7 March 2016].

O’Shea SH, Schwarz J, Kosyk O, Ross PK, Ha MJ, Wright FA, et al. 2011. In vitro screening for population variability in chemical toxicity. Toxicol Sci 119:398–407.

Padilla S, Corum D, Padnos B, Hunter DL, Beam A, Houck KA, et al. 2012. Zebrafish developmental screening of the ToxCast™ Phase I chemical library. Reprod Toxicol 33:174–187.

Painter K, McConnell ER, Sahasrabudhe S, Burgoon L, Powers CM. 2014. What do the data show? Knowledge map development for comprehensive environmental assessment. Integr Environ Assess Manag 10:37–47.

Patel CJ, Chen R, Butte AJ. 2012. Data-driven integration of epidemiological and toxicological data to select candidate interacting genes and environmental factors in association with disease. Bioinformatics 28:i121–i126.

Patel CJ, Chen R, Kodama K, Loannidis JP, Butte AJ. 2013a. Systematic identification of interaction effects between genome- and environment-wide associations in type 2 diabetes mellitus. Hum Genet 132:495–508.

Patel CJ, Sivadas A, Tabassum R, Preeprem T, Zhao J, Arafat D, et al. 2013b. Whole genome sequencing in support of wellness and health maintenance. Genome Med 5:58, doi: 10.1186/gm462.

Perkins EJ, Ankley GT, Crofton KM, Garcia-Reyero N, LaLone CA, Johnson MS, et al. 2013. Current perspectives on the use of alternative species in human health and ecological risk assessments. Environ Health Perspect 121:1002–1010, doi: 10.1289/ehp.1306638.

Politi R, Rusyn I, Tropsha A. 2014. Prediction of binding affinity and efficacy of thyroid hormone receptor ligands using QSAR and structure-based modeling methods. Toxicol Appl Pharmacol 280:177–189.

Rotroff DM, Dix DJ, Houck KA, Kavlock RJ, Knudsen TB, Martin MT, et al. 2013. Real-time growth kinetics measuring hormone mimicry for ToxCast chemicals in T-47D human ductal carcinoma cells. Chem Res Toxicol 26:1097–1107.

Rotroff DM, Wetmore BA, Dix DJ, Ferguson SS, Clewell HJ, Houck KA, et al. 2010. Incorporating human dosimetry and exposure into high-throughput in vitro toxicity screening. Toxicol Sci 117:348–358.

Rusyn I, Sedykh A, Low Y, Guyton KZ, Tropsha A. 2012. Predictive modeling of chemical hazard by integrating numerical descriptors of chemical structures and short-term toxicity assay data. Toxicol Sci 127:1–9.

Ryan KR, Sirenko O, Parham F, Hsieh JH, Cromwell EF, Tice RR, et al. 2016. Neurite outgrowth in human induced pluripotent stem cell-derived neurons as a high-throughput screen for developmental neurotoxicity or neurotoxicity. Neurotoxicology 53:271–281.

Schadt EE, Björkegren JL. 2012. NEW: network-enabled wisdom in biology, medicine, and health care. Sci Transl Med 4(115):115rv1, doi: 10.1126/scitranslmed.3002132.

Settivari RS, Ball N, Murphy L, Rasoulpour R, Boverhof DR, Carney EW. 2015. Predicting the future: opportunities and challenges for the chemical industry to apply 21st-century toxicity testing. J Am Assoc Lab Anim Sci 54(2):214–223.

Shah I, Wambaugh J. 2010. Virtual tissues in toxicology. J Toxicol Environ Health B Crit Rev 13:314–328.

Sipes NS, Martin MT, Kothiya P, Reif DM, Judson RS, Richard AM, et al. 2013. Profiling 976 ToxCast chemicals across 331 enzymatic and receptor signaling assays. Chem Res Toxicol 26:878–895.

Sipes NS, Martin MT, Reif DM, Kleinstreuer NC, Judson RS, Singh AV, et al. 2011a. Predictive models of prenatal developmental toxicity from ToxCast high-throughput screening data. Toxicol Sci 124:109–127.

Sipes NS, Padilla S, Knudsen TB. 2011b. Zebrafish: as an integrative model for twenty-first century toxicity testing. Birth Defects Res C Embryo Today 93:256–267.

Smith MT. 2010. Advances in understanding benzene health effects and susceptibility. Annu Rev Public Health 31:133–148.

Smith MT, Guyton KZ, Gibbons CF, Fritz JM, Portier CJ, Rusyn I, et al. 2016. Key characteristics of carcinogens as a basis for organizing data on mechanisms of carcinogenesis. Environ Health Perspect 124:713–721, doi: 10.1289/ehp.1509912.

Smith MT, Zhang L, McHale CM, Skibola CF, Rappaport SM. 2011. Benzene, the exposome and future investigations of leukemia etiology. Chem Biol Interact 192:155–159.

Thayer KA, Heindel JJ, Bucher JR, Gallo MA. 2012. Role of environmental chemicals in diabetes and obesity: a National Toxicology Program workshop review. Environ Health Perspect 120:779–789, doi: 10.1289/ehp.1104597.

Thienpont B, Tingaud-Sequeira A, Prats E, Barata C, Babin PJ, Raldúa D. 2011. Zebrafish eleutheroembryos provide a suitable vertebrate model for screening chemicals that impair thyroid hormone synthesis. Environ Sci Technol 45:7525–7532.

Thomas R, Hubbard AE, McHale CM, Zhang L, Rappaport SM, Lan Q, et al. 2014. Characterization of changes in gene expression and biochemical pathways at low levels of benzene exposure. PloS One 9:e91828, doi: 10.1371/journal.pone.0091828.

Thomas R, McHale CM, Lan Q, Hubbard AE, Zhang L, Vermeulen R, et al. 2013. Global gene expression response of a population exposed to benzene: a pilot study exploring the use of RNA-sequencing technology. Environ Mol Mutagen 54:566–573.

Thomas R, Phuong J, McHale CM, Zhang L. 2012. Using bioinformatic approaches to identify pathways targeted by human leukemogens. Int J Environ Res Public Health 9:2479–2503.

Thomas RS, Black MB, Li L, Healy E, Chu TM, Bao W, et al. 2012a. A comprehensive statistical analysis of predicting in vivo hazard using high-throughput in vitro screening. Toxicol Sci 128(2):398–417.

Thomas RS, Clewell HJ III, Allen BC, Wesselkamper SC, Wang NC, Lambert JC, et al. 2011. Application of transcriptional benchmark dose values in quantitative cancer and noncancer risk assessment. Toxicol Sci 120:194–205.

Thomas RS, Clewell HJ III, Allen BC, Yang L, Healy E, Andersen ME. 2012b. Integrating pathway-based transcriptomic data into quantitative chemical risk assessment: a five chemical case study. Mutat Res 746:135–143.

Thomas RS, Himmelstein MW, Clewell HJ III, Yang Y, Healy E, Black MB, et al. 2013a. Cross-species transcriptomic analysis of mouse and rat lung exposed to chloroprene. Toxicol Sci 131:629–640.

Thomas RS, Philbert MA, Auerbach SS, Wetmore BA, DeVito MJ, Cote I, et al. 2013b. Incorporating new technologies into toxicity testing and risk assessment: moving from 21st century vision to a data-driven framework. Toxicol Sci 136:4–18.

Thomas RS, Wesselkamper SC, Wang NC, Zhao QJ, Petersen DD, Lambert JC, et al. 2013c. Temporal concordance between apical and transcriptional points of departure for chemical risk assessment. Toxicol Sci 134:180–194.

Tice RR, Austin CP, Kavlock RJ, Bucher JR. 2013. Improving the human hazard characterization of chemicals: a Tox21 update. Environ Health Perspect 121:756–765, doi: 10.1289/ehp.1205784.

Tollefsen KE, Scholz S, Cronin MT, Edwards SW, de Knecht J, Crofton K, et al. 2014. Applying Adverse Outcome Pathways (AOPs) to support Integrated Approaches to Testing and Assessment (IATA). Regul Toxicol Pharmacol 70:629–640.

U.S. EPA (U.S. Environmental Protection Agency). 2011a. Advancing the Next Generation (NexGen) of Risk Assessment: The Prototypes Workshop. 1–3 November 2010, Research Triangle Park, North Carolina. EPA/600/R-11/100. Washington, DC:U.S. EPA.

U.S. EPA. 2011b. Advancing the Next Generation (NexGen) of Risk Assessment: Public Dialogue Conference. Summary Report. EPA/600/R-11/101. Washington, DC:U.S. EPA. https://archive.epa.gov/risk/nexgen/web/pdf/nexgen-public-conf-summary.pdf [accessed 7 March 2016].

U.S. EPA. 2011c. Endocrine Disrupter Sceening Program for the 21st Century: (EDSP21 Work Plan). The Incorporation of In Silico Models and In vitro High Throughput Assays in the Endocrine Disruptor Screening Program (EDSP) for Prioritization and Screening. Summary Overview. Washington, DC:U.S. EPA. http://www.epa.gov/sites/production/files/2015-07/documents/edsp21_work_plan_summary_overview_final.pdf [accessed 7 March 2016].

U.S. EPA. 2011d. Use of “Omic” Technology to Inform the Risk Assessment Support Document for Case Study: Propiconazole. Appendix A & B. Washington, DC:U.S. EPA. https://www.regulations.gov/contentStreamer?documentId=EPA-HQ-OPP-2011-0284-0003&disposition=attachment&contentType=pdf [accessed 2 September 2016].

U.S. EPA. 2013a. Final Report: Integrated Science Assessment of Ozone and Related Photochemical Oxidants. EPA/600/R-10/076. Washington, DC:U.S. EPA. http://cfpub.epa.gov/ncea/isa/recordisplay.cfm?deid=247492 [accessed 7 March 2016].

U.S. EPA. 2013b. IRIS Toxicological Review of Benzo[a]pyrene (Public Comment Draft). Washington, DC:U.S. EPA. http://cfpub.epa.gov/ncea/iris_drafts/recordisplay.cfm?deid=66193 [accessed 7 March 2016].

U.S. EPA. 2014. Next Generation Risk Assessment: Incorporation of Recent Advances in Molecular, Computational, and Systems Biology (Final Report). EPA/600/R-14/004. Washington, DC:U.S. EPA. http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=286690 [accessed 7 March 2016].

U.S. EPA. 2015a. Chemical Safety for Sustainability: Strategic Research Action Plan 2016-2019. EPA 601/K-15/003. https://www.epa.gov/research/chemical-safety-sustainability-strategic-research-action-plan-2016-2019 [accessed 2 September 2016].

U.S. EPA. 2015b. Human Health Risk Assessment: Strategic Research Action Plan 2016-2019. EPA 601/K-15/002. https://www.epa.gov/research/human-health-risk-assessment-strategic-research-action-plan-2016-2019 [accessed 2 September 2016].

U.S. EPA. 2015c. TSCA Chemical Substance Inventory. http://www.epa.gov/oppt/existingchemicals/pubs/tscainventory/basic.html#background [accessed 2 September 2016].

U.S. EPA. 2016. Science in Action. Innovative Research for a Sustainable Future. Computational Toxicology Research [fact sheet]. https://www.epa.gov/sites/production/files/2016-03/documents/comptox_factsheet_mar2016.pdf [accessed 2 September 2016].

Venkatapathy R, Wang NC. 2013. Developmental toxicity prediction. Methods Mol Biol 930:305–340.

Vermeulen R, Li G, Lan Q, Dosemeci M, Rappaport SM, Bohong X, et al. 2004. Detailed exposure assessment for a molecular epidemiology study of benzene in two shoe factories in China. Ann Occup Hyg 48:105–116.

Villeneuve D, Volz DC, Embry MR, Ankley GT, Belanger SE, Léonard M, et al. 2014. Investigating alternatives to the fish early-life stage test: a strategy for discovering and annotating adverse outcome pathways for early fish development. Environ Toxicol Chem 33:158–169.

Vinken M. 2013. The adverse outcome pathway concept: a pragmatic tool in toxicology. Toxicology 312:158–165.

Wambaugh JF, Wetmore BA, Pearce R, Strope C, Goldsmith R, Sluka JP, et al. 2015. Toxicokinetic triage for environmental chemicals. Toxicol Sci 147(1):55–67.

Wang NC, Jay Zhao Q, Wesselkamper SC, Lambert JC, Petersen D, Hess-Wilson JK. 2012a. Application of computational toxicological approaches in human health risk assessment. I. A tiered surrogate approach. Regul Toxicol Pharmacol 63:10–19.

Wang NC, Rice GE, Teuschler LK, Colman J, Yang RS. 2012b. An in silico approach for evaluating a fraction-based, risk assessment method for total petroleum hydrocarbon mixtures. J Toxicol 2012:410143, doi: 10.1155/2012/410143.

Wang NC, Venkatapathy R, Bruce RM, Moudgal C. 2011. Development of quantitative structure-activity relationship (QSAR) models to predict the carcinogenic potency of chemicals. II. Using oral slope factor as a measure of carcinogenic potency. Regul Toxicol Pharmacol 59:215–226.

Weiss JN, Karma A, MacLellan WR, Deng M, Rau CD, Rees CM, et al. 2012. “Good enough solutions” and the genetics of complex diseases. Circ Res 111:493–504.

Wetmore BA, Wambaugh JF, Ferguson SS, Li L, Clewell HJ III, Judson RS, et al. 2013. Relative impact of incorporating pharmacokinetics on predicting in vivo hazard and mode of action from high-throughput in vitro toxicity assays. Toxicol Sci 132:327–346.

Wetmore BA, Wambaugh JF, Ferguson SS, Sochaski MA, Rotroff DM, Freeman K, et al. 2012. Integration of dosimetry, exposure, and high-throughput screening data in chemical toxicity assessment. Toxicol Sci 125:157–174.

Wu W, Wages PA, Devlin RB, Diaz-Sanchez D, Peden DB, Samet JM. 2015. SRC-mediated EGF receptor activation regulates ozone-induced interleukin 8 expression in human bronchial epithelial cells. Environ Health Perspect 123:231–236, doi: 10.1289/ehp.1307379.

Wright FA, Shabalin AA, Rusyn I. 2012. Computational tools for discovery and interpretation of expression quantitative trait loci. Pharmacogenomics 13:343–352.

Zhang L, McHale CM, Greene N, Snyder RD, Rich IN, Aardema MJ, et al. 2014. Emerging approaches in predictive toxicology. Environ Mol Mutagen 55(9):679–688.

Zeise L, Bois FY, Chiu WA, Hattis D, Rusyn I, Guyton KZ. 2013. Addressing human variability in next-generation human health risk assessments of environmental chemicals. Environ Health Perspect 121:23–31, doi: 10.1289/ehp.1205687.


WP-Backgrounds Lite by InoPlugs Web Design and Juwelier Schönmann 1010 Wien