Research Volume 125 | 2017
Prenatal Residential Proximity to Agricultural Pesticide Use and IQ in 7-Year-Old Children
1 School of Public Health, University of California, Berkeley, Berkeley, California, USA
PDF Version (177 KB)
- Residential proximity to agricultural pesticide use has been associated with neural tube defects and autism, but more subtle outcomes such as cognition have not been studied.
- We evaluated the relationship between prenatal residential proximity to agricultural use of potentially neurotoxic pesticides and neurodevelopment in 7-year-old children.
- Participants included mothers and children (n=283) living in the agricultural Salinas Valley of California enrolled in the Center for the Health Assessment of Mothers and Children of Salinas (CHAMACOS) study. We estimated agricultural pesticide use within 1 km of maternal residences during pregnancy using a geographic information system, residential location, and California’s comprehensive agricultural Pesticide Use Report data. We used regression models to evaluate prenatal residential proximity to agricultural use of five potentially neurotoxic pesticide groups (organophosphates, carbamates, pyrethroids, neonicotinoids, and manganese fungicides) and five individual organophosphates (acephate, chlorpyrifos, diazinon, malathion, and oxydemeton-methyl) and cognition in 7-year-old children. All models included prenatal urinary dialkyl phosphate metabolite concentrations.
- We observed a decrease of 2.2 points [95% confidence interval (CI): −3.9, −0.5] in Full-Scale IQ and 2.9 points (95% CI: −4.4, −1.3) in Verbal Comprehension for each standard deviation increase in toxicity-weighted use of organophosphate pesticides. In separate models, we observed similar decrements in Full-Scale IQ with each standard deviation increase of use for two organophosphates (acephate and oxydemeton-methyl) and three neurotoxic pesticide groups (pyrethroids, neonicotinoids, and manganese fungicides).
- This study identified potential relationships between maternal residential proximity to agricultural use of neurotoxic pesticides and poorer neurodevelopment in children. https://doi.org/10.1289/EHP504
Received: 08 January 2016
Revised: 13 May 2016
Accepted: 14 June 2016
Published: 25 May 2017
Address correspondence to R. Gunier, Center for Environmental Research and Children’s Health (CERCH), School of Public Health, University of California at Berkeley, 1995 University Ave., Suite 265, Berkeley, CA 94704. Telephone: (510) 847-3858. E-mail: firstname.lastname@example.org.
A.B. is a volunteer member of the Board for The Organic Center, a nonprofit organization addressing scientific issues around organic food and agriculture. All other authors declare they have no actual or potential competing financial interests.
Note to readers with disabilities: EHP strives to ensure that all journal content is accessible to all readers. However, some figures and Supplemental Material published in EHP articles may not conform to 508 standards due to the complexity of the information being presented. If you need assistance accessing journal content, please contact email@example.com. Our staff will work with you to assess and meet your accessibility needs within 3 working days.
Supplemental Material PDF (178 KB)
Note to readers with disabilities: EHP has provided a 508-conformant table of contents summarizing the Supplemental Material for this article (see below) so readers with disabilities may determine whether they wish to access the full, nonconformant Supplemental Material. If you need assistance accessing journal content, please contact firstname.lastname@example.org. Our staff will work with you to assess and meet your accessibility needs within 3 working days.
Supplemental Table of Contents PDF (102 KB)
In 2007, 311 million kg (684 million pounds) of pesticide active ingredients were used in agriculture in the United States (Grube et al. 2011). California, the state with the largest agricultural output, uses 25% of all U.S. agricultural pesticides, or 84.5 million kg (186 million pounds) annually [California Department of Pesticide Regulation (CDPR) 2014]. Although recent studies have demonstrated widespread organophosphate (OP) pesticide exposures in the general U.S. population, including pregnant women and children [Whyatt et al. 2003; Berkowitz et al. 2003; Lu et al. 2001; Adgate et al. 2001;
Centers for Disease Control and Prevention (CDC) 2009], exposures are often higher in agricultural populations (Lu et al. 2004; Fenske et al. 2002). Among pregnant women in the Center for the Health Assessment of Mothers and Children of Salinas (CHAMACOS) study, all of whom lived in an agricultural region, and many of whom either worked in agriculture or lived with people who did, OP urinary metabolites or dialkylphosphate (DAP) levels were ∼40% higher than those in a representative sample of U.S. women of childbearing age (Bradman et al. 2005). We observed adverse associations in the CHAMACOS study between prenatal maternal DAP concentrations and children’s performance on the Bayley Scales of Infant Development at 2 y (Eskenazi et al. 2007), measures of attention at 5 y (Marks et al. 2010), and on the Wechsler Intelligence Scale for Children (WISC) at 7 y (Bouchard et al. 2011a). Several studies in other populations have similarly reported adverse associations of prenatal exposure to OP pesticides and child neurodevelopment (Engel et al. 2011; Rauh et al. 2011), but few studies have examined the effects of other potentially neurotoxic pesticides on child cognitive development.
Populations residing in agricultural areas may be exposed to a complex mixture of neurotoxic pesticides through pesticide drift and para-occupational exposures (Fenske et al. 2002). Some of these pesticide classes—including OPs as well as carbamates—share at least one mode of action, depression of acetylcholinesterase (AChE), and there is in vitro evidence that there may be additive inhibitory effects from exposure to certain pesticide mixtures (Mwila et al. 2013). Furthermore, although biomarkers such as DAP metabolites are an important tool for assessing exposures to pesticides, several challenges limit the utility of pesticide biomarkers in epidemiologic analyses. For example, many pesticides have a short half-life in the body, and biomarkers reflect only very recent exposures (on the order of hours to days) (Bradman et al. 2013). In addition, no biomarkers are available for some pesticides, leaving only environmental concentrations or modeling to characterize exposure.
Since 1990, all agricultural pesticide applications in California have been compiled in the uniquely comprehensive Pesticide Use Reporting (PUR) database. In several studies, PUR data have been shown to correlate with pesticide levels in various media. For example, we have shown significant associations between nearby use of specific pesticides based on the PUR data and levels in house dust (Harnly et al. 2009; Gunier et al. 2011), and moderate to strong associations (R2 ∼ 0.28 − 0.65 between agricultural use of malathion, chlorpyrifos, and diazinon with community air samples (Harnly et al. 2005). In addition, several epidemiologic studies conducted in California have shown that higher nearby agricultural pesticide use is associated with various adverse health outcomes, including OP and fungicide use with Parkinson disease (Costello et al. 2009; Wang et al. 2011), OP and pyrethroid use with autism (Shelton et al. 2014) and birth defects (Carmichael et al. 2014), organochlorine pesticide use with autism (Roberts et al. 2007), and the use of carbamates (benomyl and methomyl) and the neonicotinoid imidacloprid with neural tube defects in children (Rull et al. 2006; Yang et al. 2014). To our knowledge, there have not been any previous studies evaluating residential proximity to reported agricultural pesticide use and more subtle neurodevelopmental outcomes such as cognitive function in children. In the present study, we evaluated the relationship between prenatal residential proximity to agricultural use of a variety of neurotoxic pesticides and neurodevelopment using WISC assessed at 7 y of age because this test provides a more specific and reliable measure of cognition than earlier neurodevelopmental assessments conducted in our cohort and may have greater implications for school performance.
Between October 1999 and October 2000, we enrolled 601 pregnant women as part of the CHAMACOS study. Women were eligible if they were ≥18 y of age, <20wk gestational age, eligible for California’s low-income health care program (MediCal), spoke English or Spanish, and were planning to deliver at the county hospital. We followed the women through the delivery of 537 live-born children. We excluded twins (n=10) and children with medical conditions that could affect neurodevelopmental assessment (n=4, one child each with Down syndrome, autism, deafness, and hydrocephalus). We included children who had a neurodevelopmental assessment at 7 y (n=330) and whose prenatal residential location was known for at least 75 d per trimester during two or more trimesters of pregnancy (n=283). We excluded two participants who did not have prenatal measurements of DAP metabolites. Our final study population for this analysis was 283. The mothers of children included in our analyses were more likely (p<0.05) than those not included to be married (85% vs. 77%), nonsmokers during pregnancy (96% vs. 92%), and older at delivery (mean=26.9 vs. 24.9 y) than the mothers of children who were not included in these analyses; otherwise, the two populations were similar demographically. Written informed consent was obtained from all women, and oral assent was obtained from all children at 7 y of age; all research was approved by the University of California, Berkeley, Committee for the Protection of Human Subjects before the study began.
We used the Wechsler Intelligence Scale for Children, fourth edition (WISC-IV) (Wechsler 2003) to assess cognitive abilities when the children were 7 y of age. All assessments were completed by a single bilingual psychometrician, who was trained and supervised by a pediatric neuropsychologist. Scores for four domains were calculated based on the following subtests: Verbal Comprehension (composed of Vocabulary and Similarities subtests), Perceptual Reasoning (Block Design and Matrix Reasoning subtests), Working Memory (Digit Span and Letter–Number Sequencing subtests), and Processing Speed (Coding and Symbol Search subtests). We administered all subtests in the dominant language of the child, which was determined through administration of the oral vocabulary subtest of the Woodcock-Johnson/Woodcock-Muñoz Tests of Cognitive Ability in both English and Spanish (Woodcock and Muñoz-Sandoval 1990) at the beginning of the assessment. Among participants included in these analyses, 68% of the children were tested in Spanish, and 32% of the children were tested in English. The psychometrician was blinded to exposure status. We standardized WISC-IV scores against U.S. population–based norms for English- and Spanish-speaking children. We did not administer Letter–Number Sequencing or Symbol Search subtests for the first 3 mo of assessments; therefore, 27 participants lacked scores for Processing Speed and Working Memory domains. A Full-Scale IQ was available for 255 children.
Maternal Interviews and Assessments
Bilingual interviewers conducted maternal interviews in Spanish or English twice during pregnancy (∼13 and 26 wk gestation), after delivery, and when the children were 6 mo and 1, 2, 3.5, 5, 7, and 9 y of age. Interviews obtained demographic information including maternal age, education, country of birth, number of years living in the United States, marital status, paternal education, and family income. We collected residential history information by asking participants if they had moved since the last interview and, if so, the dates of all moves. We conducted home visits shortly after enrollment (∼16 wk gestation) and when the child was 6 mo of age. For both visits, the latitude and longitude coordinates of the participant’s home were determined using a handheld global positioning system unit. Residential mobility during pregnancy was common in our cohort, with 53% of all participants moving at least once during pregnancy. For the present analysis, we included women in the sample if their residential location was known for ≥75 d per trimester for at least two trimesters of pregnancy.
Mothers were administered the Peabody Picture Vocabulary Test (PPVT) for English speakers or the Test de Vocabulario en Imagenes Peabody (TVIP) for Spanish speakers at the 6-mo visit to assess verbal intelligence (Dunn and Dunn 1981). If maternal PPVT or TVIP scores were unavailable from the 6-mo visit, we used scores from the re-administration of the test conducted at a 9-y visit (n=5) or assigned the mean score of the sample (n=2). A short version of the HOME (Home Observation for Measurement of the Environment) inventory was completed during the 7-y visit (Caldwell and Bradley 1984).
Geographic-Based Estimates of Agricultural Pesticide Use
We estimated agricultural pesticide use near each woman’s residence during pregnancy using California PUR data from 1999–2001 (CDPR 2015). We selected potentially neurotoxic pesticides with agricultural use in our study area (Monterey County, CA) during the prenatal period, including fifteen OPs and six carbamates (see Table S1), two manganese (Mn)-based fungicides (maneb and mancozeb), eight pyrethroids (permethrin, cypermethrin, tau-fluvalinate, cyfluthrin, fenpropathrin, lambda-cyhalothrin, bifenthrin, and esfenvalerate), and one neonicotinoid (imidacloprid). The PUR data include the amount (in kilograms) of active ingredient applied, the application date, and the location, defined as a 1-mi2 section (1.6km×1.6km) defined by the Public Land Survey System (PLSS). We edited the PUR data to correct for likely outliers that had unusually high application rates by replacing the amount of pesticide applied based on the median application rate for that pesticide and crop combination (Gunier et al. 2001). For each woman, we estimated the amount of all pesticides in each pesticide class used within a 1-km radius of the pregnant woman’s residence using the latitude and longitude coordinates and a geographic information system. In all cases, the 1-km buffer around the home included more than one PLSS section; thus, we weighted the amount of pesticide applied in each section by the proportion of land area that was included in the buffer. We selected a 1-km buffer distance for this analysis because it best captures the spatial scale most strongly correlated with measured agricultural pesticide concentrations in house-dust samples (Harnly et al. 2009; Gunier et al. 2011). Detailed descriptions of the equations and methods that we used to calculate nearby pesticide use have been published previously (Gunier et al. 2011). We estimated pesticide use within 1 km of the maternal residence during each trimester of pregnancy for participants with residential location information available for two or more trimesters (n=283) and computed the average pesticide use during pregnancy by summing the trimester-specific values and dividing by the number of trimesters included. We also created individual variables for nearby use of each of the five individual OP pesticides (acephate, chlorpyrifos, diazinon, malathion, and oxydemeton-methyl) with the highest use in our study area during the prenatal period (Table S1).
Toxicity Weighting and Neurotoxic Pesticide Index
In addition to examining the simple sum of pesticide use in each class, we also used relative potency factors (RPFs) to generate class-specific toxicity-weighted sums to account for differences in the neurotoxicity of individual pesticides in OP and carbamate classes. The RPF of a chemical is the ratio of the relevant toxicological dose of an index chemical to the relevant toxicological dose of the chemical of interest. Currently, RPFs are available for OP and carbamate pesticides [U.S. Environmental Protection Agency (EPA) 2006, 2007], but not for neonicotinoids, pyrethroids, or Mn-fungicides. Thus, we were only able to create toxicity-weighted sum variables for OPs and carbamates. We calculated the toxicity-weighted use for each OP or carbamate pesticide, expressed as kg-equivalents of chlorpyrifos, by multiplying the kilograms of pesticide used within 1 km of the maternal residence during each trimester for each pesticide by the RPF of that pesticide, and summing to create the toxicity-weighted use for the fifteen OP and six carbamate pesticides. Additionally, because these pesticides share a common mechanism of toxicity [acetylcholinesterase (AChE) inhibition], we also generated a toxicity-weighted sum for OPs and carbamates combined (Jensen et al. 2009; U.S. EPA 2002). The RPFs, total kilograms and toxicity-weighted kilograms of use in the Salinas Valley in 2000 for each OP and carbamate pesticide are provided in Table S1.
Finally, we created a rank index of neurotoxic pesticide use that included the five pesticide classes of interest (i.e., OPs, carbamates, neonicotinoids, pyrethroids, and Mn-fungicides) by generating a percentile rank of the participants from lowest to highest use for each neurotoxic pesticide class and then calculating the average percentile rank across the five classes. We also explored principal components analysis (PCA) as a method for combining pesticide use across the five different classes of neurotoxic pesticides.
We log10-transformed continuous pregnancy average and trimester-specific sums of pesticide use (kilograms/year+1) to reduce heteroscedasticity and the influence of outliers and to improve the linear fit of the model. The scores for Full-Scale IQ and for the four subdomains were normally distributed and were modeled as continuous outcomes.
We selected model covariates a priori based on factors associated with infant neurodevelopment in previous analyses [i.e., child’s exact age at assessment, sex, maternal PPVT score (continuous) and maternal education (<sixth grade vs.≥seventh grade)]. We considered the following variables as additional covariates in our models (Table 1): maternal country of birth, maternal age at delivery, marital status at enrollment, and maternal depression [using the Center for Epidemiologic Studies Depression Scale (CES-D)] (≥16 on CES-D) at the child’s 7-y visit. In addition, we considered covariates collected at each visit including housing density (number of persons per room), HOME score (continuous), household poverty level (<federal poverty level vs.≥federal poverty level), presence of father in the home (yes/no), maternal work status, location of assessment (field office or recreational vehicle), and season of assessment. We imputed missing values (<10% missing) at a visit point using data from the nearest available visit.
|Maternal country of birth|
|United States and other||34 (12.0)|
|Maternal age at delivery (years)|
|≤6th grade||132 (46.6)|
|≥7th grade||151 (53.4)|
|Marital status at enrollment|
|Married/living as married||239 (84.4)|
|Married/living as married||239 (84.4)|
|Not married||44 (15.6)|
|≤6th grade||129 (60.6)|
|≥7th grade||84 (39.4)|
|Family income at 7-y visit|
|<Poverty level||203 (71.7)|
|≥Poverty level||80 (28.3)|
|Maternal depression at 7-y visit|
|Language of WISC-IV tests|
|Note: WISC-IV, Wechsler Intelligence Scale for Children, fourth edition.|
We retained covariates that were significant (p<0.2) at any time point in the multivariate regression models and used the same covariates in all models. We fit separate regression models for each pesticide class or individual pesticide; we also fit models including multiple classes or pesticides.
We used generalized additive models (GAMs) with a three-degrees-of-freedom cubic spline function to test for nonlinearity. None of the digression from linearity tests was significant (p<0.05); therefore, we expressed neurotoxic pesticide use linearly (on the log10 scale) in regression models.
We controlled for DAP metabolites of OP insecticides measured in maternal urine samples (Bradman et al. 2005) collected during prenatal interviews at 13 and 26 wk gestation (n=283) in all models. Prenatal DAPs and agricultural use of OPs were not highly correlated in our cohort (ρ=0.04); therefore, we believe that they provide complementary measures of exposure to OP pesticides. We averaged the two prenatal DAP measurements and used log10-transformed concentrations (nanomoles/liter) in our analyses. In separate sensitivity analyses, we controlled for exposure to other neurotoxicants, which we have previously found to be related to child IQ in our cohort (Eskenazi et al. 2013; Gaspar et al. 2015). Specifically, we considered log10-transformed lipid-adjusted concentrations (nanograms/gram-lipid) measured in prenatal maternal blood samples of p,p′-dichlorodiphenyltrichloroethylene (DDT), p,p′-dichlorodiphenyldichloroethylene (DDE) (n=219) (Bradman et al. 2007), and polybrominated diphenyl ether flame retardants (PBDEs) (n=221) (Castorina et al. 2011). We used the sum of the four major congeners (BDE-47, BDE-99, BDE-100, and BDE-153) to estimate PBDE exposure (Eskenazi et al. 2013).
In other sensitivity analyses, we excluded outliers identified with studentized residuals >3. To control for potential selection bias resulting from loss to follow-up, we ran regression models with weights determined as the inverse probability of inclusion in our analyses at each time point (Hogan et al. 2004). We determined the probability of inclusion using multiple logistic regression models with baseline covariates as potential predictors.
Most mothers were born in Mexico (88.0%), under 30 y of age at delivery (72.8%), and married or living as married (84.4%) at the time of enrollment (Table 1). Nearly half of the mothers (46.6%) and most fathers (60.6%) had a sixth-grade education or less, and most families (71.7%) were living below the poverty level at the time of the 7-y visit. Slightly more than half of the children were girls (53.7%), and most children completed their WISC-IV assessment in Spanish (68.2%). The distributions of agricultural use of neurotoxic pesticides within 1 km of maternal residences during pregnancy are shown in Table 2. The most heavily used pesticide class was OPs, followed by Mn-fungicides. The geometric mean (GM) and geometric standard deviation (GSD) for the cumulative use of 15 OP pesticides within 1 km of maternal residences during pregnancy was 75 (5) kg. For individual OP pesticides, the GM (GSD) ranged from 5 (6) kg for malathion to 23 (3) kg for diazinon, and the use of these five individual OP pesticides was moderately (0.40 for malathion and diazinon) to highly (0.91 for acephate and oxydemeton-methyl) correlated. The GM (GSD) of the other neurotoxic pesticide groups ranged from 4 (2) kg for neonicotinoids to 54 (4) kg for Mn-fungicides. There was moderate to high correlation (0.68− 0.90) between the use of the five different neurotoxic pesticide groups within one kilometer of the maternal residence during pregnancy (Table S2).
|Kilograms used (2000)||Kilograms used within 1 km of residence during pregnancy|
|Neurotoxic pesticides||Monterey||p25||p50||p75||Max||GM (GSD)|
|OPs toxicity weighted||964,130||59||331||891||8,587||175 (9)|
|Carbamates toxicity weighted||335,611||14||66||291||9,222||66 (7)|
|Carbamates and OPs toxicity weighted||1,299,741||95||485||1,379||9,774||270 (9)|
|Notes: GM, geometric mean; GSD, geometric standard deviation; Max, maximum; Mn, manganese; OPs, organophosphates; p25, 25th percentile; p50, 50th percentile; p75, 75th percentile.|
In general, IQ scores decreased across all domains with increasing use of OP pesticides within 1 km of the maternal residence during pregnancy. Each SD increase in toxicity-weighted OP use during pregnancy was associated with an estimated 2.2-point [95% confidence interval (CI): −3.9, −0.5] (Table 3) decrease in Full-Scale IQ, which was very similar to but slightly greater than that for the nonweighted use of all OP pesticides combined (−2.1 points; 95% CI: −3.8, −0.3) (Table 4). A 1-SD increase in toxicity-weighted OP pesticide use during pregnancy was also associated with a 2.9-point decrease in Verbal Comprehension scores (95% CI: −4.4, −1.3) (Table 3). The results were similar in the unadjusted and adjusted models. For the other WISC domains, there was a nonsignificant decrease of ∼1.4 points per SD increase in toxicity-weighted OP pesticide use. The results were similar whether or not we included the maternal prenatal urinary DAP concentrations in the model. In fact, we observed independent and similar decreases in WISC scores for both a 1-SD increase in prenatal urinary DAPs and a 1-SD increase in toxicity-weighted OP pesticide use when both exposures were included in the same model (Table 3).
|Unadjusted model||Adjusteda models|
|Toxicity-weighted OP pesticides (kg)||Toxicity-weighted OP pesticides (kg)||Urinary DAPs (nmol/L)|
|Cognitive test (WISC-IV Scale)||n||β||(95% CI)||β||(95% CI)||β||(95% CI)|
|Working Memory||256||−0.9||(−2.7, 0.8)||−1.4||(−3.1, 0.4)||−1.7||(−3.3, 0.1)*|
|Processing Speed||256||−0.9||(−2.6, 0.9)||−1.3||(−3.0, 0.5)||−1.2||(−2.8, 0.4)|
|Verbal Comprehension||283||−3.5||(−5.5, 1.5)**||−2.9||(−4.4, 1.3)**||−2.8||(−4.3, 1.4)**|
|Perceptual Reasoning||283||−1.2||(−3.1, 0.7)||−1.4||(−3.3, 0.5)||−1.7||(−3.5, 0.1)|
|Full-Scale IQ||255||−2.1||(−4.0, −0.3)*||−2.2||(−3.9, −0.5)*||−2.4||(−4.0, −0.9)**|
|Notes: CI, confidence interval; DAPs, dialkylphosphates; OPs, organophosphates; WISC-IV, Wechsler Intelligence Scale for Children, fourth edition.
aAdjusted for child’s age at assessment, sex, language of assessment, maternal education, maternal intelligence, maternal country of birth, maternal depression at 7-year visit, Home Observation for Measurement of the Environment (HOME) Score at 7-year visit and household poverty level at 7-year visit. *p<0.05. ** p<0.01.
OPs−2.1(−3.8, −0.3)−1.3(−3.1, 0.4)−1.1(−2.9, 0.7)−1.8(−3.7, 0.1)−2.5(−4.1, −1.0)** Acephate−2.3(−3.9, −0.6)**−1.4(−3.1, 0.3)−1.4(−3.1, 0.2)−1.8(−3.6, 0.1)−2.7(−4.3, −1.2)** Chlorpyrifos−1.4(−3.0, 0.2)−1.3(−3.0, 0.3)−0.3(−1.9, 1.4)−0.8(−2.6, 1.1)−2.2(−3.7, −0.7)**
|Full-Scale IQ (n=255)||Working Memory (n=256)||Processing Speed (n=256)||Perceptual Reasoning (n=283)||Verbal Comprehension (n=283)|
|Neurotoxic pesticides||β||(95% CI)||β||(95% CI)||β||(95% CI)||β||(95% CI)||β||(95% CI)|
|Diazinon||−1.7||(−3.4, 0.1)||−1.3||(−3.0, 0.5)||−1||(−2.8, 0.7)||−1.7||(−3.6, 0.2)||−1.6||(−3.2, −0.1)*|
|Malathion||−0.8||(−2.5, 0.8)||−0.8||(−2.4, 0.9)||0.8||(−0.9, 2.5)||−1.2||(−3.0, 0.6)||−1.3||(−2.8, 0.2)|
|Oxydemeton–methyl||−2.3||(−4.0, −0.7)**||−1.5||(−3.2, 0.2)||−1.5||(−3.2, 0.2)||−1.5||(−3.4, 0.4)||−2.8||(−4.3, −1.3)**|
|OPs toxicity weighted||−2.2||(−3.9, −0.5)**||−1.4||(−3.1, 0.4)||−1.3||(−3.0, 0.5)||−1.4||(−3.3, 0.5)||−2.9||(−4.4, −1.3)**|
|Carbamates||−1.2||(−2.8, 0.4)||−0.4||(−2.0, 1.2)||−0.2||(−1.8, 1.4)||−1.1||(−2.9, 0.7)||−2.4||(−3.9, −1.0)**|
|Carbamates toxicity weighted||−1.3||(−2.9, 0.3)||−0.6||(−2.2, 1.1)||−0.1||(−1.7, 1.5)||−1||(−2.8, 0.8)||−2.5||(−4.0, −1.0)**|
|Carbamates and OPs toxicity weighted||−2.1||(−3.7, −0.4)*||−1.1||(−2.9, 0.6)||−1||(−2.7, 0.7)||−1.4||(−3.3, 0.5)||−2.9||(−4.5, −1.4)**|
|Neonicotinoids||−1.7||(−3.3, 0.0)*||−1.1||(−2.8, 0.5)||−0.8||(−2.4, 0.9)||−1.9||(−3.8, −0.1)*||−1.9||(−3.5, −0.3)*|
|Pyrethroids||−2||(−3.7, −0.3)*||−1.5||(−3.2, 0.2)||−1.1||(−2.8, 0.6)||−2.1||(−4.0, −0.2)*||−1.8||(−3.4, −0.3)*|
|Mn-fungicides||−2||(−3.7, −0.2)||−1.2||(−2.9, 0.6)||−1.2||(−2.9, 0.6)||−1.7||(−3.6, 0.1)||−2.1||(−3.7, −0.6)**|
|Rank index of 5 neurotoxic groups||−2||(−3.7, −0.4)*||−1.3||(−3.0, 0.4)||−1||(−2.7, 0.7)||−1.9||(−3.8, −0.1)*||−2.4||(−4.0, −0.9)**|
|Notes: Mn, manganese. Adjusted for child’s age at assessment, sex, language of assessment, maternal education, maternal intelligence, maternal country of birth, maternal depression at 7-y visit, Home Observation for Measurement of the Environment (HOME) score at 7-y visit, household poverty level at 7-year visit and prenatal urinary dialkylphosphates. *p<0.05. ** p<0.01.|
In separate adjusted models for individual OP pesticides (Table 4), there was a 2.3-point decrease in Full-Scale IQ for each SD increase in acephate (95% CI: −3.9, −0.6) or oxydemeton-methyl (95% CI: −4.0, −0.7). Although they were also negatively related, there was no significant relationship between the use of chlorpyrifos, malathion, or diazinon and Full-Scale IQ. There was a significant negative relationship between agricultural use of acephate, chlorpyrifos, diazinon, and oxydemeton-methyl and Verbal Comprehension. There was no relationship with Working Memory, Processing Speed, or Perceptual Reasoning for any of the individual OPs evaluated. We also included the top 5 OP pesticides in the same model (Model 1, Table S3) and found no association (p<0.05) with Full-Scale IQ for any individual OP pesticide, but the strongest relationship was with oxydemeton-methyl (−4.2 points; 95% CI: −10.1, 18).
Other Pesticide Groups
We observed a nearly universal trend of lower IQ scores for all domains with greater use of individual OP pesticides and other potentially neurotoxic pesticide groups within 1 km of the maternal residence during pregnancy (Table 4). The combined toxicity-weighted use of OPs and carbamates was associated with decreased Full-Scale IQ, although the unweighted and toxicity-weighted use of carbamates alone was not significantly associated with Full-Scale IQ. In separate models with the other neurotoxic pesticide groups, the use of neonicotinoids, pyrethroids, and Mn-fungicides was each significantly associated (p<0.05) with an approximately 2-point decrease in Full-Scale IQ, Perceptual Reasoning, and Verbal Comprehension (Table 4).
In a single model that included all five neurotoxic pesticide groups, the strongest association with Full-Scale IQ was for toxicity-weighted OP pesticide use with a 2.8-point decrease (95% CI: −6.8, 1.3) for each SD increase (Model 2, Table S3). The associations with IQ scales were nearly identical for the first component from PCA, which explained ∼80% of the variance and weighted the five neurotoxic pesticide groups equally, and the average rank index, with a 2.0-point decrease (95% CI: −3.7, −0.4) in Full-Scale IQ for each SD increase in estimated exposure (Table 4).
For all analyses, point estimates were similar but confidence intervals were wider when we restricted the study population to those mothers with residential location known for three trimesters of pregnancy (n=150 with Full-Scale IQ), and associations were weaker when we included all mothers with residential location known for one trimester during pregnancy (n=284 with Full-Scale IQ) (data not shown). Our results were very similar after excluding the relatively few outliers (1–3 participants per exposure/outcome) based on studentized residuals (data not shown). Associations between prenatal neurotoxic pesticide use and WISC scores at 7 y of age became slightly stronger when we used inverse probability weighting to adjust for potential selection bias (data not shown). Including other prenatal exposures that have been related to WISC scores at 7 y of age (DDT/DDE and PBDEs) reduced the number of participants (n=191 with Full-Scale IQ), but the results were very similar for toxicity-weighted OP pesticide use and the other neurotoxic pesticide groups for all WISC scales with and without inclusion of these other prenatal exposures in the models.
We observed an inverse relationship between the agricultural use of OP pesticides within 1 km of maternal residences during pregnancy and cognitive development in children at 7 y of age. For each standard deviation increase in agricultural use of total OPs (237 kg) or toxicity-weighted OPs, there was a 2-point decrease (15% of a standard deviation) in Full-Scale IQ. To put these findings in perspective, other authors have estimated that each 1-point decrease in IQ decreases worker productivity by ∼2% (Grosse et al. 2002), reducing lifetime earnings by US$18,000 in 2005 dollars (Nedellec and Rabl 2016). The results were independent of prenatal urinary DAP concentrations in the model, and the effect estimates of nearby OP use and urinary DAPs were of similar magnitude. These independent associations suggest that our previous observation of a relationship between prenatal urinary DAPs and IQ (Bouchard et al. 2011a) did not completely account for exposure to OP pesticides during pregnancy and that using both urinary DAPs and PUR data seems to provide a more complete characterization of OP pesticide exposure. Urinary DAP concentrations provide an estimate of exposure to some, but not all, of the OP pesticides we evaluated using PUR data (Castorina et al. 2010) and primarily reflect dietary exposures (McKone et al. 2007; Bradman et al. 2015). The two individual OP pesticides that had the strongest inverse relationship with Full-Scale IQ were acephate and oxydemeton-methyl, but agricultural use of these two pesticides was highly correlated (r=0.91). It is important to note that although oxydemeton-methyl devolves to urinary DAPs, acephate does not; it is also important to note that oxydemeton-methyl is the most toxic of all the OPs used in the Salinas Valley (∼11 times more toxic than acephate, based on the RPF).
Agricultural use of other potentially neurotoxic pesticide classes was correlated with the use of OPs, and there were also significant inverse associations between Full-Scale IQ and nearby agricultural use of pyrethroid insecticides, Mn-based fungicides (mostly maneb), and a neonicotinoid insecticide (imidacloprid). The combined agricultural use of pesticides from five neurotoxic pesticide classes based on an average rank index and PCA produced similar results to those observed for toxicity-weighted OP pesticide use alone, making it difficult to determine whether OP pesticide use alone is driving the relationship or if the results are caused by the combined use of neurotoxic pesticides that are highly correlated.
This is the first study to evaluate the relationship between cognitive abilities in children and reported agricultural use of neurotoxic pesticides near maternal residences during pregnancy. A recent study conducted in Spain that used residential proximity to agricultural fields as a proxy for pesticide exposure observed an inverse relationship between postnatal, but not prenatal, hectares of crops near the residence and Full-Scale IQ, Verbal Comprehension, and Processing Speed in children 6–11 y of age (González-Alzaga et al. 2015). A study in California utilizing PUR data found that any agricultural use of OPs or pyrethroids within 1.5 km of maternal residences during the third trimester of pregnancy compared with no agricultural use of these pesticides was associated with an approximately doubled risk of autism spectrum disorder (Shelton et al. 2014). Higher concentrations of pyrethroid metabolites in children’s urine have been associated with an increased risk of behavioral problems in school-age children (Oulhote and Bouchard 2013) and with attention deficit/hyperactivity disorder in one study (Wagner-Schuman et al. 2015) but not in another (Quirós-Alcalá et al. 2014). Previous studies have observed inverse associations between children’s cognition and levels of manganese in blood (Riojas-Rodríguez et al. 2010) and hair (Bouchard et al. 2011b; Menezes-Filho et al. 2011). Proximity to agricultural use of the neonicotinoid imidacloprid during pregnancy has been associated with an increased risk of neural tube defects (Rull et al. 2006; Yang et al. 2014), but there are no previously published studies evaluating cognition in children.
The main strength of this study is the use of PUR data, which provide the amount of active ingredients in and the locations of all agricultural pesticide applications, and which represent a major improvement in exposure classification compared with using only crop locations, as in the recent Spanish study (González-Alzaga et al. 2015). The PUR data allowed us to examine pesticides that do not have biomarkers and to assess pesticide mixtures. We also had extensive information on potential confounders and other chemical exposures available for the CHAMACOS cohort. However, there are some limitations of our study. We were able to determine proximity to agricultural pesticide use only at the maternal residence, but not at other locations where the mother may have spent time. Although we used residential proximity to agricultural pesticide use as a proxy for pesticide exposure, previous studies have shown that PUR data are correlated with environmental pesticide concentrations (Harnly et al. 2009; Harnly et al. 2005), suggesting that these data provide a meaningful indicator of pesticide exposure. We did not account for prenatal exposure information from other potential sources of pesticide exposure including home use, occupational take-home, and dietary intake, but our models included prenatal urinary DAPs, which we believe primarily plect both dietary and residential exposures (McKone et al. 2007). In general, these limitations would likely lead to exposure misclassification and would bias our results toward the null.
People living in agricultural communities are exposed to a complex mixture of many individual pesticide active ingredients as well as to potentially neurotoxic adjuvants included in the formulation. Better methods are needed for toxicity weighting across neurotoxic pesticide classes. To improve pesticide exposure assessment based on PUR data, exposure models should be optimized using measured pesticide concentrations in air or house-dust samples. In future analyses, we intend to use more complex statistical methods to address collinearity of exposures, such as weighted quantile sum regression (Gennings et al. 2013) and hierarchical Bayesian models (Kalkbrenner et al. 2010; Rull et al. 2009). The results from the present study need to be replicated in studies that include children living in both agricultural and nonagricultural communities to obtain more variability in exposure to pesticide mixtures.
We observed an inverse association between prenatal residential proximity to agricultural use of OPs and other neurotoxic pesticides and cognition in children at 7 y of age. The results for OPs based on PUR data remained significant when we included prenatal urinary maternal DAP concentrations in the model, and the effect estimates of nearby OP use and urinary DAPs were of similar magnitude. The association also remained after adjustment for prenatal exposure to other neurotoxic chemicals. Agricultural use of individual pesticides and classes of neurotoxic pesticides were highly correlated, making it difficult to identify the specific pesticides that were driving these associations.
This study was funded by the National Institute of Environmental Health Sciences grants P01 ES009605 and R56ES023591 and U.S. Environmental Protection Agency grants R82670901 and RD83451301.
Adgate JL, Barr DB, Clayton CA, Berly LE, Freeman NC, Lioy PJ, et al. 2001. Measurement of children’s exposure to pesticides: analysis of urinary metabolite levels in a probability-based sample. Environ Health Perspect 109(6):583–590, 10.1289/ehp.01109583.
Bouchard MF, Chevrier J, Harley KG, Kogut K, Vedar M, Calderon N, et al. 2011a. Prenatal exposure to organophosphate pesticides and iq in 7-year-old children. Environ Health Perspect 119(8):1189–1195, 10.1289/ehp.1003185.
Bouchard MF, Sauve S, Barbeau B, Legrand M, Brodeur ME, Bouffard T, et al. 2011b. Intellectual impairment in school-age children exposed to manganese from drinking water. Environ Health Perspect 119(1):138–143.
Bradman A, Eskenazi B, Barr DB, Bravo R, Castorina R, Chevrier J, et al. 2005. Organophosphate urinary metabolite levels during pregnancy and after delivery in women living in an agricultural community. Environ Health Perspect 113(12):1802–1807, 10.1289/ehp.7894.
Bradman AS, Schwartz JM, Fenster L, Barr DB, Holland NT, Eskenazi B. 2007. Factors predicting organochlorine pesticide levels in pregnant Latina women living in a United States agricultural area. J Expos Sci Environ Epidemiol 17(4):388–399, 10.1038/sj.jes.7500525.
Bradman A, Kogut K, Eisen EA, Jewell NP, Quirós-Alcalá L, Castorina R, et al. 2013. Variability of organophosphorous pesticide metabolite levels in spot and 24-hr urine samples collected from young children during 1 week. Environ Health Perspect 121(1):118–124, PMID: 23052012.
Bradman A, Quirós-Alcalá L, Castorina R, Schall RA, Camacho J, Holland NT, et al. 2015. Effect of organic diet intervention on pesticide exposures in young children living in low-income urban and agricultural communities. Environ Health Perspect 123(10):1086–1093, 10.1289/ehp.1408660.
Caldwell BM, Bradley RH. 1984. Home observation for measurement of the environment. Little Rock, AR:University of Arkansas.
CDPR (California Department of Pesticide Regulation). 2014. Pesticide Use Reporting Data for 2012. Sacramento, CA:CDPR.
CDPR. 2015. Pesticide Use Reporting (PUR). Available: http://www.cdpr.ca.gov/docs/pur/purmain.htm [accessed December 3 2015].
Carmichael SL, Yang W, Roberts E, Kegley SE, Padula AM, English PB, et al. 2014. Residential agricultural pesticide exposures and risk of selected congenital heart defects among offspring in the San Joaquin Valley of California. Environ Res 135C:133–138, 10.1016/j.envres.2014.08.030.
Castorina R, Bradman A, Sjödin A, Fenster L, Jones RS, Harley KG, et al. 2011. Determinants of serum polybrominated diphenyl ether (PBDE) levels among pregnant women in the CHAMACOS cohort. Environ Sci Technol 45(15):6553–6560, PMID: 21793581, 10.1021/es104295m.
CDC (Centers for Disease Control and Prevention). 2009. Fourth National Report on Human Exposure to Chemicals. Atlanta, GA:CDC.
Costello S, Cockburn M, Bronstein J, Zhang X, Ritz B. 2009. Parkinson’s Disease and Residential Exposure to Maneb and Paraquat From Agricultural Applications in the Central Valley of California. Am J Epidemiol. 169(8):919–926, PMID: 19270050, 10.1093/aje/kwp006.
Dunn L, Dunn L. 1981. Peabody Picture Vocabulary Test, Revised. Circle Pines, MN: American Guidance Service.
Engel SM, Wetmur J, Chen J, Zhu C, Barr DB, Canfield RL, et al. 2011. Prenatal exposure to organophosphates, paraoxonase 1, and cognitive development in childhood. Environ Health Perspect 119(8):1182–1188, PMID: 21507778, 10.1289/ehp.1003183.
Eskenazi B, Marks AR, Bradman A, Harley K, Barr DB, Johnson C, et al. 2007. Organophosphate pesticide exposure and neurodevelopment in young Mexican-American children. Environ Health Perspect 115(5):792–798, PMID: 17520070, 10.1289/ehp.9828.
Eskenazi B, Chevrier J, Rauch SA, Kogut K, Harley KG, Johnson C, et al. 2013. In utero and childhood polybrominated diphenyl ether (PBDE) exposures and neurodevelopment in the CHAMACOS Study. Environ Health Perspect 121(2):257–262, PMID: 23154064, 10.1289/ehp.1205597.
Fenske RA, Lu C, Barr D, Needham L. 2002. Children’s exposure to chlorpyrifos and parathion in an agricultural community in central Washington State. Environ Health Perspect 110(5):549–553, 10.1289/ehp.02110549.
Gaspar FW, Harley KG, Kogut K, Chevrier J, Mora AM, Sjödin A, et al. 2015. Prenatal DDT and DDE exposure and child IQ in the CHAMACOS cohort. Environ Int 85:206–212, PMID: 26414943, 10.1016/j.envint.2015.09.004.
Gennings C, Carrico C, Factor-Litvak P, Krigbaum N, Cirillo PM, Cohn BA. 2013. A cohort study evaluation of maternal PCB exposure related to time to pregnancy in daughters. Environ Health 12(1):66, 10.1186/1476-069X-12-66.
González-Alzaga B, Hernández AF, Rodríguez-Barranco M, Gómez I, Aguilar-Garduño C, López-Flores I, et al. 2015. Pre- and postnatal exposures to pesticides and neurodevelopmental effects in children living in agricultural communities from South-Eastern Spain. Environ Int 85:229–237, 10.1016/j.envint.2015.09.019.
Grosse SD, Matte TD, Schwartz J, Jackson RJ. 2002. Economic gains resulting from the reduction in children’s exposure to lead in the United States. Environ Health Perspect 110(6):563–569, PMID: 12055046, 10.1289/ehp.02110563.
Grube A, Donaldson D, Kiely T, Wu L. 2011. Pesticides Industry Sales and Usage: 2006 and 2007 Market Estimates. Washington, DC:U.S. EPA, https://www.epa.gov/sites/production/files/2015-10/documents/market_estimates2007.pdf.
Gunier RB, Harnly ME, Reynolds P, Hertz A, Von Behren J. 2001. Agricultural pesticide use in California: pesticide prioritization, use densities, and population distributions for a childhood cancer study. Environ Health Perspect 109(10):1071–1078, PMID: 11689348, 10.1289/ehp.011091071.
Gunier RB, Ward MH, Airola M, Bell EM, Colt J, Nishioka M, et al. 2011. Determinants of agricultural pesticide concentrations in carpet dust. Environ Health Perspect 119(7):970–976, PMID: 21330232, 10.1289/ehp.1002532.
Harnly M, McLaughlin R, Bradman A, Anderson M, Gunier R. 2005. Correlating agricultural use of organophosphates with outdoor air concentrations: a particular concern for children. Environ Health Perspect 113(9):1184–1189, PMID: 16140625, 10.1289/ehp.7493.
Harnly ME, Bradman A, Nishioka M, McKone TE, Smith D, McLaughlin R, et al. 2009. Pesticides in dust from homes in an agricultural area. Environ Sci Technol 43(23):8767–8774, PMID: 19943644, 10.1021/es9020958.
Jensen BH, Petersen A, Christensen T. 2009. Probabilistic assessment of the cumulative dietary acute exposure of the population of Denmark to organophosphorus and carbamate pesticides. Food Add Contam Part A 26(7):1038–1048, 10.1080/02652030902859754.
Lu C, Kedan G, Fisker-Andersen J, Kissel JC, Fenske RA. 2004. Multipathway organophosphorus pesticide exposures of preschool children living in agricultural and nonagricultural communities. Environ Res 96(3):283–289, PMID: 15364595, 10.1016/j.envres.2004.01.009.
Marks AR, Harley K, Bradman A, Kogut K, Barr DB, Johnson C, et al. 2010. Organophosphate pesticide exposure and attention in young Mexican-American children: The CHAMACOS study. Environ Health Perspect 118(12):1768–1774, PMID: 21126939, 10.1289/ehp.1002056.
McKone TE, Castorina R, Harnly ME, Kuwabara Y, Eskenazi B, Bradman A. 2007. Merging models and biomonitoring data to characterize sources and pathways of human exposure to organophosphorus pesticides in the Salinas Valley of California. Environ Sci Technol 41(9):3233–3240, PMID: 17539531, 10.1021/es0618447.
Mwila K, Burton MH, Van Dyk JS, Pletschke BI. 2013. The effect of mixtures of organophosphate and carbamate pesticides on acetylcholinesterase and application of chemometrics to identify pesticides in mixtures. Environ Monit Assess 185(3):2315–2327, 10.1007/s10661-012-2711-0.
Nedellec V, Rabl A. 2016. Costs of health damage from atmospheric emissions of toxic metals: Part 2-analysis for mercury and lead. Risk Anal, 10.1111/risa.12598. [Epub ahead of print]
Oulhote Y, Bouchard M. 2013. Urinary metabolites of organophosphate and pyrethroid pesticides and behavioral problems in Canadian children. Environ Health Perspect 121(11–12):1378–1384, 10.1289/ehp.1306667.
Quirós-Alcalá L, Mehta S, Eskenazi B. 2014. Pyrethroid pesticide exposure and parental report of learning disability and attention deficit/hyperactivity disorder in U.S. children: NHANES 1999–2002. Environ Health Perspect 122(12):1336–1342, 10.1289/ehp.1308031.
Rauh V, Arunajadai S, Horton M, Perera F, Hoepner L, Barr DB, et al. 2011. Seven-year neurodevelopmental scores and prenatal exposure to chlorpyrifos, a common agricultural pesticide. Environ Health Perspect 119(8):1196–1201, PMID: 21507777, 10.1289/ehp.1003160.
Riojas-Rodríguez H, Solís-Vivanco R, Schilmann A, Montes S, Rodríguez S, Ríos C, et al. 2010. Intellectual function in Mexican children living in a mining area and environmentally exposed to manganese. Environ Health Perspect 118(10):1465–1470, 10.1289/ehp.0901229.
Roberts EM, English PB, Grether JK, Windham GC, Somberg L, Wolff C. 2007. Maternal residence near agricultural pesticide applications and autism spectrum disorders among children in the California Central Valley. Environ Health Perspect 115(10):1482–1489, PMID: 17938740, 10.1289/ehp.10168.
Rull RP, Gunier R, Von Behren J, Hertz A, Crouse V, Buffler PA, et al. 2009. Residential proximity to agricultural pesticide applications and childhood acute lymphoblastic leukemia. Environ Res 109(7):891–899, PMID: 19700145, 10.1016/j.envres.2009.07.014.
Shelton JF, Geraghty EM, Tancredi DJ, Delwiche LD, Schmidt RJ, Ritz B, et al. 2014. Neurodevelopmental disorders and prenatal residential proximity to agricultural pesticides: The CHARGE Study. Environ Health Perspect 122(10):1103–1109, PMID: 24954055, 10.1289/ehp.1307044.
U.S. EPA (U.S. Environmental Protection Agency). 2006. Organophosphorous Cumulative Risk Assessment – 2006 Update. Washington, DC:U.S. EPA.
U.S. EPA (U.S. Environmental Protection Agency). 2007. Revised N-Methyl Carbamate Cumulative Risk Assessment. Washington, DC:U.S. EPA.
Wagner-Schuman M, Richardson JR, Auinger P, Braun JM, Lanphear BP, Epstein JN, et al. 2015. Association of pyrethroid pesticide exposure with attention-deficit/hyperactivity disorder in a nationally representative sample of U.S. children. Environ Health 14:44, 10.1186/s12940-015-0030-y.
Whyatt RM, Barr DB, Camann DE, Kinney PL, Barr JR, Andrews HF, et al. 2003. Contemporary-Use Pesticides in Personal Air Samples during Pregnancy and Blood Samples at Delivery among Urban Minority Mothers and Newborns. Environ Health Perspect 111(5):749–756, PMID: 12727605, 10.1007/s10654-011-9574-5.
Wechsler D. 2003. Wechsler Intelligence Scale for Children – Fourth Edition (WISC-IV) Administration and Scoring Manual. San Antonio, TX: Harcourt Assessment Incorporated.
Woodcock R, Muñoz-Sandoval A. 1990. Woodcock-Johnson Psycho-Educational Battery – Revised. Itasca, IL: Riverside Publishing.
Yang W, Carmichael SL, Roberts EM, Kegley SE, Padula AM, English PB, et al. 2014. Residential agricultural pesticide exposures and risk of neural tube defects and orofacial clefts among offspring in the San Joaquin Valley of California. Am J Epidemiol 179:740–748.
Berkowitz GS, Obel J, Deych E, Lapinski R, Godbold J, Liu Z, et al. 2003. Exposure to indoor pesticides during pregnancy in a multiethnic, urban cohort. Environ Health Perspect 111:79–84.
Castorina R, Bradman A, Fenster L, Barr DB, Bravo R, Vedar MG, et al. 2010. Comparison of current-use pesticide and other toxicant urinary metabolite levels among pregnant women in the CHAMACOS cohort and NHANES. Environ Health Perspect 118:856–863.
Kalkbrenner AE, Daniels JL, Chen JC, Poole C, Emch M, Morrissey J. 2010. Perinatal exposure to hazardous air pollutants and autism spectrum disorders at age 8. Epidemiology 21(5):631–641.
Lu C, Knutson DE, Fisker-Andersen J, Fenske RA. 2001. Biological monitoring survey of organophosphorus pesticide exposure among pre-school children in the Seattle metropolitan area. Environ Health Perspect 109:299–303.
Menezes-Filho JA, Novaes Cde O, Moreira JC, Sarcinelli PN, Mergler D. 2011. Elevated manganese and cognitive performance in school-aged children and their mothers. Environ Res 111:156–163.
U.S. EPA (U.S. Environmental Protection Agency). 2002. Revised OP (Organophosphate) Cumulative Risk Assessment: Volume I. Washington, DC:U.S. EPA.
Wang A, Costello S, Cockburn M, Zhang X, Bronstein J, Ritz B. 2011. Parkinson?s disease risk from ambient exposure to pesticides. Eur J Epidemiol 26(7):547–555, 10.1007/s10654-011-9574-5.