Research July 2017 | Volume 125 | Issue 7
Pesticide Exposure and Risk of Rheumatoid Arthritis among Licensed Male Pesticide Applicators in the Agricultural Health Study
1Occupational and Environmental Health Branch, Public Health Institute, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil
2Epidemiology Branch, National Institute of Environmental Health Sciences, National Institutes of Health, Department of Health and Human Services,
3Occupational and Environmental Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Department of Health and Human Services, Bethesda, Maryland, USA
PDF Version (161 KB)
- The occupation of farming has been associated with rheumatoid arthritis (RA); pesticides may account for this association, but there are few studies.
- We investigated associations between RA and use of pesticides in the Agricultural Health Study.
- The study sample was drawn from male pesticide applicators enrolled in 1993–1997 who provided questionnaire data at baseline and at least once during follow-up (over a median 18 y; interquartile range 16–19). Incident RA cases (n=220), confirmed by physicians or by self-reported use of disease-modifying antirheumatic drugs, were compared with noncases (n=26,134) who did not report RA. Odds ratios (ORs) and 95% confidence intervals (CIs) were estimated using logistic regression, adjusting for enrollment age, state, smoking pack-years, and education. We evaluated the association of RA with the use of 46 pesticides and across 4 levels (never use and tertiles) of lifetime days of use for 16 pesticides with OR≥1.2 for ever use.
- Incident RA was associated with ever use of fonofos (OR = 1.70; 95% CI: 1.22, 2.37), carbaryl (OR = 1.51; 95% CI: 1.03, 2.23), and chlorimuron ethyl (OR = 1.45; 95% CI: 1.01, 2.07) compared with never use. Statistically significant exposure–response trends in association with RA were observed for lifetime days of use of atrazine [ORtertile3= 1.62 (95% CI: 1.09, 2.40); ptrend=0.01] and toxaphene [ORtertile3= 2.42 (95% CI: 1.03, 5.68); ptrend=0.02]. Exposure–response was nonlinear for fonofos [ORtertile1= 2.27 (95% CI: 1.44, 3.57); ORtertile2= 0.98 (95% CI: 0.54, 1.80); ORtertile3= 2.10 (95% CI: 1.32, 3.36); ptrend=0.005] and suggestive for carbaryl (ptrend=0.053).
- Our results provide novel evidence of associations between exposure to some pesticides and RA in male farmers. https://doi.org/10.1289/EHP1013
Received: 24 August 2016
Revised: 19 January 2017
Accepted: 13 February 2017
Published: 14 July 2017
Address correspondence to C.G. Parks, Epidemiology Branch, A3-05, National Institite of Environmental Health Sciences, National Institutes of Health, PO Box 12233, Research Triangle Park, NC 27599 USA; Telephone: (919) 541-2577; Email: Parks1@mail.nih.gov
Supplemental Material is available online (https://doi.org/10.1289/EHP1013).
The authors declare that they have no actual or potential competing financial interests.
Note to readers with disabilities: EHP strives to ensure that all journal content is accessible to all readers. However, some figures and Supplemental Material published in EHP articles may not conform to 508 standards due to the complexity of the information being presented. If you need assistance accessing journal content, please contact firstname.lastname@example.org. Our staff will work with you to assess and meet your accessibility needs within 3 working days.
Supplemental Material PDF (210 KB)
Note to readers with disabilities: EHP has provided a 508-conformant table of contents summarizing the Supplemental Material for this article (see below) so readers with disabilities may determine whether they wish to access the full, nonconformant Supplemental Material. If you need assistance accessing journal content, please contact email@example.com. Our staff will work with you to assess and meet your accessibility needs within 3 working days.
Supplemental Table of Contents PDF (88 KB)
Rheumatoid arthritis (RA) is a systemic autoimmune inflammatory disease (McInnes and Schett 2011) that affects 0.5–1% of the world’s adult population (Alamanos and Drosos 2005), with higher rates in North America and northern Europe (Helmick et al. 2008; Tobón et al. 2010; Widdifield et al. 2014). Because of its chronic nature and elevated mortality, RA is a disease of great personal, socioeconomic, and public health concern (Cross et al. 2014; Firestein 2003).
RA is a multifactorial disease that is influenced by genetic and environmental risk factors (Silman and Pearson 2002). Established environmental risk factors for RA include cigarette smoking and silica exposure (Alamanos and Drosos 2005; Mostafalou and Abdollahi 2013; Tobón et al. 2010). Several studies have suggested farming, pesticides, or both as a risk factor for RA (Gold et al. 2007; Li et al. 2008; Lundberg et al. 1994; Milham 1988; Olsson et al. 2004). Although pesticides have not been studied in animal models of RA, diverse immunotoxic effects seen across different types of pesticides support a range of plausible mechanisms (Corsini et al. 2013; Mokarizadeh et al. 2015) consistent with the hypothesis that pesticides may contribute to RA. However, only a few studies have examined types of pesticides in relation to RA.
Parks et al. (2011) observed a higher risk of RA and a related disease, systemic lupus erythematosus, among women who self-reported use of insecticides in the Women’s Health Initiative Observational Study, with a greater risk in women reporting a farming background. In a sample of the U.S. population in the National Health and Nutrition Examination Survey, higher serum levels of organochlorine insecticides were associated with self-reported arthritis, including RA (Lee et al. 2007). In the Agricultural Health Study (AHS), a prospective study of licensed pesticide applicators from Iowa and North Carolina, an early analysis did not show any statistically significant associations between specific pesticides and RA among female spouses (cases: n=135; controls: n=675) of licensed pesticide farmer-applicators (De Roos et al. 2005). However, a more recent analysis of the spouses including a larger number of incident cases (n=275) found a statistically significant association of incident RA with the fungicides maneb/mancozeb and the herbicide glyphosate in addition to elevated association with dichlorodiphenyltrichloroethane (DDT) (Parks et al. 2016).
Because of the higher prevalence of RA in females, with few exceptions (Gold et al. 2007; Lee et al. 2007; Olsson et al. 2004), studies of farming or pesticides and RA have focused on women. However, in the AHS, most farming activities, including pesticide handling and application, are performed by licensed pesticide applicators, of whom 97% are men (Blair et al. 2005). The present study investigated the risk of RA associated with pesticide exposures among male licensed pesticide applicators enrolled in the AHS. Given the limited prior research in humans (and none in animal models of RA), no prior hypotheses were specified, and the full range of pesticides was considered in our analyses. To our knowledge, this is the first time that the association of RA with specific pesticides has been investigated in males.
The AHS has been previously described (Alavanja et al. 1996; Alavanja et al. 2003). Between 1993 and 1997 (phase 1), 52,394 private pesticide applicators (most of whom were male farmers) enrolled when they completed a questionnaire (“enrollment questionnaire”) at the licensing site where they sought to obtain or renew their licenses for pesticide application in North Carolina and Iowa. Enrolled farmers were given another questionnaire to complete at home (“take-home questionnaire”). Over 80% of eligible applicators completed the enrollment questionnaire, and 44% of those who completed the enrollment questionnaire returned the take-home questionnaire. The questionnaires (http://aghealth.nih.gov/collaboration/questionnaires.html) collected information on demographics, lifetime pesticide use, and medical history. Participants were asked to complete follow-up questionnaires at three time points [phase 2 (P2: 1999–2003), phase 3 (P3: 2005–2010), and phase 4 (P4: 2013–2015)] to update their exposures and health status, including RA. This study was approved by the relevant institutional review boards.
Individuals were considered eligible to be incident cases if they answered yes to the P3 and/or P4 questionnaire question “Have you ever been diagnosed specifically with rheumatoid arthritis (an autoimmune disease)?” RA was not assessed for male applicators at P2. Prevalent cases reported RA diagnosed at or before the year of enrollment. Because self-report has proven to overestimate the prevalence of RA (Cross et al. 2014; Walitt et al. 2008), we sought additional information to confirm diagnoses. Eligible cases identified through P3 were screened by telephone to confirm their diagnosis and to provide additional information on the disease, including the use of disease-modifying antirheumatic drugs (DMARDs), as previously described (Parks et al. 2016). In the P4 questionnaire, all self-reported cases were asked about current or past use of DMARDs. Previous validation studies showed that self-reported RA together with DMARD use dramatically increases positive and negative predictive values (Formica et al. 2010).
Inclusion Criteria for Cases and Noncases
Of 51,036 enrolled male pesticide applicators, 20,733 (40.5%) did not return questionnaires in P3 or P4 (see Figure S1). Among the remaining 30,303 applicators, we excluded 503 with missing data on RA or age at diagnosis, 33 with childhood RA, 537 who reported RA at enrollment or at P3 that they refuted in a later questionnaire, and 580 prevalent RA cases (self-reported, including 152 confirmed by DMARD use or physician validation). Of the remaining 28,650 participants eligible for the study sample, 906 individuals were eligible for further RA screening questions (99 were eligible owing to a related autoimmune disease, lupus): of these, 84 could not be reached or had missing data on RA status on the screening interview, 211 reported not having RA, and 200 were considered unlikely to have RA because they did not report use of DMARDs or steroids for RA or being diagnosed or seen by a rheumatologist, leaving 411 identified as potential RA cases. Excluding those missing data on potential confounders, 220 cases were classified as “probable RA” for primary analyses of cases confirmed by DMARD use, and 160 were classified as “possible RA” for sensitivity analyses if they took steroids for RA or were diagnosed or seen by a rheumatologist. Of 27,744 potential noncases (no reported RA), 26,134 had complete covariate data.
Pesticide Exposure Assessment
In the enrollment questionnaire, applicators provided information on ever use of 50 pesticides and detailed data on duration (years) and frequency (days per year) of use for 22 of those pesticides. Duration and frequency of use for the remaining 28 pesticides was collected in the take-home questionnaire. Cumulative lifetime days of use was calculated by multiplying the midpoint of each duration and frequency stratum. This product was multiplied by an intensity score to estimate intensity-weighted lifetime days of pesticide use, a measure that takes into account information on repair of pesticide application equipment, use of personal protective equipment, and application methods (Coble et al. 2011). Based on the distribution of lifetime days (LD) of use (or intensity-weighted LD) in cases, each pesticide was grouped by tertiles and compared with never users. We also evaluated lifetime number of pesticides used (0–5, 6–9, 10–13, ≥14).
Odds ratios (ORs) and 95% confidence intervals (95% CIs) calculated by logistic regression were used to estimate the association of RA with pesticide exposure variables. Covariates were identified based on observed or hypothesized associations with RA and overall pesticide use, and confirmed through selection by stepwise regression. The final model included age at enrollment (continuous), state of enrollment (NC or IA), number of cigarette packs-years (none, <5, 5–18, >18), and education (≤high school degree and >high school). Current body mass index (BMI) and alcohol use at enrollment were considered as potential confounders but were not independently associated with RA and so were not included in final model. Other characterizations of age (e.g., age squared) did not materially affect results.
Adjusted ORs estimated the association of incident RA with ever use of each of 46 pesticides (with at least five exposed cases) and the lifetime number of pesticides used. Exposure–response analyses examined associations across tertiles of lifetime days of pesticide use and intensity-weighted lifetime days compared with no use for those pesticides with ≥20 exposed cases and an OR≥1.2 for ever use. Statistical tests for trend were performed on the categorical (4 levels) exposure variables, comparing tertiles with no use.
In exploratory analyses, models for ever use were stratified by smoking and by age (<50 and ≥50 y), which may be related to the number and types of pesticides used. Tests for interaction and analyses of lifetime days within strata were not performed owing to small numbers and lack of prior hypotheses. We also explored the impact of case definition on results for ever use and lifetime days by including possible incident cases who reported steroid use or being seen or treated by a rheumatologist for RA (160 possible+220 probable cases; total n=380).
RA cases tended to be older at enrollment (Table 1), and after adjusting for age, incident RA was associated with NC residence [OR = 1.60 ( 95% CI: 1.22, 2.10)], current smoking [OR = 1.67 (95% CI: 1.01, 2.76)], and pack-years [e.g., OR = 1.95 (95% CI: 1.39, 2.75), >18 pack-years vs. never].
|Noncases (n=26,134)||Incident RA cases (n=220)|
|Characteristic||n (%)||n (%)||ORa (95% CI)|
|Age at enrollment (years)|
|<40||7,858 (30)||39 (18)||Reference|
|40–49||7,956 (30)||72 (33)||1.82 (1.23, 2.70)|
|50–59||6,035 (23)||75 (34)||2.50 (1.70, 3.69)|
|≥60||4,285 (16)||34 (15)||1.60 (1.01, 2.54)|
|Body mass index (kg/m2)|
|<25||6,157 (26)||41 (20)||Reference|
|25 to <30||12,353 (51)||114 (55)||1.36 (0.95, 1.94)|
|≥30||5,551 (23)||51 (25)||1.36 (0.90, 2.06)|
|State of enrollment|
|Iowa||17,972 (69)||125 (57)||Reference|
|North Carolina||8,162 (31)||95 (43)||1.60 (1.22, 2.10)|
|White||25,454 (97)||212 (96)||Reference|
|Nonwhite||659 (3)||8 (4)||1.39 (0.69, 2.84)|
|High school or less||14,117 (54)||137 (62)||Reference|
|More than high school||12,017 (46)||83 (38)||0.77 (0.58, 1.02)|
|Smoking status at enrollment|
|Never smoker||15,046 (58)||100 (45)||Reference|
|Past smoker||7,637 (29)||74 (34)||1.30 (0.96, 1.78)|
|Current smoker||3,392 (13)||46 (21)||2.04 (1.44, 2.90)|
|Pack-years of cigarette smoked|
|None||15,046 (58)||100 (45)||Reference|
|<5||3,460 (13)||23 (10)||0.98 (0.62, 1.54)|
|5–18||3,760 (15)||42 (19)||1.60 (1.11, 2.30)|
|>18||3,868 (15)||55 (25)||1.95 (1.39, 2.75)|
|Alcohol consumption (times/wk)|
|None||8,143 (32)||82 (38)||Reference|
|≤1||11,778 (47)||84 (39)||0.78 (0.57, 1.07)|
|>1||5,331 (21)||49 (23)||1.01 (0.70, 1.44)|
Note: CI, confidence interval; OR, odds ratio.
aAdjusted for continuous age.
Incident RA was associated with ever use of the organophosphate insecticide fonofos [OR = 1.70 (95% CI: 1.22, 2.37)], the carbamate insecticide carbaryl [OR = 1.51 (95% CI: 1.03, 2.23)], and the sulfonylurea herbicide chlorimuron ethyl [OR = 1.45 (95% CI: 1.01, 2.07)] compared with never use (Table 2). Although confidence limits included the null, elevated odds ratios (≥1.40) were also observed for the organochlorine insecticides dieldrin [OR = 1.63 (95% CI: 0.77, 3.43)] and toxaphene [OR = 1.44 (95% CI: 0.90, 2.14)], the organophosphate insecticide dichlorvos [OR = 1.40 (95% CI: 0.91, 2.14)], and the fumigant methyl bromide [OR = 1.42 (95% CI: 0.97, 2.08)]. RA was also associated with more pesticides reported [e.g., OR = 1.52 (95% CI: 1.02, 2.32), for ≥14 vs.≤5].
|Noncases (n=26,134)||Incident RA cases (n=220)|
|Pesticide||n (%)||n (%)||ORa (95%CI)|
|Aldrin||2,370 (17)||29 (23)||1.30 (0.82, 2.05)|
|Chlordane||2,590 (19)||36 (28)||1.32 (0.88, 1.98)|
|DDT||2,920 (21)||40 (31)||1.15 (0.75, 1.75)|
|Dieldrin||491 (4)||8 (6)||1.63 (0.77, 3.43)|
|Heptachlor||1,723 (13)||15 (12)||0.88 (0.49, 1.55)|
|Lindane||2,006 (15)||18 (14)||0.96 (0.58, 1.59)|
|Toxaphene||1,487 (11)||23 (18)||1.44 (0.90, 2.29)|
|Chlorpyrifos||11,261 (43)||104 (48)||1.30 (0.99, 1.70)|
|Coumaphos||2,259 (10)||13 (7)||0.70 (0.40, 1.23)|
|Diazinon||2,791 (21)||34 (27)||1.16 (0.77, 1.75)|
|Dichlorvos||2,855 (12)||26 (13)||1.40 (0.91, 2.14)|
|Fonofos||5,739 (24)||58 (29)||1.70 (1.22, 2.37)|
|Malathion||8,983 (66)||87 (67)||1.05 (0.73, 1.53)|
|Parathion||1,022 (8)||11 (8)||0.85 (0.45, 1.60)|
|Phorate||4,345 (32)||40 (31)||1.14 (0.76, 1.70)|
|Terbufos||10,169 (42)||86 (43)||1.24 (0.93, 1.66)|
|Aldicarb||950 (7)||13 (10)||1.08 (0.58, 2.01)|
|Carbaryl||5,708 (42)||74 (57)||1.51 (1.03, 2.23)|
|Carbofuran||7,066 (29)||63 (32)||1.08 (0.80, 1.46)|
|Permethrin Cb||3,352 (14)||31 (16)||1.17 (0.79, 1.73)|
|Permethrin Pb||3,637 (15)||25 (13)||1.05 (0.68, 1.62)|
|2,4-D||20,436 (79)||169 (79)||1.16 (0.83, 1.64)|
|2,4,5-T||2,612 (19)||29 (22)||1.11 (0.72, 1.71)|
|2,4,5-TP||675 (5)||7 (5)||1.00 (0.46, 2.15)|
|Alachlor||13,695 (56)||122 (60)||1.26 (0.95, 1.68)|
|Atrazine||19,320 (74)||166 (76)||1.29 (0.94, 1.79)|
|Butylate||3,912 (29)||35 (27)||1.04 (0.70, 1.54)|
|Chlorimuron ethyl||4,470 (33)||51 (39)||1.45 (1.01, 2.07)|
|Cyanazine||10,960 (45)||76 (38)||0.96 (0.69, 1.31)|
|Dicamba||13,402 (55)||92 (46)||0.90 (0.65, 1.25)|
|EPTC||5,181 (22)||24 (12)||0.62 (0.40, 0.96)|
|Glyphosate||20,060 (77)||166 (76)||0.90 (0.65, 1.24)|
|Imazethapyr||11,292 (45)||86 (44)||1.32 (0.94, 1.86)|
|Metolachlor||11,909 (49)||85 (43)||0.90 (0.67,1.20)|
|Metribuzin||5,506 (40)||48 (37)||1.08 (0.73, 1.59)|
|Paraquat||2,131 (16)||24 (18)||0.92 (0.57, 1.49)|
|Pendimethalin||5,183 (38)||49 (37)||0.98 (0.69, 1.40)|
|Trifluralin||13,590 (56)||112 (56)||1.23 (0.92, 1.66)|
|Benomyl||979 (7)||9 (7)||0.64 (0.32, 1.31)|
|Captan||2,933 (12)||21 (11)||0.90 (0.57, 1.43)|
|Chlorothalonil||1,861 (7)||24 (11)||1.27 (0.81, 2.01)|
|Maneb||1,031 (8)||14 (11)||0.97 (0.53, 1.78)|
|Metalaxyl||2,511 (18)||35 (27)||1.20 (0.77, 1.88)|
|80/20 mix||595 (4)||5 (4)||0.76 (0.31, 1.89)|
|Methylbromide||3,825 (15)||53 (24)||1.42 (0.97, 2.08)|
|Petroleum oil||3,028 (22)||19 (14)||0.65 (0.40, 1.06)|
|Total use reported|
|0 – 5 pesticides||5,741 (22)||44 (20)||Reference|
|6 – 9||7,520 (29)||68 (31)||1.41 (0.96, 2.09)|
|10 – 13||6,274 (24)||47 (21)||1.26 (0.82, 1.93)|
|≥14||6,494 (25)||60 (27)||1.54 (1.02, 2.32)|
Note: 2,4-D, 2,4-dichlorophenoxyacetic acid; 2,4,5-T, 2,4,5-Trichlorophenoxyacetic acid; 2,4,5-TP, 2- propionic acid (fenoprop); 80/20 mix, carbon tetrachloride/carbon disulfide; CI, confidence interval; DDT, dichlorodiphenyltrichloroethane; EPTC, S-ethyl dipropylthiocarbamate; OR, odds ratio.
aAdjusted for age, state of enrollment, pack-years smoking, and education.
bPermethrin C: used on crops; permethrin P: used on poultry and livestock.
For lifetime days of pesticide use (Table 3), positive exposure–response relationships with incident RA were observed for toxaphene [ORtertile3=2.42 (95% CI: 1.03, 5.68); ptrend=0.02] and for the herbicide atrazine [ORtertile3=1.62 (95% CI: 1.09, 2.40); ptrend=0.01], but not for fonofos [ORtertile1=2.10 (95% CI: 1.32, 3.36), ORtertile3=2.77 (95% CI: 1.44, 3.57)] or chlorimuron ethyl (ptrend=0.10). Other positive associations were observed for which confidence limits excluded the null: the lowest tertile of the herbicide imazethapyr [ORtertile1=1.70 (95% CI: 1.18, 2.55)] and the top tertile of the fungicide chlorothalonil [ORtertile3=2.35 (95% CI: 1.07, 5.14)] compared with never use. Exposure–response patterns were similar for intensity-weighted lifetime days (see Table S1), but we also noted a new trend for the herbicide alachlor [ORtertile3=1.44 (95% CI: 1.00, 2.08); ptrend=0.04] and an association for the top tertile of the fungicide metalaxyl [ORtertile3=1.98 (95% CI: 1.08, 3.64)].
|Noncases (n=26,134)||RA cases (n=220)|
|Insecticides||n (%)||n (%)||ORa (95%CI)|
|Never||11,257 (83)||99 (78)||Reference|
|<20||1,095 (8)||14 (11)||1.38 (0.77, 2.50)|
|≥20 to <24.5||116 (1)||2 (2)||NC|
|≥24.5||1,084 (8)||12 (10)||1.18 (0.63, 2.22)|
|Never||11,035 (81)||93 (72)||Reference|
|<8.75||639 (5)||8 (6)||1.14 (0.54, 2.38)|
|≥8.75 to <20||1,051 (8)||15 (12)||1.41 (0.81, 2.48)|
|≥20||825 (6)||13 (10)||1.48 (0.81, 2.69)|
|Never||12,143 (89)||104 (83)||Reference|
|<24.5||859 (6)||7 (6)||0.82 (0.38, 1.78)|
|≥24.5 to <62.5||392 (3)||9 (7)||2.00 (0.99, 4.04)|
|≥62.5||202 (1)||6 (5)||2.42 (1.03, 5.68)|
|Never||14,678 (57)||111 (52)||Reference|
|<20||3,756 (15)||34 (16)||1.27 (0.86, 1.88)|
|≥20 to <56||3,727 (15)||37 (17)||1.42 (0.97, 2.06)|
|≥56||3,602 (14)||32 (15)||1.25 (0.84, 1.86)|
|Never||21,220 (88)||170 (87)||Reference|
|<24.5||1,052 (4)||9 (5)||1.33 (0.67, 2.63)|
|≥24.5 to <173.25||936 (4)||7 (4)||1.14 (0.53, 2.44)|
|≥173.25||796 (3)||9 (5)||1.76 (0.87, 3.48)|
|Never||18,544 (77)||141 (71)||Reference|
|<20||1,825 (8)||24 (12)||2.27 (1.44, 3.57)|
|≥20 to <56||2,074 (9)||12 (6)||0.98 (0.54, 1.80)|
|≥56||1,725 (7)||22 (11)||2.10 (1.32, 3.36)|
|Never||14,178 (59)||116 (58)||Reference|
|<24.5||3,548 (15)||27 (14)||1.14 (0.74, 1.75)|
|≥24.5 to <63.75||3,464 (14)||29 (15)||1.25 (0.82, 1.89)|
|≥63.75||2954 (12)||28 (14)||1.33 (0.88, 2.03)|
|Never||7,966 (59)||55 (44)||Reference|
|<20||2,339 (17)||23 (18)||1.33 (0.81, 2.18)|
|≥20 to <62.5||1,775 (13)||25 (20)||1.62 (0.96, 2.71)|
|≥62.5||1,434 (11)||23 (18)||1.61 (0.90, 2.86)|
|Never||10,698 (45)||80 (40)||Reference|
|<38.75||5,931 (25)||50 (25)||1.25 (0.87, 1.79)|
|≥38.75 to <108.5||3,001 (13)||24 (12)||1.17 (0.74, 1.86)|
|≥108.5||4,362 (18)||44 (22)||1.37 (0.94, 1.98)|
|Never||6,685 (26)||52 (24)||Reference|
|<50.75||8,199 (32)||60 (28)||1.10 (0.75, 1.62)|
|≥50.75 to <178.5||6,002 (23)||49 (23)||1.28 (0.85, 1.91)|
|≥178.5||4,914 (19)||54 (25)||1.62 (1.09, 2.40)|
|Never||9,217 (68)||81 (63)||Reference|
|<8.75||1,116 (8)||12 (9)||1.47 (0.79, 2.72)|
|≥8.75 to <24.5||1,881 (14)||20 (16)||1.34 (0.82, 2.21)|
|≥24.5||1,397 (10)||16 (12)||1.42 (0.83, 2.45)|
|Never||12,842 (54)||111 (57)||Reference|
|<20||4,910 (21)||50 (26)||1.74 (1.18, 2.55)|
|≥20 to <24.5||599 (3)||5 (3)||1.42 (0.56, 3.59)|
|≥24.5||5,582 (23)||30 (15)||0.97 (0.62, 1.52)|
|Never||10,781 (45)||89 (45)||Reference|
|<38.75||4,520 (19)||35 (18)||1.11 (0.75, 1.66)|
|≥38.75 to <108.5||3,572 (15)||31 (16)||1.34 (0.87, 2.06)|
|≥108.5||5,204 (22)||43 (22)||1.25 (0.85, 1.84)|
|Never||24,052 (93)||194 (91)||Reference|
|<54.25||862 (3)||7 (3)||0.82 (0.38, 1.77)|
|≥54.25 to <200||572 (2)||6 (3)||1.02 (0.44, 2.35)|
|≥200||304 (1)||7 (3)||2.35 (1.07, 5.14)|
|Never||11,164 (83)||96 (74)||Reference|
|<8||714 (5)||7 (5)||1.05 (0.48, 2.28)|
|≥8 to <50.75||991 (7)||14 (11)||1.20 (0.64, 2.23)|
|≥50.75||616 (5)||13 (10)||1.76 (0.91, 3.41)|
|Never||22,116 (86)||166 (76)||Reference|
|<12.25||1,004 (4)||16 (7)||1.74 (1.00, 3.02)|
|≥12.25 to <54.25||1,469 (6)||17 (8)||1.18 (0.68, 2.05)|
|≥54.25||1,203 (5)||19 (9)||1.54 (0.90, 2.63)|
Note: CI, confidence interval; OR, odds ratio; NC, not calculated.
aAdjusted for age, state of enrollment, pack-years smoking, and education.
Most pesticide and RA associations observed in the overall sample did not substantially differ by smoking status (see Table S2), but some new statistically positive associations were seen in never smokers, including chlordane, DDT, dieldrin, the herbicide 2,4,5-trichlorophenoxyacetic acid (2,4,5-T), and the fungicide chlorothalonil. No differences were found for total number of pesticides used, but we did not examine exposure–response for individual pesticides owing to the smaller sample size in these stratified analyses.
Most associations between RA and ever use of specific pesticides were similar by age (<50 y and ≥50 y) or were stronger in older participants (see Table S3) except for two new associations observed only in older participants: dichlorvos [OR = 1.82 (95% CI: 1.04, 3.20) and imazethapyr [OR = 1.85 (95% CI: 1.11, 3.05). A higher total number of pesticides (≥14 vs.≤5) was associated with RA in older [≥50 y, OR=2.16 (95% CI: 1.19, 3.90)] but not younger [<50 y, OR = 0.99 (95% CI: 0.55, 1.78)] participants. When limited to chemicals with suggestive associations (OR≥1.2 for ever use), less of an age difference was seen [e.g., ≥7 vs.≤2 pesticides: ≥50 y, OR=2.57 (95% CI: 1.45, 4.56); <50 y, OR = 1.97 (95% CI: 1.10, 3.54)].
In a sensitivity analysis including an additional 160 possible incident cases (total n=380 probable+possible cases; see Tables S4–S6), ORs with RA were increased with age [OR = 2.92 (95% CI: 2.07, 4.11), ≥ 60 y vs. <40 y] and for nonwhites [OR=2.07 (95% CI: 1.27, 3.17)]. Compared with the primary analysis of probable cases, the association was confirmed for fonofos, whereas the OR was reduced for carbaryl and increased for chlorothalonil). Dose–response results were similar for atrazine, but a new exposure–response trend was noted for phorate, and our findings confirmed trends for alachlor and chlorothalonil as well as associations with the highest levels of toxaphene and metalaxyl.
In this prospective cohort of male farmers, we observed increased associations of incident RA, and exposure–response trends, for several pesticides. To the best of our knowledge, this is the first epidemiologic study to investigate the development of RA in men in relation to many types of pesticides. Given the limited prior research, our results provide novel evidence on the role of pesticides as environmental risk factors for RA, including robust findings of dose–response associations of RA with the commonly used herbicide atrazine and an organochlorine insecticide, toxaphene. Along with other associations for lifetime use of the insecticides fonofos and carbaryl and the herbicide chlorimuron ethyl, these results warrant replication in other samples.
The AHS offers a unique opportunity to examine the development of RA in a large cohort of farmers with a detailed lifetime history of pesticide use, which is infeasible for population-based case–control studies. However, statistical power is limited for analyses of uncommon outcomes such as RA, particularly when considering exposure–response data on chemicals no longer on the market in the mid-1990s that were covered in the take-home questionnaire completed by only 44% of the sample (e.g., toxaphene).
We observed expected associations of RA with older age and smoking (Karlson and Deane 2012; Linos et al. 1980; Symmons et al. 1994; Uhlig et al. 1998). Smoking-stratified analyses showed no substantial differences in primary results for ever use and total number used (except for some new associations for organochlorines). However, most associations were stronger or only apparent in older participants (e.g., dichlorvos, imazehapyr, chlorimuron ethyl). Explanations for this outcome could include differences in types of pesticide exposures in older cohorts, age at exposure, and opportunities over time for higher doses or multiple exposures. Genetic predisposition may have a lesser impact on older-onset RA (Scott et al. 2013); therefore, it seems plausible that pesticide associations with RA might simply be more apparent in the absence of strong risk factors such as smoking or genotypes associated with early-onset disease.
In these analyses of incident disease, exposures were reported before RA diagnosis, reducing the likelihood of recall bias. Nondifferential exposure misclassification is a possible limitation. However, self-reported pesticide use is considered accurate in the AHS (Hoppin et al. 2002), and detailed data were collected on application frequency, duration, and practices that might influence exposure levels. Indeed, exposure–response trends for lifetime days were observed for cumulative days of use for some pesticides, and intensity-weighted results were not notably different. However, if short-term, intense exposure were more relevant to immune effects, these metrics may not have adequately captured the relevant dose. Future analyses could explore potential determinants of higher-intensity exposures based on average days used per year and on other types of data. Although farmers may experience similar background exposures to the general population from residential and other sources, agricultural pesticides are typically more concentrated chemicals, and most farmers used multiple types. For each comparison, the “unexposed” referent group included those with background exposures and exposures to other pesticides, which could potentially influence he appearance or strength of individual effect estimates. Furthermore, our analyses focused on lifetime exposures reported at baseline but did not include those more proximal to RA diagnoses. The timing of and required dose for pesticide effects on RA pathogenesis is not known, and the relevant window could extend from early-life immune development through processes involved in the generation of preclinical autoimmunity and the development of symptomatic disease.
Early-life pesticide exposures are likely in the cohort but may not be captured by self-reported lifetime exposure questions. Moreover, most participants experienced exposures to several different pesticides over the course of adult use; this is part of a broader picture of multiple pesticide exposures in the AHS, in addition to other potentially relevant exposures, such as organic and inorganic dusts, solvents, and heavy metals. Methods are being developed to help identify critical components of exposure mixtures or potential synergy across correlated exposures (Sun et al, 2013), but they are not robust when the number of cases is small and the number of potential exposures is great. In our study, RA was associated with increasing lifetime number of pesticides used, but the small number of RA cases using individual combinations of chemicals limits our ability to evaluate multiple exposures. Moreover, because we did not identify strong correlations between pesticides, these were not considered as potential confounders. Other occupational exposures, both on and off the farm, may also be risk factors for RA but were not considered as confounders or modifiers for pesticide associations in this analysis. For example, silica dust is an established risk factor for RA (Parks et al. 2003; Stolt et al. 2005); soil silica levels vary across the study region (Stopford and Stopford 1995). Future analyses are planned to explore these questions in the cohort.
Self-reported RA is nonspecific, and many AHS participants who initially reported RA later refuted it in subsequent follow-up and case confirmation. For those with available data (not shown), a refuted RA diagnosis from phase 3 showed high reliability, that is to say, 155 of 180 (86%) repeated their negative response for RA in phase 4. Along with unconfirmed RA cases, these were excluded from the analysis sample, although their potential influence as false negatives would have been unsubstantial compared with their influence as false positives. In contrast, most self-reported RA cases at phase 3 who used DMARDs confirmed their self-report at phase 4, that is to say, 91 of 98 cases (93%). Of 906 potential RA cases eligible for screening questions about their diagnosis and medications, fewer than half (n=411) were considered as possible cases, and only half of these were classified as probable cases confirmed by DMARDs use. Although identifying RA cases based on self-report and use of DMARDs has high specificity (Formica et al. 2010; Walitt et al. 2008), this definition may lack sensitivity for males and for older individuals, who may be less likely to use DMARDs (Schmajuk et al. 2011; Solomon et al. 2012); therefore, it could have limited generalizability and missed associations if use of specialized medical treatment is related to pesticide use. Analyses adding in the possible RA cases, who reported use of steroids for RA or being diagnosed or treated by a rheumatologist, yielded similar results for many, but not all, pesticides. We found increased associations for RA with older age and with nonwhite race for the more inclusive group of RA cases (probable+possible). This finding confirms the potential influence of biased case ascertainment; therefore, differences in pesticide associations may warrant further consideration.
Loss to follow-up could also have biased the results. Smoking is related to loss to follow-up in AHS participants, but recent analyses show little impact on cancer associations (and a lack of compensation by inverse-probability weighting) when the associations with smoking were not extreme (Rinsky et al. 2017). Smoking was a moderate risk factor for RA in this sample, suggesting limited potential for bias.
Although the immune effects of pesticides are variable, several associations observed in the study suggest plausible biologic mechanisms related to RA pathogenesis. For example, fonofos altered the methylation levels of genes involved in the regulation of the immune response (Zhang et al. 2012). Immunotoxic effects of other organophosphates have been described (Corsini et al. 2013), and a more detailed assessment of updated AHS organophosphate exposures is justified given their widespread use and the changes in types of organophosphates used over time in the cohort and in the general population. Interestingly, carbamates (e.g., carbaryl) and organophosphates share the same main mechanism of toxicity: inhibition of acetylcholinesterase enzyme activity in neuronal and neuromuscular synapses (Fukuto 1990). This ability to inhibit serine hydrolase and protease enzymes, which play an essential role in the immune system, may hold a common explanation for some alterations in the immune function induced by pesticides of both classes (Casale et al. 1992; Galloway and Handy 2003; Guo et al. 2014; Long and Cravatt 2011).
We previously noted an association of DDT use with incident RA in female AHS spouses (Parks et al. 2016). Although incident RA was not significantly associated with DDT in the present study overall, associations with DDT (and other organochlorines) were apparent in never smokers. Other evidence of organochlorine pesticide associations with RA includes elevated serum organochlorines in female RA cases in the National Health and Nutrition Examination Survey (1999–2002) (Lee et al. 2007) and experimental studies that have suggested a role for DDT in inflammatory processes and immune disruption. (Cardenas-Gonzalez et al. 2013; Kim et al. 2004; Dutta et al. 2008). Toxaphene was associated with RA in our analyses, although the estimates were imprecise because of small numbers. Banned in 1990, toxaphene was often used on cotton in the southeastern United States, along with other organochlorines.
We observed an exposure–response trend in the association of incident RA with atrazine, a triazine herbicide for which immunotoxic effects have been observed in experimental animals. In mice, atrazine has been shown to induce apoptosis of splenocytes, whereas leukocytes increased in a dose-dependent manner (Zhang et al. 2011); to elicit a broad inhibitory effect on cell-mediated, humoral, and nonspecific immunity function (Chen et al. 2013); and to significantly decrease natural killer (NK) cell lysis activity in vitro (Rowe et al. 2007; Whalen et al. 2003). The sulfonylurea herbicide chlorimuron ethyl was also associated with RA risk. Sulfonylurea herbicides have been considered to be relatively nontoxic to humans (Mushak and Piver 1992). Another common herbicide, glyphosate, was previously associated with incident RA in AHS female spouses (Parks et al. 2016), but we saw no glyphosate association with incident RA in the male applicators. Glyphosate use in U.S. agriculture has dramatically increased over the past two decades; therefore, updated glyphosate exposure data are needed to properly examine associations with incident RA.
The results from our study provide evidence that exposure to some pesticides may play a role in the development of RA among male farmers regardless of age, smoking, and educational level. Because this is the first study to observe increased RA risk associated with these pesticides among men, our findings need confirmation in other populations.
The authors thank S. Long (Westat) for programming support and for helpful review of manuscript drafts.
This work was supported by the Intramural Research Program of the National Institutes of Health/National Institute of Environmental Health Sciences (Z01-ES049030) and by the National Cancer Institute (Z01-CP010119). A.M. was funded by Coordination for the Improvement of Higher Education Personnel (CAPES), Brazil (grant no. 2478/2015-03).
Alavanja MCR, Samanic C, Dosemeci M, Lubin J, Tarone R, Lynch CF, et al. 2003. Use of agricultural pesticides and prostate cancer risk in the Agricultural Health Study cohort. Am J Epidemiol 157(9):800–814, 10.1093/aje/kwg040.
Alavanja MC, Sandler DP, McMaster SB, Zahm SH, McDonnell CJ, Lynch CF, et al. 1996. The Agricultural Health Study. Environ Health Perspect 104(4):362–369, PMID: 8732939.
Blair A, Sandler DP, Tarone R, Lubin J, Thomas K, Hoppin JA, et al. 2005. Mortality among participants in the Agricultural Health Study. Ann Epidemiol 5(4):279–285, PMID: 15780775, 10.1016/j.annepidem.2004.08.008.
Cardenas-Gonzalez M, Gaspar-Ramirez O, Perez-Vazquez FJ, Alegria-Torres JA, Gonzalez-Amaro R, Perez-Maldonado IN. 2013. p,p′-DDE, a DDT metabolite, induces proinflammatory molecules in human peripheral blood mononuclear cells “in vitro”. Exp Toxicol Pathol 65(5):661–665, 10.1016/j.etp.2012.07.006.
Casale GP, Bavari S, Gold RE, Vitzthum EF. 1992. Inhibition of interleukin-2-stimulated enhancement of human natural killer (NK) cell activity by carbaryl, an anticholinesterase insecticide. Toxicol Lett 63(3):299–311, PMID: 1488778.
Coble J, Thomas KW, Hines CJ, Hoppin JA, Dosemeci M, Curwin B, et al. 2011. An updated algorithm for estimation of pesticide exposure intensity in the Agricultural Health Study. Int J Environ Res public health 8(12):4608–4622, PMID: 22408592, 10.3390/ijerph8124608.
Corsini E, Sokooti M, Galli CL, Moretto A, Colosio C. 2013. Pesticide induced immunotoxicity in humans: a comprehensive review of the existing evidence. Toxicol 307:123–135, 10.1016/j.tox.2012.10.009.
Cross M, Smith E, Hoy D, Carmona L, Wolfe F, Vos T, et al. 2014. The global burden of rheumatoid arthritis: estimates from the Global Burden of Disease 2010 study. Ann Rheum Dis 73(7):1316–1322, PMID: 24550173, 10.1136/annrheumdis-2013-204627.
De Roos AJ, Cooper GS, Alavanja MC, Sandler DP. 2005. Rheumatoid arthritis among women in the Agricultural Health Study: risk associated with farming activities and exposures. Ann Epidemiol 15(10):762–770, PMID: 16257361, 10.1016/j.annepidem.2005.08.001.
Dutta R, Mondal AM, Arora V, Nag TC, Das N. 2008. Immunomodulatory effect of DDT (bis[4-chlorophenyl]-1,1,1-trichloroethane) on complement system and macrophages. Toxicol 252(1):78–85, 10.1016/j.tox.2008.07.063.
Formica MK, McAlindon TE, Lash TL, Demissie S, Rosenberg L. 2010. Validity of self-reported rheumatoid arthritis in a large cohort: results from the Black Women’s Health Study. Arthritis Care Res (Hoboken) 62:235–241.
Fukuto TR. 1990. Mechanism of action of organophosphorus and carbamate insecticides. Environ Health Perspect 87:245–254, PMID: 2176588.
Galloway T, Handy R. 2003. Immunotoxicity of organophosphorous pesticides. Ecotoxicology 12(1–4):345–363, PMID: 12739880.
Helmick CG, Felson DT, Lawrence RC, Gabriel S, Hirsch R, Kwoh CK, et al. 2008. Estimates of the prevalence of arthritis and other rheumatic conditions in the United States: part I. Arthritis Rheum 58(1):15–25, PMID: 18163481, 10.1002/art.23177.
Hoppin JA, Yucel F, Dosemeci M, Sandler DP. 2002. Accuracy of self-reported pesticide use duration information from licensed pesticide applicators in the Agricultural Health Study. J Expo Anal Environ Epidemiol 12(5):313–318, PMID: 12198579, 10.1038/sj.jea.7500232.
Kim JY, Choi CY, Lee KJ, Shin DW, Jung KS, Chung YC, et al. 2004. Induction of inducible nitric oxide synthase and proinflammatory cytokines expression by o,p′-DDT in macrophages. Toxicol Lett 147(3):261–269, PMID: 15104118.
Lee DH, Steffes M, Jacobs DR. 2007. Positive associations of serum concentration of polychlorinated biphenyls or organochlorine pesticides with self-reported arthritis, especially rheumatoid type, in women. Environ Health Perspect 115(6):883–888, PMID: 17589595, 10.1289/ehp.9887.
Li X, Sundquist J, Sundquist K. 2008. Socioeconomic and occupational risk factors for rheumatoid arthritis: a nationwide study based on hospitalizations in Sweden. J Rheumatol 36(6):986–991, PMID: 18464310.
Linos A, Worthington JW, O’Fallon M, Kurland LT. 1980. The epidemiology of rheumatoid arthritis in Rochester, Minnesota: a study of incidence, prevalence, and mortality. Am J Epidemiol 111(1):87–98, 10.1093/oxfordjournals.aje.a112878.
Lundberg I, Alfredsson L, Plato N, Sverdrup B, Klareskog L, Kleinau S. 1994. Occupation, occupational exposure to chemicals and rheumatological disease. A register based cohort study. Scand J Rheumatol 23(6):305–310, PMID: 7801054.
Milham S. Jr
1988. Using multiple cause of death coding in occupational mortality studies. Am J Ind Med 14(3):341–344, 10.1002/ajim.4700140311.
Mokarizadeh A, Faryabi MR, Rezvanfar MA, Abdollahi M. 2015. A comprehensive review of pesticides and the immune dysregulation: mechanisms, evidence and consequences. Toxicol Mech Methods 25(4):258–278, PMID: 25757504, 10.3109/15376516.2015.1020182.
Mushak EW, Piver WT. 1992. Agricultural chemical utilization and human health. Environ Health Perspect 97:269–274, PMID: 1396466.
Olsson AR, Skogh T, Axelson O, Wingren G. 2004. Occupations and exposures in the work environment as determinants for rheumatoid arthritis. Occup Environ Med 61(3):233–238, PMID: 14985518.
Parks CG, Cooper GS, Nylander-French LA, Storm JF, Archer JD. 2003. Assessing exposure to crystalline silica from farm work: a population-based study in the Southeastern United States. Ann Epidemiol 13(5):385–392, PMID: 12821278.
Parks CG, Hoppin JA, De Roos AJ, Costenbader KH, Alavanja MC, Sandler DP. 2016. Rheumatoid arthritis in Agricultural Health Study spouses: associations with pesticides and other farm exposures. Environ Health Perspect 124(11):1728–1734, PMID: 27285288, 10.1289/EHP129.
Parks CG, Walitt BT, Pettinger M, Chen JC, de Roos AJ, Hunt J, et al. 2011. Insecticide use and risk of rheumatoid arthritis and systemic lupus erythematosus in the Women’s Health Initiative Observational Study. Arthritis Care Res (Hoboken) 63(2):184–194, PMID: 20740609, 10.1002/acr.20335.
Rinsky JL, Richardson DB, Wing S, Beard JD, Alavanja M, Beane Freeman LE, et al. 2017. Assessing the potential for bias from non-response to a study follow-up interview: an example from the Agricultural Health Study. Am J Epidemiol. 10.1093/aje/kwx098.
Rowe AM, Brundage KM, Barnett JB. 2007. In vitro atrazine-exposure inhibits human natural killer cell lytic granule release. Toxicol Appl Pharmacol 221(1):179–188, PMID: 17475299, 10.1016/j.taap.2007.01.012.
Schmajuk G, Trivedi AN, Solomon DH, et al. 2011. Receipt of disease-modifying antirheumatic drugs among patients with rheumatoid arthritis in medicare managed care plans. JAMA 305(5):480–486, PMID: 21285425, 10.1001/jama.2011.67.
Scott IC, Seegobin SD, Steer S, et al. 2013. Predicting the risk of rheumatoid arthritis and its age of onset through modelling genetic risk variants with smoking. PLoS Genet 9(9):e1003808 PMID: 24068971, 10.1371/journal.pgen.1003808.
Silman AJ, Pearson JE. 2002. Epidemiology and genetics of rheumatoid arthritis. Arthritis Res 4:S265–S272.
Solomon DH, Ayanian JZ, Yelin E, Shaykevich T, Brookhart MA, Katz JN. 2012. Use of disease-modifying medications for rheumatoid arthritis by race and ethnicity in the National Ambulatory Medical Care Survey. Arthritis Care Res (Hoboken) 64(2):184–189, PMID: 22012868, 10.1002/acr.20674.
Stolt P, Källberg H, Lundberg I, Sjögren B, Klareskog L, Alfredsson L. 2005. Silica exposure is associated with increased risk of developing rheumatoid arthritis: results from the Swedish EIRA study. Ann Rheum Dis 64(4):582–586, PMID: 15319232, 10.1136/ard.2004.022053.
Stopford CM, Stopford W. 1995. Potential for respirable quartz exposure from North Carolina farm soils. Scand J Work Environ Health 21(Suppl 2):44–46, PMID: 8929688.
Sun Z, Tao Y, Li S, Ferguson KK, Meeker JD, Park SK, et al. 2013. Statistical strategies for constructing health risk models with multiple pollutants and their interactions: possible choices and comparisons. Environ Health 12(1):85, PMID: 24093917, 10.1186/1476-069X-12-85.
Symmons DP, Barrett EM, Bankhead CR, Scott DG, Silman AJ. 1994. The incidence of rheumatoid arthritis in the United Kingdom: results from the Norfolk Arthritis Register. Br J Rheumatol 33(8):735–739, PMID: 8055200.
Uhlig T, Kvien TK, Glennas A, Smedstad LM, Forre O. 1998. The incidence and severity of rheumatoid arthritis, results from a county register in Oslo, Norway. J Rheumatol 25(6):1078–1084, PMID: 9632067.
Walitt BT, Constantinescu F, Katz JD, Weinstein A, Wang H, Hernandez RK, et al. 2008. Validation of self-report of rheumatoid arthritis and systemic lupus erythematosus: The Women’s Health Initiative. J Rheumatol 35(5):811–818, PMID: 18398940.
Whalen MM, Loganathan BG, Yamashita N, Saito T. 2003. Immunomodulation of human natural killer cell cytotoxic function by triazine and carbamate pesticides. Chem Biol Interact 145(3):311–319, PMID: 12732457.
Widdifield J, Paterson JM, Bernatsky S, Tu K, Tomlinson G, Kuriya B, et al. 2014. The epidemiology of rheumatoid arthritis in Ontario, Canada. Arthritis Rheumatol 66(4):786–793, 10.1002/art.38306.
Zhang X, Wallace AD, Du P, Kibbe WA, Jafari N, Xie H, et al. 2012. DNA methylation alterations in response to pesticide exposure in vitro. Environ Mol Mutagen 53(7):542–549, PMID: 22847954, 10.1002/em.21718.
EHP is pleased to announce that “The Prevalence of Antibiotic-Resistant Staphylococcus aureus Nasal Carriage among Industrial Hog Operation Workers, Community Residents, and Children Living in Their Households: North Carolina, USA,” published in EHP in the April 2017 issue, has been selected by the Children’s Environmental Health Network (CEHN) as its October 2017 Article of the Month. (more…)
EHP is pleased to announce that “A Difference-in-Differences Approach to Assess the Effect of a Heat Action Plan on Heat-Related Mortality, and Differences in Effectiveness According to Sex, Age, and Socioeconomic Status (Montreal, Quebec)” has been awarded Honorable Mention as one of the Best Environmental Epidemiology Papers published in 2016. (more…)