Litigation-related Research.

Imagesp512-a

statements which we feel compelled to bring to your attention. First and foremost is the authors' failure to mention the source of their funding. This study was funded by lawyers for approximately 70 persons (including all 42 of the "exposed subjects") (2). Rather than being merely passive financiers of the study, the lawyers were actively involved in the design and administration of this study, thus seriously compromising the scientific value.
As a result, Madison et al. failed to make any attempt to adjust for the many biases such as self-selection and financial gain. Many other persons were also in the Crown Point area at the time of the release and met the author's loose definition of "exposed." Such persons were never contacted to determine their health status (and hence, whether "exposure" or litigation was the more important factor). The Cooper Landing "control" group was equally biased, but in the opposite direction, i.e., this group was significantly healthier than the Cooper Landing community as a whole. Of 600 surveys mailed to the residents of Cooper Landing, only 30 were initially returned (3). Eventually, 65 persons completed surveys, but only 29 showed up to have blood samples drawn (4). The authors failed to mention that they performed a follow-up on nonrespondents in the Cooper Landing area to determine if the volunteers they used were significantly healthier than the population at large (5). This follow-up survey clearly demonstrated that the 29 Cooper Landing volunteers were not at all representative of the Cooper Landing community. A number of the nonrespondents had conditions that Broughton has stated previously affect Tal levels and auto-antibodies (6).
Other major differences also existed between the exposed subjects and the control group. The Cooper Landing controls were significantly older, did not contain a single child (versus 15 children in the "exposed" group), and the formaldehyde antibodies of the control group were measured with a different microtiter reader than was used for the exposed subjects (7). The questionnaires and blood were obtained from the control group during a short period of time during the summer of 1989, whereas the questionnaires and the respiratory infections are more prevalent (8). Different questionnaires were administered via different methods to the two groups. Most of the flaws and inadequacies in the design and administration of the study can be attributed to the rather unorthodox involvement in the study of the plaintiffs' lawyers and a marketing research firm (9). There was no written protocol (10), and the questionnaires were never validated (11) and contained poorly designed and ambiguous questions. Inquiries from respondents about ambiguities were handled in varying and inconsistent ways by the market survey firm and legal assistants, as they were given no instructions as to how to handle such inquiries (12). As originally planned, the Cooper Landing controls were to be randomly selected, matched for age and sex, and administered an inhome survey by trained interviewers (13). These protocol requirements were all subsequently vetoed by the plaintiffs' attorneys (14).
The poor quality of the data is clearly demonstrated by a comparison of the subjects' responses to prior questionnaires they had completed. Regarding potential formaldehyde exposure, 10 of the original 54 exposed subjects gave inconsistent answers regarding their smoking status, and 13 exposed subjects gave inconsistent answers as to whether they had ever lived in a mobile home. Madison  The authors' definition of "exposure" is also extremely ambiguous. When Madison et al. originally presented these data during the litigation, they included 50 persons in the exposed group for whom they had blood test results (49 with F-HSA antibody results) (16). During the course of their deposition testimony, the authors learned that as many as 13 of their original 50 "exposed" subjects were either away from Crown Point (one was unborn) at the time of the release or did not even move into Crown Point until more than a year after the incident (17). The authors have excluded eight persons (presumably because they were trying to exclude such "unexposed" persons). However, they nowhere discuss how they excluded such persons or how "exposure" was defined, and it appears that at least five "unexposed" persons are still included in the group of exposed subjects.
The formaldehyde and trimethylamine (TMA) data cited are equally misleading. The "exposure" data the authors presented for the first 48 hours (0.1-5.0 ppm) was based not upon measurements but rather a dispersion model performed three-and-a-half years after the release by an "expert" retained by plaintiffs' lawyers (18). We found major flaws in this model and the assumptions used for it. Most important, it did not fit the real data that were available. For example, the model predicted that subjects were exposed to mean levels of 2.0 to 5.0 ppm (with peaks as high as 500 ppm) for nearly 24 hours; however, few subjects reported any odor or discomfort during this period. Actual measurements made hours after the release by various independent labs contacted by the railroad strongly suggest the inaccuracy of the model-not a single formaldehyde measurement out of more than 100 exceeded 0.1 ppm. The authors report that TMA levels exceeded 0.2 ppm. This is completely untrue. Of over 100 TMA measurements taken indoors in the days after the release, the majority were negative, and none exceed 0.2 ppm. All outdoor measurements other than two taken in the plume itselfwere negative (19).
The clinical laboratory techniques used also were a source of error which render the data unreliable and make any comparison uninterpretable. The difference between the different microtiters used on the two groups is an obvious confounding factor (20). None of the Crown Point plaintiffs tested on the new machine were positive, whereas 58% of the plaintiffs tested on the old machine were positive, a clearly significant difference: 1-.--HSA samples contained two-and-a-half times as much HSA as the pure HSA blanks (21). None of these subjects was given more than one blood test to assess the reproducibility of any of these tests. We learned through the California Department of Health Services that blood samples submitted by a Michigan doctor to this laboratory for formaldehyde antibody testing and split into duplicates yielded inconsistent results (22). We, therefore, as part of the litigation investigation, undertook a doubleblind study to determine the reproducibility of several tests including the autoimmune antibody tests. Samples of blood were drawn from 20 persons and split into triplicate and submitted blindly to Broughton's laboratory. Of the 20 blood samples submitted, eight (40%) yielded autoimmune antibody results which did not agree among the split samples (23). This inconsistency has been confirmed by researchers at the University of Washington, who also submitted split samples for testing of immunologic parameters (24). These split samples also yielded inconsistent results, particularly with regard to the lymphocyte subsets (Tal positive cells) for which the correlation coefficient was only 0.11, with greater than 50% discordance relative to the laboratory's reference ranges. The lack of reproducibility in the autoimmune antibody test was further demonstrated by the authors' own data from the Crown Point subjects and Cooper Landing controls. Most incredible, however, is the authors' failure to report that three of the Cooper Landing Controls were initially determined to be positive for anti-parietal cell antibodies, but were subsequently changed to negative based upon retesting of the very same sera only two days later (25). Also, two anti-smooth muscle antibody results from Cooper Landing were initially reported as positive but switched to negative upon reanalysis only two days later (even though one was initially a "strong positive") (26). Even more disturbing is the fact that only one of the initial negative results in the control group was retested two days later; it turned out to be positive upon reanalysis, but the authors chose to report it as negative (27)! When confronted about this under oath, Broughton testified that he would either have to discard these data or at a minimum report the changing of results prior to publication (28). For some reason, he has chosen to do neither. As recently as October 1989, Broughton was cited by the state of California for failing to run negative quality controls despite his previous representations to state inspectors that he would begin doing so (29). [The authors' statement that "quality assurance was performed by positive and negative controls run simultaneously with the unknown samples" appears untrue. In fact, Broughton admitted under oath that no negative quality control is performed on the Tal and formaldehyde tests (30)]. Moreover, an unlicensed technician performed the formaldehyde antibody test in violation of state regulations (31). In addition, Dr. Broughton testified under oath that he disagreed with a number of interpretations made by his own lab technician and that other of the formaldehyde antibody results were suspect and should have been rerun (32). It is disconcerting that the author would publish these data having acknowledged these problems.
The authors' failure to adjust for multiple endpoint analysis is also very misleading. More than 25 blood tests were run on each exposed subject, most of which were normal. Madison testified under oath that multiple endpoint analysis should have been done; moreover, when appropriate adjustments for multiple comparisons were made by Madison, none of the blood tests were significantly different between the two groups (33).
Another gross misstatement is the authors' assertion that "subjects were referred for diagnostic testing by their physician." Broughton testified that the testing was ordered by the plaintiffs' attorneys in consultation with him (34). Another false statement in the article is that cell viability was 90% by trypan blue exclusion. Broughton testified under oath that his laboratory no longer performs cell viability testing (35) (even though it is in the written protocol he provides to state inspectors) (36).
Ultimately, one of the most glaring inconsistencies is the authors' failure to explain why only 4 of the 50 plaintiffs tested showed evidence of exposure to formaldehyde prior to 1987 (i.e., IgG) when the incident occurred in February 1986. In other words, their own tests showed that only 4 of the 50 "exposed" subjects were actually exposed to formaldehyde during the release. Either their formaldehyde antibody test is not a valid indicator of exposure or the definition for exposure in the study was inappropriate.
The authors conclude that persons such as those from Crown Point with known chemical exposure should be examined for autoimmunity and be monitored for "subtle alterations in their immune system, i.e., activation." It must be noted that Broughton has testified that persons who pump their own gasoline should be similarly monitored (37).
What can be done to prevent such reports from appearing in scholarly journals? Environmental Health Perspectives should seriously consider revising its editorial policy to make acknowledgment of financial support mandatory rather than optional, as many other journals have recently done (38). Also, closer scrutiny of litigation-related research (e.g., requiring submission of raw data and related testimony) might also enable reviewers to better assess the quality and integrity of the report under submission. Finally, authors should clearly identify a laboratory which essentially has a vested interest in the validation of a proprietary test. The medical and scientific community is already recognizing and protecting against fraud for academic gain. We must also protect the literature from becoming a marketing tool for people with an interest (financial or other) in supporting a particular view or methodology.

Philip Edelman
Orange, California