| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
PUBLIC HEALTH MATTERS |
The author is with the University of Texas School of Law, Austin.
Correspondence: Request for reprints should be sent to Wendy Wagner, University of Texas School of Law, 727 East Dean Keeton Street, Austin, TX 78705-3299 (e-mail: wwagner{at}mail.law.utexas.edu).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONINTEREST GROUPS AND REGULATORY...THEORY AND EXPERIENCEREFORMSUMMARYReferences |
|---|
Recently, there has been a trend in both civil litigation and regulatory law to circumvent the scientific community’s collective judgment on the quality of individual studies with an adversarial process of evaluating scientific quality using interest groups. The Supreme Court’s Daubert v Merrell Dow Pharmaceuticals, Inc opinion and two recent "good science" laws passed by Congress adopt an adversarial process informed by affected parties for reviewing and screening scientific quality. These developments are unwise. Both theory and experience instruct that an adversarial, interest group–dominated approach to evaluating scientific quality will lead to the unproductive deconstruction of science, further blur the distinction between policy and scientific judgments, and result in poor decisions because the courts and agencies that preside over these "good science" contests sometimes lack the scientific competency needed to make sound decisions.
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONINTEREST GROUPS AND REGULATORY...THEORY AND EXPERIENCEREFORMSUMMARYReferences |
|---|
It is no wonder, then, that many scientists worry about several recent legal reforms that rely on affected groups to determine not just the social relevance of scientific studies but the scientific credibility and validity of the actual research without soliciting meaningful input from the scientific community. Scientists who have not been recruited to support one of the warring parties are generally excluded from these adversarial debates over scientific quality. The agenda is instead set by the affected interests. But, as scientists know all too well, affected parties who are not burdened with scientific scruples can make sound science appear controversial by challenging individual methodological decisions, even when scientists themselves would find the choices necessary and appropriate. Affected parties can also conduct ends-oriented research, replete with undisclosed methodological and design decisions selected precisely because they produce a desired, predetermined result. The scientific community strives to avoid these manufactured controversies by sanctioning unethical practices and by adopting disclosure requirements for conflicts of interest, or, in cases where ends-oriented biases seem especially difficult to detect, barring publication completely.
Yet, just as the scientific community condemns biased review of scientific quality by affected parties, the legal system seems to be actively soliciting this input. In a series of disconnected legal reforms, both the courts and the administrative system have begun to abandon processes that seek guidance from "neutral" or balanced scientific panels on the quality of science used for regulation in favor of an adversarial process that relies on interest groups to conduct the review of scientific quality.
These recent developments will weaken, rather than strengthen, the scientific foundations of government regulation and judicial decisions, at least in the areas of environmental and public health protection. The argument proceeds in three parts. In the first part, the move from an expert to an interest group-dominated process for reviewing the quality of scientific research used in regulation is detailed both in the courts and the agencies. The second section then consults both theory and practice to suggest that this trend is neither good for science nor policy. The final section offers proposals for incremental reform.
|
INTEREST GROUPS AND REGULATORY SCIENCE |
|---|
|
TOPABSTRACTINTRODUCTIONINTEREST GROUPS AND REGULATORY...THEORY AND EXPERIENCEREFORMSUMMARYReferences |
|---|
Although a separate review process is not always formalized or explicit, most courts and agencies use the equivalent of a two-track review of how science should be incorporated into regulation. The first review is done by a disinterested or balanced group of scientific experts who review technical features of the relevant research, such as the validity of the experimental methods, data analysis, and statistical practices. This "Expert Review" relies on collaborative efforts by a representative group of experts to isolate points of consensus on important technical and scientific features of policy-relevant research.3 The second review solicits input from interest groups and affected public on how the science should be used for policy. "Interest Group Review" generally involves adversarial processes used by those most adversely affected by a scientific result or series of studies. These two complementary review processes attempt, in a very approximate way, to funnel expert review towards narrow technical issues, while reserving policy decisions for the broader public and affected parties. Courts and regulators then take both types of input into account in reaching a final science-policy decision.
Over the past decade, however, this science-policy review process in both the courts and agencies has begun to shift towards using interest groups for both functions—the evaluation of scientific quality, as well as how that science should be used in public policy—without soliciting the advice or input of the scientific community. Whereas increased reliance by courts and regulators on interest groups to assess scientific reliability does not always supplant the use of scientific advisory panels, in many cases these new processes that solicit interest group review of scientific quality effectively eliminate the need to consult with experts. Perhaps more importantly, in the new review processes, interest groups are portrayed as legitimate and constructive sources of advice on technical issues of scientific quality. This trend, and the accompanying problems that arise from using interest groups, rather than experts, to review scientific quality, is discussed in detail below.
Use of Interest Groups to Review Scientific Quality in the Agencies
Until the late 1990s, science advisory boards and agency peer-review processes provided the primary source of advice on scientific quality for agencies engaged in science-based regulation.4 This "expert" model helped to vet and anchor the relevant science through a balanced committee or group of scientists before subjecting it to the adversarial, ends-oriented attacks of stakeholders.5 Over time, interest groups grew increasingly impatient with primary agency reliance on the expert model (and the accompanying presumption that experts provided the most legitimate form of review of scientific quality) precisely because it constrained the types of attacks they could mount against the underlying science. Interest group frustration piqued in the late 1990s with the results of a high-profile, large epidemiological study called the Harvard Six Cities Study.6 The Six Cities Study, which reported a statistically significant correlation between the concentration of small particulates and early mortality for residents of large cities, was influential in the US Environmental Protection Agency’s (EPA’s) promulgation of an expensive new air quality standard for fine particulates. Because the new air standard threatened to substantially increase compliance costs for many industries emitting these pollutants, the study ignited a veritable firestorm of protest from the affected industries.
In the wake of the Six Cities Studies, disaffected interest groups were instrumental in the passage of two "good science" laws that legitimated their role in reviewing scientific quality and provided them with expanded mechanisms for challenging the quality of science used in regulation.7 The first "good science" law, the Data Access Amendment of 1999, provides affected parties with access to all of the raw data underlying published studies produced with federal support so that they can review and even reanalyze the data in the course of their scientific assessment.8,9 The second "good science" law, the Data Quality Act (or Information Quality Act), supplements the Data Access Amendment by providing interest groups with a formal administrative process for challenging the quality of science (and information) that agencies use.10 Under the Act, if a person (or interest group) believes that the information is not reliable, objective, of utility, or is biased, they can file a request for the "correction" of information "disseminated" by an agency.11 The agency must provide an internal appeal process for correction requests that have been denied.
The "good science" laws provide powerful new processes for affected parties to "review" scientific quality in ways that advance their interests, especially when the tools are used in tandem. If a scientific study suggests that unexpected harms result from a product or activity, for example, interest groups adversely affected by the finding can use the "good science" laws to obtain the data and reanalyze them until they obtain a more favorable outcome. The reanalysis can then be included in the group’s subsequent request for correction under the Data Quality Act, along with a number of other possible scientific disagreements over assumptions, methodology, and statistical techniques.
Use of Interest Groups to Review Scientific Quality in the Courts
Although disconnected in philosophy and underlying motivation, the courts’ review of scientific evidence in civil litigation appears to have experienced a similar shift towards increased reliance on interest groups and adversarial challenges, as opposed to expert consensus, in their review of scientific quality. Like the administrative process, the courts effectively use two types of review processes for the use of science in legal decisionmaking. The first review screens the proffered scientific testimony to ensure its reliability and validity in order to preserve judicial resources and avoid confusing the jury. The inquiry at this stage is largely technical and considers whether the testimony and underlying research is reliable and valid. The second stage, roughly analogous to stakeholder processes in the agencies (interest group review), tackles the more value-laden inquiry of how or whether to weigh the relevant available scientific information in applying the law to a particular case.
In the courts, it is this first stage of review—the technical assessment of scientific quality—that has shifted away from the expert model. Until 10 years ago, federal judges determined whether scientific testimony was reliable based in large part on whether the testimony and underlying studies were "generally accepted" by the scientific community or scientific subfield—an analog of the expert model—set forth in Frye v U.S.12 Frye did not involve convening a representative panel of experts as in the administrative agencies, but consisted of a more amorphous inquiry to determine whether an expert’s testimony was consistent with the general views of their particular expert community.13 Unfortunately, despite its theoretic appeal, practical application of the Frye test was fraught with problems. Courts were dependant on the advocates to inform them of what was "generally accepted," thus deferring to expert consensus in practice only when the litigants were actually faithful in representing that consensus. Even when the courts were accurately informed of the views of the applicable scientific community, however, they rarely attempted to ensure that the relevant scientific community was legitimate (an especially common problem for criminal forensic evidence where the science is developed at the behest of prosecutors) and lacked criteria for making this evaluation even if they wanted to undertake such an inquiry. The Frye test also was biased against novel research that had not had sufficient time to be "generally accepted," although it might be very reliable and probative.14
Things changed in 1992 in Daubert v Merrell Dow Pharmaceuticals, Inc.15 In Daubert, the Supreme Court, faced with revised Federal Rules of Evidence, concluded that Frye was no longer the appropriate test for assessing the reliability and validity of expert testimony.16 Instead, it held that Rule 702 creates a slightly different scientific screening test that positions litigants as the informants on the quality of science and the judge as the arbiter of quality, rather than looking to the scientific community for primary guidance. Although the Court held that courts could still consider whether research underlying expert testimony was peer reviewed and accepted in its field (analogous to an expert review that relies on consensus from within the scientific community), the primary test stepped outside the scientific community and asked, from this external vantage point, whether the science was reliable based on whether the scientific methods used by the expert to support his or her conclusions were capable of being tested and replicated17 And this determination, the Court held, was a judgment that ultimately the courts, not the scientific community, must make. The parties to the litigation, moreover, would provide the primary, if not exclusive, source of information on whether scientific testimony met this "testability" test.
Although more subtle than the shift from the expert to interest group model evident in the administrative arena, the move from Frye to Daubert similarly involves less deference to the scientific community with regard to what constitutes valid and reliable science and greater reliance on the judge’s nonexpert assessment, informed by the litigants. Daubert thus changes the basis for the decision about scientific reliability from one that relies on experts’ collective judgment (albeit filtered through the litigants) to one that demands a judge’s independent assessment of the reliability of the proffered scientific testimony, ultimately moving away from the expert model implicit in Frye.18
In many respects, and despite the move away from deferring to experts to assess scientific reliability, Daubert did overcome a number of problems that afflicted Frye and on balance may be an improvement. Nevertheless, by positioning judges as the primary assessors of scientific quality, Daubert presents its own share of problems. Like their interest group counterparts in administrative practice, litigants have little interest in ensuring scientific quality and are instead primarily concerned with striking testimony that is unfavorable to their position. The information they provide the judge on scientific quality and reliability, then, is likely to be skewed to the tails of scientific opinion and may omit scientific mainstream views. Yet the judges must still rule on whether the proffered scientific testimony is "reliable" based on their nonexpert assessment of the testimony’s faithfulness to the scientific method, rather than looking to the scientific community for assistance.19 In fact, in high-stakes mass litigation, some judges have found the parties so unhelpful in informing their assessment of scientific reliability that they switch back to an expert review model and empanel independent scientific advisors (similar to those traditionally used by the agencies) to assist them in making decisions on scientific evidence challenged under Daubert.20 This occasional retreat to the expert model suggests that judges are not always comfortable presiding over disputes about scientific quality when the primary sources of information are provided by the affected parties.
|
THEORY AND EXPERIENCE |
|---|
|
TOPABSTRACTINTRODUCTIONINTEREST GROUPS AND REGULATORY...THEORY AND EXPERIENCEREFORMSUMMARYReferences |
|---|
This section details the specific types of problems that arise from reliance on interested participants (the interest group model) to resolve disputes over scientific quality, looking first to theory for predictions, and then, when possible, testing those predictions against practice. The analysis suggests that using a scientific review process that is informed primarily by interest groups can lead to lopsided attention to studies that adversely affect powerful partisan interests; aggravation of an existing tendency to blur policy with science; harmful deconstruction of science and research, with backwash effects on the scientists themselves; and errors by lay or political decisionmakers. Each of these problems is explored in more detail below based on current experience with the two "good science" laws and Daubert.
Biased Review
Relying on affected parties and adversarial processes for the review of scientific quality violates one of the fundamental tenets of science, namely that scientific research, as well as peer review of that research, should be unbiased, objective, and disinterested. This central principle of disinterested scientific inquiry runs through all phases of science, from funding decisions to decisions on publication.22 Indeed, the very productivity of the scientific enterprise depends, in large part, on the commitment of each researcher to perform studies in a disinterested way. Whereas numerous opportunities exist to "throw" a study through the multiple, typically invisible judgments involved in the design and implementation of a study, scientists understand that such activities undermine the foundation on which the edifice of scientific inquiry is built. Although studies must be replicable, and typically are, precious research resources are best used when scientists can build on the objective work of their peers to advance the scientific enterprise. Individual scientists are in fact so wedded to this norm that they are reported to often refrain from political activity, at least regarding their research, to ensure that their findings are not tainted by the appearance of bias.23
The scientific community enforces the disinterested norm in a variety of ways, including often requiring the disclosure of possible conflicts of interest before scientists are allowed to publish scientific findings or to serve as peer reviewers.24 In some cases, conflicts of interest actually serve to preclude publication or service as a peer reviewer.25 Forceful commentaries in both the New England Journal of Medicine and Journal of the American Medical Association underscore the importance of researcher compliance with disclosure requirements.26,27 The central importance of this disinterested norm to the conduct of science is further underscored by the considerable angst generated within the scientific community when the objectivity of individual researchers or an entire area of scientific practice is called into question. The growing role of private funding of academic scientific research, for example, has led to a near crisis in the scientific community.28 In addition, researchers charged with biased research through scientific misconduct allegations, although later exonerated, complain of residual and sometimes considerable damage to their reputation.29 This insistence, indeed near obsession with disinterested scientific practice, also extends to less visible issues. For example, research on the peer-review process has suggested that matters as seemingly insignificant as the affiliation or fame of the researcher can affect the outcome of peer review.30 Even these relatively minor deviations from the requirement that scientists remain objective and disinterested has led to calls for reform within the scientific community. Any deviation from the norm of disinterested research and review is, quite legitimately, viewed with concern.
Deconstruction
A second and more practical problem with interest group review is the risk that credible research will be subjected to damaging "deconstruction" by affected parties when lay persons (including political officials) preside over disputes about scientific quality.31 Prof. Sheila Jasanoff defines deconstruction as a challenge brought by outsiders (i.e., those adversely impacted by the research) to the prevailing scientific conventions within a research field.32 Credible studies, traditional research methods, and respected researchers (from the perspective of a "realist-constructionist") may all be deconstructed if those judging or scrutinizing the science do not respect the vulnerable, socially constructed features of traditional research methods, especially those unique to particular disciplines.33 To require the testing and validation of each assumption that underlies a study would result in an infinite regress—the never-ending exposure of assumptions that lack validation. To circumvent this logical problem, established scientific communities informally agree on "accepted methods," some of which are necessarily based on consensual, but technically unvalidated, assumptions.34 If a court or agency responsible for serving as the arbiter of scientific quality is unaware or unconcerned about the necessity of these constructed features of research, attacks against the accepted conventions are likely to succeed. And once deconstructed, a study could be difficult to rehabilitate. Although deconstructing a study can be done simply by calling into question pivotal design features, rehabilitation requires the audience to understand why each of the study’s unsupported decisions is generally accepted by a larger "scientific community," even though at least some of the socially constructed choices are not testable.35
Ends-oriented critiques of scientific research by affected parties are precisely the types of processes likely to lead to the damaging deconstruction of valid science, especially when the scientific community is not involved in the final evaluation of scientific quality. Because cross-examinations are as likely as Daubert hearings to lead to the deconstruction of credible research, it is not obvious that Daubert increases the risk of this counterproductive approach to reviewing science. Damaging deconstruction of credible science, however, would seem to be a common feature of "good science" challenges in the agencies given affected parties’ tendency to discredit research when the results are adverse to their interests. In fact, deconstruction is already evident in several of the data quality correction requests filed to date, and as sophistication grows in filing challenges, deconstruction can be expected to become more common.35
A related, but more dire method that affected parties have employed to discredit research is to disparage the researchers themselves, whether through the complaint process of the Data Quality Act or through other legal avenues made available by the courts or administrative processes. There is worrisome evidence of vigorous use of this tactic both in the courts and agencies.36,37 Whereas professional denigrations are not central to deconstruction, they do complement it. Whether this professional discrediting ultimately impairs the researcher’s reputation within the scientific community or has other negative spillover effects is another issue that deserves study.38 And although the new "good science" laws and Daubert are not essential to enable such attacks, they do provide additional public platforms for publicizing discreditation efforts.
Blurring of Science and Policy
Theory and experience predict that both Daubert and the "good science" laws could also obscure distinctions between science and science policy, since both reforms carelessly enlist affected stakeholders to review scientific quality. Expanding avenues for interest groups to challenge agency regulations but requiring the challenges to be directed only at the quality of agency science provides strong incentives for these groups to mischaracterize fundamental policy conflicts as instead disagreements over "good science." Indeed, affected parties eager to undercut a result can reframe science-policy decisions as if they are really "scientific" findings and challenge their validity by arguing that the findings are not scientifically validated. Lay adjudicators may be sympathetic to these technical challenges if they do not understand that the real disagreement is over policy assumptions. Agency officials or judges who are particularly sensitive to charges of scientific incompetence or are politically inclined to agree with a challenger could even become conspirators in this charade.
The possibility of blurring science and policy is confirmed in practice in both Daubert decisions and the "good science" laws. The most controversial Daubert rulings involve judges who exclude expert weight-of-the-evidence testimony because the expert’s overarching conclusion is not testable, although the component studies can be tested and are often reliable.39,40 Critics argue that the circumstantial quality of this type of testimony, coupled with the underlying rigor of the individual research, present policy-based questions that juries should be empowered to make; but in excluding this evidence under Daubert, some judges have confused these policy-laden inquiries with the assessment of scientific quality.41
Evidence of the blurring of science and science policy is even more dramatic with respect to the Data Quality Act. Virtually every request for correction filed to date involves covert challenges to agency policies disguised as disputes over "good science." In one of the most significant data quality challenges brought thus far, the affected industries claim that the EPA must exclude from its atrazine risk assessment a series of studies published by Dr Tyrone Hayes and colleagues at the University of California, Berkeley, that detect significant endocrine effects on frogs exposed to low levels of atrazine.42 Despite appearances to the contrary, the industries’ grievances in this complaint are not about the credibility of the Hayes studies, but rather about the EPA’s policies for using cutting-edge research in assessing the risks of herbicides. First, industry argues that because the Hayes protocols were not preapproved by the EPA (indeed, the EPA has not yet approved any protocols for studying endocrine disruption), the studies had to be excluded in assessing the herbicide’s risks.43 Second, industry argues that the agency could not rely on published studies until the results had been validated.43 Both arguments—that agencies must preapprove protocols and research must be validated or even replicated before it can be used in regulatory decisionmaking—are predominantly policy, not technical, disagreements that directly conflict with the agency’s protective statutory mandate in this case.44 Perhaps even more worrisome, the EPA equivocated in responding to the industry’s complaint and ultimately capitulated by agreeing to certify the two questions to its Science Advisory Panel, further obscuring the policy-related issues at stake in the industry’s challenge.45
Other Data Quality Act complaints that purport to challenge the validity of agency science similarly take issue, at least in part, with the agency’s value judgments or policy extrapolations adopted within the context of a larger risk assessment. In one complaint, industry challenged the EPA’s barium risk assessment, in part because the industry disagreed with the agency’s conservative interpretation of the data for preventive regulation.46 In several other complaints, the Competitive Enterprise Institute argued that the EPA, the National Oceanic and Atmospheric Administration, and by association, the Office of Science and Technology Policy used a flawed model to predict global warming and that all of the reports and data relying on that model should be withdrawn.47 Competitive Enterprise Institute did not, however, offer more accurate models as an alternative and is thus essentially challenging the use of predictive models to anticipate harm when information is incomplete.48
Imbalance in the Studies Scrutinized for Scientific Quality
By definition, the only scientific studies subject to scrutiny under the interest group model will be those that adversely affect a party significantly enough to justify the costs of a challenge. The expert model, by contrast, provides a more comprehensive review of the quality of science informing a given policy. The resulting shift from an expert to an interest group model can thus be expected to lead to a much more selective, incomplete set of studies subject to heightened review. This prediction is supported in practice with Daubert and the "good science" laws, sometimes in unexpected ways.
A first, rather dramatic indication of an imbalanced scientific review process under the "good science" laws is their exemption of much of the science done by private and regulated parties, even when private parties are required by law to produce the information.49 These exemptions betray the possibility that the "good science" laws may not really be intended to improve scientific quality, because they provide processes for challenging public science while insulating private research from scrutiny—even though private research appears to be more inclined towards bias.50
Second, there are distinct imbalances in the groups that bring "good science" challenges, in contrast to more neutral review of research by scientific committees. Parties in civil cases are finding that Daubert challenges are expensive to mount and defend, a feature, that is a disadvantage to those with limited resources available for litigation.51 For example, plaintiff attorneys report that the added cost of Daubert hearings not only impacts their decisions about the types of expert evidence to introduce but also affects their decisions about which cases to take.52 If a case is likely to involve one or more expensive Daubert hearings, for example, the plaintiff-victim’s potential damages judgment must be large enough to include an allocation for legal expenses that compensate for the costs of the hearing, as well as for the other costs of bringing suit. Similarly, the dearth of successful Daubert challenges to forensic evidence introduced by prosecutors in criminal trials is likely attributable in part to the expense of these challenges.53 Because counsel for many criminal defendants provide representation either as a public service or through under-funded public defenders’ offices, most criminal defendants will be unable to finance Daubert challenges.
As in the case of Daubert, one would expect those with fewer resources, such as public interest groups and private citizens, to be less able to mount sophisticated "good science" complaints under the Data Quality Act. The one-directional nature of the "good science" reforms—directed at public science, while exempting most private research—additionally limits the opportunities available to public interest groups to bring "good science" challenges. Finally, delays that could result from "good science" challenges will often act at cross purposes with public interest groups’ emphasis on swift regulatory action in the face of uncertain but probable harm. As anticipated, the vast majority of "good science" complaints filed to date under the Data Quality Act against the EPA have been submitted by regulated parties.54,55
Competence of Lay or Political Decisionmakers
Compounding worries about incomplete disclosures and skewed information provided by interested parties in their review of scientific quality are concerns about the decision-makers’ scientific competence to assess scientific reliability, especially when their sources of information on these questions are so badly biased and incomplete. Judges and regulatory officials, to a lesser extent, are not typically scientists, and we know from a large body of critical literature and a growing number of published opinions that they sometimes make decisions about science that are wrong.56–58 The literature on judicial review of agency technical decisions is perhaps the most negative about the capacity of judges to review the science used by agencies in rulemakings.57,58 But scientific errors committed by lay persons who preside over challenges to the quality of scientific studies arise throughout the science-policy literature, with documented problems arising in scientific misconduct hearings, discovery disputes, and evidence determinations.59
To the extent that errors emerge from these frailties in the system, the most serious error is the possibility that good studies will be excluded from the policymaking process. This is especially troubling because scientific research is already in short supply in many areas of environmental and health policy. The probability that good science will be erroneously excluded seems most likely when an affected party has a great deal to lose from a research study and has the resources to invest in discrediting it.60 Indeed, one would expect Daubert and the "good science" challenges to be used only when a party is adversely affected by scientific knowledge and has the resources to mount an expensive technical attack against it. Research on Daubert’s effects in erroneously excluding good research is ongoing, although preliminary evidence suggests that this is a problem in some cases.61 Experience thus far with the Data Quality Act indicates that the agencies are resisting petitions for correction or exclusion of scientific studies, but future responses will depend in part on the political inclinations of the agency.
The "Good Science" Reforms: Potentially a Greater Danger
In assessing the wisdom of the increased reliance on interest groups to review scientific quality, it is important to consider differences, as well as similarities, between courts and agencies. Because of several major institutional differences, use of the interest group model to review scientific quality is likely to present more serious dangers in the regulatory arena than in the courts. First, rather than a supposedly "neutral" jurist who will preside over challenges to the veracity of a scientific study, the "good science" laws task a politically appointed official to conduct this review. This presents significant added dangers that political considerations will bias the review of scientific quality. Even in the courts, covert values have arguably afflicted some judges’ Daubert rulings, particularly rulings on the sufficiency of a party’s evidence under Daubert-Joiner.62 Because agencies are fundamentally political entities, one would expect their decisions to be subject to even greater blurring of science and policy.63 Moreover, because challenges can be brought when an agency first considers a study, that study’s relationship to the larger scientific literature and to the agency’s mandate for regulating in a preventive way is at risk of being ignored in the agency’s determination of whether an individual study qualifies as reliable and objective. Even more perversely, the "good science" laws may enable the agency to advance its political agenda covertly, because the technical appearance of the complaints reduces the ability of attentive public, busy nonprofits, and even elected officials to understand the underlying policy choices, much less their implications.
Second, "good science" challenges in the administrative arena are likely to have wider ranging implications than their judicial analogs. Daubert rulings (including erroneous ones) are generally limited to a single federal district, which is often a section of a state. By contrast, the final decision on an interest group challenge to scientific quality under the "good science" laws is made by a federal agency and applied nationwide. Indeed, for this very reason a law firm defending the asbestos companies in the asbestos litigation has filed a Data Quality Act petition seeking the withdrawal of evidence damaging to their client’s case.64 If successful, their Data Quality Act challenge will expunge the bothersome information, and they will no longer be forced to seek its exclusion, sometimes unsuccessfully, in individual court cases.
Third and finally, because the "good science" reforms create added processes for interest groups to unilaterally challenge the quality of scientific research, these adversarial processes are capable of becoming substantially more imbalanced than their civil litigation counterparts. The greater stakes at issue—national rules that apply across multiple industries—mean that the resources dedicated to challenges by regulated parties could be well in excess of those dedicated to single Daubert challenges. The opportunity for multiple interest groups to file cumulative, complementary challenges in this national setting could additionally increase the strain put on the validity of a single study. Perhaps the most important contrast to Daubert hearings, however, is the fact that there may be no interest groups willing or able to invest resources in defending a challenged study, leaving the uncompensated researcher and possibly the agency to rehabilitate deconstructed research without assistance.
|
REFORM |
|---|
|
TOPABSTRACTINTRODUCTIONINTEREST GROUPS AND REGULATORY...THEORY AND EXPERIENCEREFORMSUMMARYReferences |
|---|
Counteracting the Opportunities for Abuse and Imbalance in Review
Several modest adjustments to Daubert and the "good science" laws will help correct the built-in incentives for abuse, as well as the imbalances that result from the resource-intensive nature of these adversarial processes. First, in keeping with the courts’ sanctions for frivolous claims, agencies could promulgate rules that make it clear that abuse of the "good science" laws will be penalized, thus moderating incentives for parties to use the challenges in a harassing way.65 Thus, if there is no good-faith basis for a "good science" challenge, penalties would be levied against the challenger. Such sanctions would penalize only the very worst abuses, but the mere threat of sanctions could help to deter marginal abuses.
Second, indigent parties to litigation, or at least indigent criminal defendants, could apply for Daubert funds to support Daubert challenges or to defend against them. These funds could be provided by a small additional tax on court filing fees, through penalties collected from nonmeritorious challenges, or through general revenues.
Third, given the resource drag that might afflict the agencies, the costs agencies incur in responding to "good science" complaints and data requests could be paid for by those bringing a challenge if the challenge is ultimately denied. In the process of developing implementing regulations for the Data Access Act, the scientific community convinced OMB to include a provision requiring the requestor to reimburse nongovernmental scientists and other researchers for out-of-pocket costs incurred in responding to data requests.66 This internalization of processing costs could be extended to the Data Quality Act. There will be greater protection against abuse of process if complainants are required to pay for the agency costs associated with responding to challenges ultimately judged to be without merit. Such a mechanism will also ensure that the agency’s scarce resources are not squandered and diverted, particularly if the resolution of a data quality correction request forces the agency into an expensive consultation with the National Academy of Sciences or a science advisory panel.
Discouraging the Blurring of Science and Science Policy
Decisionmakers should be vigilant in ensuring that policy disputes are not mixed into the assessment of scientific quality. Accordingly, in adjudicating Daubert challenges and "good science" complaints, the very first step a lay decisionmaker should take is to ensure that the challenge is one against science rather than policy or science policy. This formal decisionmaking step will introduce some accountability for affected parties who are otherwise free to mischaracterize value disputes as if they were instead technical disagreements.
Protection against Unproductive Deconstruction
One of the greatest dangers of the "good science" reforms is that they will be used to deconstruct valid studies and disparage credible researchers. Sanctions should be awarded when a challenge is blatantly frivolous as discussed earlier. In addition, particularly when researchers themselves are implicated in the deconstruction or disparagement of research, scientific organizations, like the American Association for the Advancement of Science or the National Academy of Science, could provide a valuable public service by establishing a balanced committee of scientists to investigate the complaints and defend innocent researchers, much as the American Association of University Professors investigates complaints of infringements on academic freedom.67 These panels could conduct full examinations of the legitimacy of these "good science" challenges and issue a report that would be made public and provided to the agency (or court) involved in the matter. In cases where individual researchers are unfairly disparaged, moreover, the group could come to the researchers’ assistance and even recommend professional or legal sanctions against those doing the disparaging. This type of respected panel could provide a valuable, neutral assessment of the challenged science. They could also raise the costs to interest groups, essentially through reputational damage, for exaggerating flaws in good research, conducting biased reanalyses, or otherwise obfuscating the value of well-done research under the pretext of "good science" challenges.
|
SUMMARY |
|---|
|
TOPABSTRACTINTRODUCTIONINTEREST GROUPS AND REGULATORY...THEORY AND EXPERIENCEREFORMSUMMARYReferences |
|---|
|
Acknowledgments |
|---|
|
Footnotes |
|---|
Accepted for publication September 1, 2004.
|
References |
|---|
|
TOPABSTRACTINTRODUCTIONINTEREST GROUPS AND REGULATORY...THEORY AND EXPERIENCEREFORMSUMMARYReferences |
|---|
2. S. Shackley, "Science and policymaking," Environment 1997; 39: 3.
3. U.S. EPA, About the EPA Science Advisory Board, available at http://www.epa.gov/sab/about.htm (last accessed April 17, 2005).
5. 5 United States Code App I, 
5(b)(2), 5(c).
6. D. Dockery, C.A. Pope III, X. Xu, et al. "An Association Between Air Pollution and Mortality in Six U.S. Cities." New England J of Med, 329 (1993): 1753–1759.
7. National Research Council, Access to Research Data in the 21st Century: An Ongoing Dialogue Among Interested Parties: Report of a Workshop. (Washington DC: National Academy Press, 2002), 8–12.
8. R. Shelby. "Accountability and Transparency: Public Access to Federally Funded Research Data," Harv J on Legis, 37 (2000): 371, 379.
9. E. Marshall. "Epidemiologists Wary of Opening Up Their Data," Science, 290 (2000): 5489.
10. Treasury and General Government Appropriations Act for Fiscal Year 2001, Pub. L. No. 105–554, 515.
11. OMB, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies; Republication, 67 Fed. Reg. 8452 (2002).
12. Frye v United States, 293 F. 1013, 1014 (D.C. Cir. 1923).
13. D.L. Faigman, D.H. Kaye, M.J. Saks, et al. Modern Scientific Evidence: The Law and Science of Expert Testimony, vol. 1. (St. Paul, MN: West Publishing Co, 2002), 7.
14. P. Giannelli. "The Admissibility of Novel Scientific Evidence: Frye v United States, a Half Century Later," Colum. L. Rev. 80 (1980): 1197.
15. Daubert v Merrell Dow Pharmaceuticals, Inc, 509 U.S. 579 (1993).
16. D.H. Kaye, D.E. Bernstein, and J.L. Mnookin. The New Wigmore: A Treatise on Evidence. (New York, NY: Aspen Publishers; 2004), 199–201.
20. Hall v Baxter Healthcare Corp, 947 F.Supp. 1387 (D.Or. 1996).
21. S. Krimsky. Science in the Private Interest: Has the Lure of Profits Corrupted Biomedical Research? (Lanham, MD: Rowman and Littlefield, 2003), 125–140.
22. R.K. Merton. The Normative Structure of Science. In J. Gaston. The Sociology of Science. (San Francisco, CA: Jossey-Bass Publishers. 1973), 267, 275.
23. V.L. Melnick, D. Melnick, H.H. Fudenberg. "Participation of Biologists in the Formulation of National Science Policy," Fed’n Proc. 35 (1976): 1957.
24. International Committee of Medical Journal Editors, Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication, October 2004, available at http://www.icmje.org/ (last accessed April 17, 2005).
26. J.M. Drazen and G.D. Curfman. "Financial Associations of Authors," N Engl J Med 346 (2002):1901–1902.
27. DeAngelis CD, Fontanarosa PB, Flanagin A. Reporting financial conflicts of interest and relationships between investigators and research sponsors. JAMA 2001;286:89–91.
28. E. Press and J. Washburn. "The Kept University," The Atlantic Monthly, 285 (2000): 38–54.
29. G.H. Reynolds. "Thank God for the Lawyers: Some Thoughts on the (Mis)regulation of Scientific Misconduct," Tenn L Rev 66(1999): 801, 814.
32. S. Jasanoff. "Research Subpoenas and the Sociology of Knowledge," L & Contemp Probs: 59 (1996): 95, 98–100.
33. S. Cole. Making Science. (Cambridge, MA: Harvard Press; 1992).
34. Jasanoff, Research Subpoena, 99–100.
35. Competitive Enterprise Institute, Petitions to Cease Dissemination of the National Assessment on Climate Change, Feb. 20, 2003 [available at www.thecre.com]
36. F.J. Johnson. "The Next Battle Over Clean Air," Rolling Stone. 48 (2001): 52.
37. A. Avery. "Frog Sex-Change Claims Flawed, Center for Global Food Issues, October 30, 2002," Center for Global Food Issues [http://www.cgfi.org/materials/articles/2002/oct_30_02.htm]. Accessed April 17, 2005.
38. P.M. Fischer, Science and Subpoenas: When Do the Courts Become Instruments of Manipulation? Law & Contemp Probs 59 (1996): 159.
39. General Electric Co v Joiner, 522 U.S. 136 (1997).
40. Siharath v Sandoz Pharmaceuticals Corp, 131 F. Supp. 2d 1347 (N.D. Ga. 2001), aff’d sub nom. Rider v Sandoz Pharmaceuticals Corp, 295 F.3d 1194 (11th Cir. 2002).
41. L.M. Finley. "Guarding the Gates to the Courthouse: How Trial Judges are Using their Evidentiary Screening Role to Remake Tort Causation Rules," DePaul L Rev 49 (1999): 335.
42. T.B. Hayes, A. Collins, M. Lee, et al. "Hermaphroditic, Demasculinized Frogs after Exposure to the Herbicide Atrazine at Low Ecologically Relevant Doses," Proc Natl Acad Sci 99 (2002): 547.
43. Kansas Corn Growers Association, the Triazine Network, and the Center for Regulatory Effectiveness, Request for Correction of Information Contained in the Atrazine Environmental Risk Assessment. Docket No. OPP – 34237A. (November 25, 2002). [available at www.thecre.com]
44. Environmental Defense Fund, Inc v EPA, 548 F.2d 998, 1004–05 (D.C. Cir. 1977), cert. denied, 431 U.S. 925.
45. EPA, Draft IRED, at 68–69, available at http://www.thecre.com/pdf/20030224-epa.pdf
46. Chemical Products Division, Request for Correction of the IRIS Barium substance file, Oct. 29, 2002; Letter from Paul Gilman, Assistant Administrator, EPA to Jerry Cook, Chemical Products Division, Jan. 30, 2003. [available at www.thecre.com.]
47. Andrew C. Revkin. "Suit Challenges Climate Change Report by U.S.," New York Times, Aug. 7, 2003.
48. Competitive Enterprise Institute, Petitions to Cease Dissemination of the National Assessment on Climate Change, Feb. 20, 2003 [available at www.thecre.com]
49. W. Wagner and D. Michaels. "Equal Treatment for Regulatory Science: Extending Controls Governing the Quality of Public Research to Private Research," Am J of L & Med 30 (2004): 117.
50. J.E. Bekelman, Y. Li, and C.P. Gross. "Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A Systematic Review," JAMA 289 (2003): 454–465.
51. E. Relkin. "To Hear or Not to Hear: When Are Daubert Hearings Appropriate?," ALI-ABA SF78 (2001): 371, 375.
52. Berger MA. What has a decade of Daubert wrought? Am J Public Health 95 (2005): S59–S65.
53. Neufeld PJ. The (near) irrelevance of Daubert to criminal justice and some suggestions for reform. Am J Public Health 95 (2005): S107–S113.
54. Center for Effective Regulation, www.thecre.com (accessed March 4, 2003).
55. EPA, Log of Petitions Filed under the Information Quality Act, available at http://www.epa.gov/quality/informationguidelines/iqg-list.html (last visited April 17, 2005).
56. N.A. Ashford, "A Hard Look at Federal Regulation of Formaldehyde: A Departure from Reasoned Decisionmaking," Harv. Envtl. L. Rev. 1983: 7; 329–333.
57. D. Davis, "The "Shotgun Wedding" of Science and Law: Risk Assessment and Judicial Review," Colum. J. Envtl. L. 1985: 10; 67, 86, 92, 98.
58. E.D. Elliott, ED, R. Pierce, A. Raul, et al., "Science, Agencies, and the Courts: Is Three a Crowd?," Envtl. L. Rep. (ELI) 2001:31; 10125, 10135–37.
59. Symposium on Court-Ordered Disclosure of Academic Research: A Clash of Values of Science and Law. Law & Contemp Probs 1996: 59 (3).
60. J. Stauber, S. Rampton. Trust Us, We’re Experts. New York, NY: J.P. Tarcher, 2002.
61. C.F. Cranor and J.A. Eastmond. "Scientific Ignorance and Reliable Patterns of Evidence in Toxic Tort Causation: Is there a Need for Liabilty Reform?" L. & Contemp. Probs. 5 (2001): 26–45.
62. Siharath v Sandoz Pharmaceuticals Corp, 131 F.Supp.2d 1347, 1351 (N.D. Ga. 2001.
63. W.E. Wagner. "The Science Charade in Toxic Risk Regulation," Columbia L Rev 95 (1995): 1613.
64. Letter from Dino Privitera, Morgan Lewis to EPA, Information Quality Guidelines Staff (August 19, 2003), available at http://thecre.com/pdf/mlb_correction.pdf. Accessed April 17, 2005.
65. Rule 11, Federal Rules of Civil Procedure.
67. American Association of University Professors, Committee A on Academic Freedom and Tenure (see www.aaup.org). Accessed April 17, 2005.
This article has been cited by other articles:
|
|
 |
|
  L. Rosenstock Protecting special interests in the name of "good science". JAMA, May 24, 2006; 295(20): 2407 - 2410. [Full Text] [PDF]
|
|
|
|
 |
|
 E. Bingham, L. Boden, R. Clapp, P. Hoppin, S. Krimsky, D. Michaels, D. Ozonoff, and A. Robbins BINGHAM ET AL. RESPOND Am J Public Health, February 1, 2006; 96(2): 206 - 207. [Full Text] [PDF]
|
|
|
|
|
|
D. Michaels Scientific Evidence and Public Policy Am J Public Health, July 1, 2005; 95(S1): S5 - S7. [Full Text] [PDF]
|
|
|
|
|
|
P. J. Hoppin and R. Clapp Science and Regulation: Current Impasse and Future Solutions Am J Public Health, July 1, 2005; 95(S1): S8 - S12. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. L. Melnick A Daubert Motion: A Legal Strategy to Exclude Essential Scientific Evidence in Toxic Tort Litigation Am J Public Health, July 1, 2005; 95(S1): S30 - S34. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
