Prevailing on General Causation: Using Risk Ratios and the Bradford-Hill Criteria to Undermine a Plaintiff’s Expert Testimony in Product Liability Actions

This article was originally published in Law360. Any opinions in this article are not those of Winston & Strawn or its clients. The opinions in this article are the authors’ opinions only. 

Several recent decisions have shed light on the courts’ willingness to dismiss a product liability action where the plaintiff lacks sufficiently reliable evidence of general causation—that is, evidence that the product can cause the purported negative outcome. 

Two related scientific concepts frequently arise, especially in cases involving pharmaceuticals or chemicals: (i) relative risk rates^[1] and (ii) the Bradford-Hill criteria for general causation. It is important for defense counsel to understand these concepts so that they can identify the best arguments for dismissal, particularly at the Daubert stage.

Relative Risk Ratios

Relative risk was succinctly described by the Northern District of Florida in a 2018 order in In re Abilify: “Relative risk is simply a comparison of the incidence of a disease in exposed individuals with its incidence in unexposed individuals.”^[2] In the pharmaceutical context, relative risk can be determined by tracking and comparing the rate of negative outcomes over a given period of time in two groups of patients—those who are taking a drug that allegedly causes the negative outcome and those who are not taking the drug.^[3] This comparison of risks results in a risk ratio, which is calculated by dividing the rate of negative outcomes in the exposed group by the rate of negative outcomes in the non-exposed group.^[4]

For instance, as the Central District of California explained in its 2004 In re Silicone Gel Breast Implants opinion:

[I]f a study found that 10 out of 1000 women with breast implants were diagnosed with breast cancer and 5 out of 1000 women without implants (the “control” group) were diagnosed with breast cancer, the relative risk of implants is 2.0, or twice as great as the risk of breast cancer without implants. This is so, because the proportion of women in the implant group with breast cancer is 0.1 (10/1000) and the proportion of women in the non-implant group with breast cancer is 0.05 (5/1000). And 0.1 divided by 0.05 is 2.0.^[5]

Thus, a relative risk of 1.0 indicates that there is no difference in the rate of negative outcomes between the exposed and non-exposed groups, meaning that the exposure and the negative outcome have no association.^[6] A relative risk of less than 1.0 indicates that fewer people in the exposed group experienced the negative outcome, suggesting a potential protective effect.^[7] A relative risk above 1.0 indicates that the exposed group experienced negative outcomes at a greater rate than the non-exposed group, suggesting that the exposure may be associated with an increased risk of the outcome in question.^[8]

Importantly though, association is not causation. As the Southern District of New York explained in its September 2021 Daniels-Feasel opinion, a reliable causation opinion must go beyond epidemiological studies showing a statistically significant increased risk to further “assess whether an exposure-disease relationship is merely associative or is in fact causal.”^[9] Thus, to determine the existence of a causal relationship, epidemiologists commonly analyze the relevant body of scientific evidence and data “using the so-called ‘Bradford Hill’ criteria.”^[10] Thus, the relative risk reported by scientific studies is relevant to a Bradford-Hill analysis but is only part of the overall assessment.

The Bradford-Hill Criteria

The overall general causation inquiry is typically guided by an analysis of nine-factors known as the Bradford-Hill criteria.^[11] As summarized in the recent Daniels-Feasel opinion, the Bradford-Hill criteria are:

1. Strength of Association/Statistical Association. There must be some degree of statistical association between a cause and its effect. A strong association is more likely to represent causation than a weak association. This is a key factor in the Bradford-Hill analysis. As discussed above, the higher the relative risk ratio the stronger the association. A weak association often raises concerns that a study’s results could be explained by bias or confounding.
2. Temporality. A cause must precede its effect. Strength in temporality, such as when a cause immediately precedes its effect, supports an inference of causation.
3. Biological Plausibility. A cause and effect relationship between exposure to medication and disease should be biologically plausible with other information about the disease or harm.
4. Biologic Coherence. A cause and effect relationship between exposure and disease should be consistent with other information about the disease or harm.
5. Biologic Gradient/Dose-Response Effect. Causation is more likely if greater amounts of the putative cause are associated with corresponding increases in the occurrence of disease or harm.
6. Consistency. When similar findings are generated by several epidemiological studies involving various investigators, causation tends to be supported.
7. Analogy. Substantiation of relationships similar to the putative causal relationship increases the likelihood of causation.
8. Experimental Evidence. Causation is more likely if removing the exposure in a population results in a decrease in the occurrence of disease or harm.
9. Specificity. When there is but a single putative cause for the disease or harm, causation is supported.^[12]

Three recent cases provide helpful illustrations of how courts have used relative risk and Bradford-Hill to inform opinions on general causation.

Carl v. Johnson & Johnson

In Carl v. Johnson & Johnson, decided in 2020, the Superior Court of New Jersey reversed the trial court, which had granted summary judgment after excluding Plaintiffs’ causation expert^.[13] Plaintiffs alleged that they developed ovarian cancer by using talc-based baby powder.^[14] The parties’ experts disagreed, among other things, about “how far above 1.0 the association needed to be” to support an inference of causation.^[15]

As relevant here, Plaintiffs’ expert formed his opinion on general causation based on studies reporting a relative risk between 1.24 and 1.61, with the view that “a risk ratio did not have to exceed 2.0 to be meaningful.”^[16] He acknowledged that 1.24 reflected only a “modest increase in risk” but reinforced his analysis by putting significant weight on other Bradford Hill factors, especially “consistent reports of an association, dose response, and biological plausibility.”^[17] He also explained that,

“in comparing study results, a lower relative risk may be more meaningful if it comes from a larger study, for which size alone often affords a tighter confidence interval.”^[18] “He added that the most fundamental question for any study was how well it was designed to identify the nature and extent of the relevant exposure . . . .”^[19]

However, the trial court required “a relative risk of 2.0 as the threshold for the result of an epidemiological study to become reliable for any purpose.”^[20] On this basis, it excluded Plaintiff’s causation evidence and granted summary judgment.^[21]

The reviewing court determined that this was an inappropriate basis for exclusion. In its interpretation, neither the Federal Judicial Center’s Reference Manual on Scientific Evidence nor the Bradford Hill criteria expressly “requires a study to report a risk or odds ratio of 2.0 to be considered support for an inference of causation.”^[22] Thus, although it recognized that a relative risk ratio “substantially lower” than 2.0 demands “greater attention to the possibility of bias, confounding, and likely alternative causes,” the court determined that Plaintiffs’ expert could not be excluded solely on the ground that he based his causation opinion on studies that reported a relative risk below 2.0.^[23]

In re Viagra & Cialis Products Liability Litigation

In In re Viagra & Cialis Products Liability Litigation, the Northern District of California explained that experts who fail to grapple with a low relative risk rate may be excluded for improper methodology and result-driven analysis.^[24] In this case, Plaintiffs alleged that their melanoma progressed due to use of Viagra or Cialis.^[25] However, the district court excluded their expert witnesses on general causation.^[26] Two and a half months later, the parties filed a joint statement agreeing that, in light of this exclusion, summary judgment in favor of Defendants was appropriate.^[27]

The relative risk indicated by the epidemiological studies in In re Viagra was approximately 1.2.^[28] The court noted that, “[a]lthough a risk factor in that range would not necessarily preclude a conclusion that causation exists, it undeniably is not a strong association.”^[29]

Despite this low relative risk, Plaintiffs’ first expert testified that she gave the strength of association factor “significant weight” in her Bradford Hill analysis.^[30] At the same time, she was “unwilling to identify what she perceived the strength of association to be” and focused instead on the “totality” of the evidence, pointing out that multiple studies showed a positive association—an observation that “would at most go to the ‘consistency’ factor under Bradford Hill, not strength of association.”^[31] She also gave virtually no weight to the lack of evidence on the “dose response” factor of the Bradford Hill analysis.^[32] Because the expert did not truly engage with these factors—despite claiming to give the strength of association significant weight—the district court determined that her Bradford Hill analysis was unduly results-driven.^[33]

Similarly, Plaintiffs’ second expert claimed to weigh the strength of the association “relatively heavily in support of the existence of causality” although he did not opine that the association was strong.^[34] And Plaintiffs’ third expert “conflate[d] strength of association with consistency, as a means of downplaying the undeniable fact that the evidence does not support a finding of a strong association.”^[35] Particularly where no experts outside of the litigation had ever concluded that Viagra or Cialis could cause melanoma progression, the court determined that the experts had not faithfully applied the Bradford Hill factors.^[36]

Daniels-Feasel v. Forest Pharmaceuticals, Inc.

The importance of evaluating a study’s reliability and situating its reported risk ratio in the context of a rigorous Bradford Hill analysis is further reinforced by the Southern District of New York’s September 2021 opinion in Daniels-Feasel.^[37] In Daniels-Feasel, Plaintiffs alleged that they developed autism as a result of their mothers’ use of Lexapro during pregnancy.^[38] However, the district court determined that they had failed to offer admissible evidence of causation.^[39] Defendants moved for summary judgment, and on October 15, 2021, Plaintiffs responded that they had no good-faith grounds for opposing summary judgment after the district court’s evidentiary ruling.^[40]

As relevant here, Plaintiff’s expert failed to apply the “strength of association” factor reliably. His analysis disregarded epidemiological studies that ran contrary to his conclusions, dismissing them on the basis of limitations that applied equally to the studies on which he himself relied.^[41] It also “fail[ed] to acknowledge the limitations expressed” by the few studies whose results he preferred.^[42] For example, his report ignored one study’s caution that its results were subject to “detection bias,” because women who received anti-depressants might be more likely to develop concerns about their children’s development and more likely to bring their children for assessments that would lead to autism diagnoses.^[43] Thus, the court doubted the reliability of the expert’s “weighting of the studies he reviewed” when opining on the strength of association.^[44]

The expert’s analysis of the other Bradford Hill factors was similarly unreliable because “he repeatedly cherry-pick[ed] favorable studies to support his conclusions, fail[ed] to explain the weight he attributed to the studies he reviewed, and also ignore[d] entire categories of relevant studies in his report.”^[45] Moreover, he failed to provide a rigorous explanation of how he “weighted the criteria he considered and the studies he cite[d].”^[46]

Daniels-Feasel makes clear that an expert opinion on causation must look beyond a study’s reported risk ratio, especially where the reported increase in risk is minimal. An expert who opines on the strength of an association without explaining the weight of the studies he has reviewed or who opines on causation without reliably evaluating all the Bradford Hill factors will be appropriately excluded.

Strategic Implications

Familiarity with the proper use of relative risk statistics and how those statistics relate to the Bradford-Hill criteria for general causation will allow a lawyer to identify the strongest possible arguments on issues that may be case dispositive.

---

^{[1] While this article generally refers to relative risk ratios, similar principles apply to epidemiological studies employing other metrics, such as odds or hazard ratios. See Carl v. Johnson & Johnson, 237 A.3d 308, 314 (N.J. Super. Ct. App. Div. 2020).}

^{[2] Amended Order at 31, In re Abilify (Aripiprazole) Prods. Liab. Litig. (“Abilify Order”), 3:16-md-2734 (N.D. Fla. Mar. 15, 2018), ECF No. 796.}

^{[3] In re Viagra & Cialis Prods. Liab. Litig., 424 F. Supp. 3d 781, 787 (N.D. Cal. 2020).}

^{[4] In re Silicone Gel Breast Implants Prods. Liab. Litig., 318 F. Supp. 2d 879, 892 (C.D. Cal. 2004).}

^{[5] Id.}

^{[6] Abilify Order at 143 n.161.}

^{[7] In re Viagra, 424 F. Supp. 3d at 793.}

^{[8] Abilify Order at 143 n.161.}

^{[9] Daniels-Feasel v. Forest Pharm., Inc., 2021 WL 4037820, at *3 (S.D.N.Y. Sept. 3, 2021). Several factors beyond the scope of this article affect the reliability of a reported risk ratio. As the Daniels-Feasel opinion explains, “Where a positive association is observed, its validity is assessed by evaluating the role of possible alternative explanations, such as chance, bias, or confounding.” Id. at *2.}

^{[10] Id. at *3.}

^{[11] Abilify Order at 15; see also Daniels-Feasel, 2021 WL 4037820 at *2. “Sir Austin Bradford Hill was a world-renowned epidemiologist who articulated a nine-factor set of guidelines that is widely accepted in the scientific community for determining whether an observed association between an agent and a disease reflects a true causal relationship.” Abilify Order at 15 n.19 (citing Federal Judicial Center Reference Manual on Scientific Evidence and Austin Bradford Hill, The Environment and Disease: Association or Causation?, 58 Proceedings Royal Soc’y Med. 295 (1965)).}

^{[12] Daniels-Feasel, 2021 WL 4037820 at *6–7.}

^{[13] Carl v. Johnson & Johnson, 237 A.3d 308, 311 (N.J. Super. Ct. App. Div. 2020).}

^{[14] Id.}

^{[15] Id. at 315.}

^{[16] Id. at 333–35.}

^{[17] Id. at 334, 339.}

^{[18] Id. at 335.}

^{[19] Id. at 339. For more detailed analysis on the importance of the specific level of exposure, see our article here.}

^{[20] Id. at 342.}

^{[21] Id. at 311.}

^{[22] Id. at 338–39.}

^{[23] Id. at 339.}

^{[24] In re Viagra & Cialis Products Liability Litigation, 424 F. Supp. 3d 781, 797 (N.D. Cal. 2020).}

^{[25] Id. at 786.}

^{[26] Id.}

^{[27] Joint Statement, In re Viagra & Cialis Prods. Liab. Litig., 3:16-md-02691 (N.D. Cal. Apr. 1, 2020), ECF No. 1020.}

^{[28] 424 F. Supp. 3d at 796.}

^{[29] Id.}

^{[30] Id.}

^{[31] Id.}

^{[32] Id.}

^{[33] Id.}

^{[34] Id. at 797.}

^{[35] Id.}

^{[36] Id. at 798–99.}

^{[37] Daniels-Feasel v. Forest Pharm., Inc., 2021 WL 4037820 (S.D.N.Y. Sept. 3, 2021)}

^{[38] Id. at *1.}

^{[39] Id.}

^{[40] Response, Daniels-Feasel v. Forest Pharm., Inc., 1:17-cv-04188 (S.D.N.Y. Oct. 15, 2021), ECF No. 107.}

^{[41] Daniels-Feasel, 2021 WL 4037820 at *8–9.}

^{[42] Id. at *8.}

^{[43] Id.}

^{[44] Id. at *9.}

^{[45] Id. at *11.}

^{[46] Id.}