Studying Herbal Remedies
 

Wallace Sampson, M.D., NEJM 2005;353:337

How plausible are claims that echinacea, or purple coneflower, a perennial that is native to North America, is an effective treatment for viral respiratory disease? Tracing the evolution of views about the benefits of echinacea from the traditions of indigenous populations to modern claims, one finds little rationale for studying the effects of this herbal remedy on colds. Indigenous populations — who used echinacea in various forms, including teas, local applications, and inhaled smoke — had no concept of disease states or their causes, nor could they distinguish medicinal effects from the natural course of an illness. Herbal texts list the use of echinacea by at least 13 tribes of Native Americans for the treatment of such widely diverse conditions as sore mouth and gums, cough, dyspepsia, toothache, bowel complaints, hydrophobia, and snakebite.

The potential for distortion of information about this herb arose between the late 1600s and the 1800s, when native people transmitted information about the uses of herbs to explorers, traders, and healers. Descriptions were translated into French, Spanish, and English and from each of those languages into others. Eventually, 19th-century physicians adopted herbs into their eclectic medicine, along with water cures, homeopathy, and manipulation. Physician H.F.C. Meyer used echinacea in his "blood purifier," a panacea for conditions ranging from migraine to wounds that were difficult to heal. More distortion probably occurred as 19th-century conditions were renamed and reclassified into modern ones.

Emerging as a panacea in 19th-century America, echinacea somehow became popular for the treatment of respiratory illness in Germany. In the early 1900s in the United States, echinacea was used as an "oral anti-infective" and a local application for wound healing; it then fell from favor after the introduction of antibiotics. Modern histories do not connect these trails. The supplement boom that started in the 1960s brought echinacea back to the United States as a cold remedy.

Between 1950 and 1991, more than 200 clinical reports of studies of echinacea appeared. Most of these were of small, inadequately controlled European studies sponsored by industry. Researchers who were looking for confirmation performed scores of in vitro studies on entire specimens of echinacea plants and on parts and extracts of plants. Positive findings included nonspecific stimulation of immune-cell division and cytokine release, but these effects have little or no correlation with clinical results. Nevertheless, advocates claimed that echinacea spurred stimulation of the immune system.

In this issue of the Journal Turner et al. report a randomized clinical trial of echinacea, now widely advertised as a treatment for viral respiratory disease. In a study sponsored by the National Center for Complementary and Alternative Medicine (NCCAM), the investigators tested three extracts of the root of the one species, Echinacea angustifolia, whose primary constituent group of chemicals had shown some immune-stimulating activity either in vitro or in vivo. The trial was multi-institutional, the numbers of subjects were adequate, and randomization and blinding were accomplished; the investigators used direct nasal viral challenge, a method that has been standardized and used in other trials of treatments for viral respiratory disease. So unless some obscure protocol violation occurred, the trial results are real. The clinical trial found no evidence of any clinically significant efficacy of echinacea.

The search for active fractions usually occurs after a whole substance shows clear efficacy. Investigation of extracts of fractions of echinacea when the results of trials of whole herbs are indeterminate is, at the least, debatable. Previous clinical trials of the whole herb as a treatment for viral respiratory disease had been mixed. Publication bias probably promoted a tendency toward positive reports, and trials that are small and not well controlled tend to show more positive results than do larger trials that are done well. Physicists tell us that negative studies (a sign of nonreproducibility) should carry more weight than positive ones. Manufacturers, whose responsibility it is to prove efficacy to support claims, did not perform definitive larger trials, often claiming a lack of patentability.

The National Institutes of Health (NIH) and pharmaceutical companies have had in place for decades mechanisms to search for potential drugs from natural products. Nevertheless, the NCCAM has developed its own reason for investigating implausible remedies — namely, the popularity of such treatments. It claims to be responding to its mission from the U.S. Congress. But research into implausible remedies rarely produces useful information. Disproof rarely leads the supplement industry to reduce production or the public to decrease use. In fact, advocates often dismiss disproof. The Web page of a naturopathic organization that participated in a recent negative trial of echinacea paraphrased the authors as follows: "Weber and the other researchers conclude that other echinacea preparations and dosing regimens may be effective for the treatment of colds, even though the product they tested in children was not."

Trials of effective treatments with objective end points usually show acceptable, consistent results. But alternative remedies are generally less effective or ineffective, and randomized clinical trials of these remedies measure mostly subjective symptoms. Systematic reviews of these remedies show positive and negative results distributed around the zero-effect line.This finding is accounted for partly by investigators' reliance on smaller studies, partly by publication bias, and partly by inconsistent study criteria, including various entry criteria, various outcome measures, and various population bases. Inconsistent outcomes from studies of alternative treatments seem to be the norm. In addition, despite adequate internal reproducibility, there is no adequate external validation for the various scales used in the evaluation of randomized clinical trials. That means there is a lack of certainty that the results of systematic reviews reflect reality. Nor is there a formula for assessing the relative validity of the conflicting results or a consensus either on how to interpret results of systematic reviews or on the proportion of negative trials necessary to declare a method to be ineffective. Reviewers simply create a consensus estimate. (Neither are there standards for validating reviewers' qualifications, expertise, or opinions.)

Carrying the argument further, there is no "demarcation of the absurd," a point at which it is unwise to pursue an investigation further. Today's literary and editorial correctness often dismisses such a conclusion as evidence of bias. Instead, we find repeated clinical trials, redundant systematic reviews of implausible methods, and indeterminate conclusions.

The inability of randomized clinical trials and systematic reviews to establish inefficacy in research into alternative treatments contributes to a recent loss of bearings. Researchers and advocates of alternative medicine present a mass of information with inadequate heuristics for making sense of it and insufficient standards for making use of it. Should there be studies of other echinacea species, of other parts of the plant, and of each extract of each part of each plant on each cold and each influenza virus? Should these studies be repeated in various combinations, with dose modifications? Why? The possible combinations increase geometrically. Since 1999, the NIH has spent almost $1.5 billion in grants for research into alternative methods. NCCAM has spent almost half that amount and has found no evidence of efficacy and little evidence of inefficacy. NCCAM has three more randomized clinical trials of echinacea that are currently active. As long as research sponsored by NCCAM and private foundations continues, advocates of alternative treatments can claim that a state of equipoise exists when, in fact, the issues should have been settled on the basis of previous knowledge.

It is time for reassessment. First, there is an answer to the question, "Why are we doing randomized clinical trials of folkway uses of herbs and sectarian remedies?" The answer is that proponents and evaluators have excluded plausibility from the equation. What is needed is knowledge-based medicine, with randomized clinical trials of treatments with histories that indicate some reasonable chance of efficacy. This approach mandates a medicine based on evidence that has passed through the sieve of plausibility and that is consistent with basic sciences, other applied sciences, and history — all molded by wisdom and common sense. NCCAM, if it is to justify its existence, must consider halting its search for active remedies through clinical trials of treatments of low plausibility. A wealth of information also awaits discovery in the psychology of personal beliefs in irrational proposals, in the study of erroneous thinking, and in the study of the mechanisms behind errant social–medical trends such as the alternative-medicine movement.