The "art of medicine" is defined traditionally as a practice combining medical knowledge (including scientific evidence), intuition, and judgment in the care of patients . EBM updates this construct by placing much greater emphasis on the processes by which clinicians gain knowledge of the most up-to-date and relevant clinical research to determine for themselves whether medical interventions alter the disease course and improve the length or quality of life. The meaning of practicing EBM becomes clearer through an examination of its four key steps:
Formulating the management question to be answered
Searching the literature and online databases for applicable research data
Appraising the evidence gathered with regard to its validity and relevance
Integrating this appraisal with knowledge about the unique aspects of the patient (including the patient's preferences about the possible outcomes)
Step 1 involves generating well-formulated questions that involve four or five components—PICOD: patient or population, intervention, comparator, outcome, and, sometimes, D for study design, (e.g., does routine percutaneous coronary intervention improve survival compared with initial medical management in 60-year-old men with stable angina and known CAD?) Steps 2 and 3 are the heart of EBM as it is currently used in practice and relate to the underlying fundamental principle that the strength of medical evidence supporting a therapy or strategy is hierarchical. The process of searching the world's research literature and appraising the quality and relevance of studies thus identified can be quite time-consuming and requires skills and training that most clinicians do not possess. Thus, the best starting point for most EBM searches is the identification of recent systematic overviews of the problem in question (Table 3-3).
Table 3-3 Selected Tools for Finding the Evidence in Evidence-Based Medicine
|
|
| Name | Description | Web Address | Availability |
|---|
| Evidence-Based Medicine Reviews | Comprehensive electronic database that combines and integrates:
- 1. The Cochrane Database of Systematic Reviews
- 2. ACP Journal Club
- 3. The Database of Abstracts of Reviews of Effectiveness
| www.ovid.com | Subscription required. Available through medical center libraries and other institutions. |
| Cochrane Library | Collection of EBM databases, including The Cochrane Database of Systematic Reviews—full text articles reviewing specific health care topics. | www.cochrane.org | Subscription required. Abstracts of systematic reviews available free online. Some countries have funding to provide free access to all residents. |
| ACP Journal Club | Collection of summaries of original studies and systematic reviews. Published bimonthly. All data since 1991 available on Web site, updated yearly. | www.acpjc.org | Subscription required. |
| Clinical Evidence | Monthly updated directory of concise overviews of common clinical interventions. | www.clinicalevidence.com | Subscription required. Free access for United Kingdom and developing countries. |
| MEDLINE | National Library of Medicine database with citations back to 1966. | www.nlm.nih.gov | Free via Internet. |
|
|
Generally, the EBM tools listed in Table 3-3 provide access to research information in one of two forms. The first, primary research reports, is the original peer-reviewed research work that is published in medical journals. Initial access to this information in an EBM search may be gained through MEDLINE, which provides access to a huge amount of data in abstract form. However, in using MEDLINE it is often difficult to locate reports that are on point in a sea of irrelevant or unhelpful information and be reasonably certain that important reports have not been overlooked. The second form, systematic reviews, comprehensively summarizes the available evidence on a particular topic up to a certain date and provides the interpretation of the reviewer and thus is the highest level of evidence in the hierarchy. Explicit criteria are used to find all the relevant scientific research and grade its quality. The prototype for this kind of resource is the Cochrane Database of Systematic Reviews. One of the key components of a systematic review is a meta-analysis. The next two sections will review some of the major types of clinical research reports available in the literature and the process of aggregating those data into meta-analyses.
SOURCES OF EVIDENCE: CLINICAL TRIALS AND REGISTRIES
The notion of learning from observation of patients is as old as medicine itself. Over the last 50 years, physicians' understanding of how best to turn raw observation into useful evidence has evolved considerably. Case reports, personal anecdotal experience, and small single-center case series are now recognized as having severe limitations in validity and generalizability, and although they may generate hypotheses or be the first reports of adverse events, they have no role in formulating modern standards of practice. The major tools used to develop reliable evidence consist of the randomized clinical trial and the large observational registry. A registry or database typically is focused on a disease or syndrome (e.g., cancer, CAD, heart failure), a clinical procedure (e.g., bone marrow transplantation, coronary revascularization), or an administrative process (e.g., claims data used for billing and reimbursement).
By definition, in observational data, the care of the patient is not controlled by the investigator. Carefully collectedprospective observational data can achieve a level of quality approaching that of major clinical trial data. At the other end of the spectrum, data collected retrospectively (e.g., chart review) are limited in form and content to what previous observers thought was important to record, which may not serve the research question under study particularly well. Data not specifically collected for research (e.g., claims data) often have important limitations that cannot be overcome in the analysis phase of the research. Advantages of observational data include the ability to capture a broader population than is typically represented in clinical trials because of inclusion and exclusion criteria. In addition, observational data are the primary source of evidence for questions for which a randomized trial cannot or will not be performed. For example, it may be difficult or unethical to randomize patients to test diagnostic or therapeutic strategies that are unproven but widely accepted in practice. In addition, patients cannot be randomized to a sex, racial/ethnic group, socioeconomic status, or country of residence. Physicians are also not willing to randomize patients to a potentially harmful intervention, such as smoking or overeating to develop obesity.
The major difference between a well-done randomized clinical trial and a well-done prospective observational study of a particular management strategy is the lack of protection from treatment selection bias in the latter. The use of observational data to compare diagnostic or therapeutic strategies assumes that there is sufficient uncertainty in practice to ensure that similar patients will be managed differently by different physicians. In short, the analysis assumes that there is an element of randomness (in the sense of disorder rather than in the formal statistical sense) to clinical management. In such cases, statistical models attempt to adjust for important imbalances and "level the playing field" so that a fair comparison among treatment options can be made. When management is clearly not random (e.g., all eligible left main coronary artery disease patients are referred for coronary bypass surgery), the problem may be too confounded (biased) for statistical correction, and observational data may not provide reliable evidence.
In general, the use of concurrent controls is vastly preferable to that of historical controls. For example, comparison of current surgical management of left main CAD with left main CAD patients treated medically during the 1970s (the last time these patients were routinely treated with medicine alone) would be extremely misleading since the quality of "medical therapy" has made substantial improvements in the interval.
Randomized controlled clinical trials include the careful prospective design features of the best observational data studies but also include the use of random allocation of treatment. This design provides the best protection against confounding due to treatment selection bias (a major aspect of internal validity). However, the randomized trial may not have good external validity (generalizability) if the process of recruitment into the trial resulted in the exclusion of many potentially eligible subjects.
Consumers of medical evidence need to be aware that randomized trials vary widely in their quality and applicability to practice. The process of designing such a trial often involves a great many compromises. For example, trials designed to gain U.S. Food and Drug Administration (FDA) approval for an investigational drug or device have to address certain regulatory requirements that may result in a trial design different from what practicing clinicians would find useful.
META-ANALYSIS
The Greek prefix meta signifies something at a later or higher stage of development. Meta-analysis is research done on research data for the purpose of combining and summarizing the available evidence quantitatively. Although it can be used to combine nonrandomized studies, meta-analysis is used most typically to summarize all the randomized trials on a particular therapeutic problem. Ideally, unpublished trials should be identified and included to avoid publication bias (i.e., "negative" trials may not be published). Furthermore, some of the best meta-analyses obtain and analyze the raw individual patient-level data from all trials rather than working only with what is available in the published reports of each trial. Not all published meta-analyses are reliable sources of evidence on a particular problem. Their methodology must be scrutinized carefully to ensure proper study design and analysis. The results of a well-done meta-analysis are likely to be most persuasive if they include at least several large-scale, properly performed randomized trials. Although meta-analysis can help detect benefits when individual trials are inadequately powered (e.g., the benefits of streptokinase thrombolytic therapy in acute MI demonstrated by ISIS-2 in 1988 were evident by the early 1970s through meta-analysis), in cases in which the available trials are small or poorly done, meta-analysis should not be viewed as a remedy for the deficiency in primary trial data.
Meta-analyses typically focus on summary measures of relative treatment benefit, such as odds ratios or relative risks. Clinicians also should examine what absolute risk reduction (ARR) can be expected from the therapy. A useful summary metric of absolute treatment benefit is the number needed to treat (NNT) to prevent one adverse outcome event (e.g., death, stroke). NNT is simply 1/ARR.
For example, if a hypothetical therapy reduced mortality rates over a 5-year follow-up by 33% (the relative treatment benefit) from 12% (control arm) to 8% (treatment arm), the absolute risk reduction would be 12% – 8% = 4% and the NNT would be 1/.04, or 25. Thus, it would be necessary to treat 25 patients for 5 years to prevent 1 death. If the hypothetical treatment was applied to a lower-risk population, say, with a 6% 5-year mortality, the 33% relative treatment benefit would reduce absolute mortality by 2% (from 6 to 4%), and the NNT for the same therapy in this lower-risk group of patients would be 50. Although not always made explicit, comparisons of NNT estimates from different studies should account for the duration of follow-up used to create each estimate
|
