“Evidence-based”

Evidence-based is a term that is as misused and misidentified as are politicians’ quotes. The term has been used to denote systematic evaluation of research, but has also been used to confuse and “sell” concepts without providing comprehensive evaluation of the evidence supporting the concepts. When clinical reference content is accurately and consistently derived from the best available research using transparent methodology, then the “evidence-based” approach allows clinicians can make informed decisions that enable them to provide the best care to their patients at the point-of-care.

Evidence-based, in any field, means that conclusions are based on the best available evidence. This doesn’t mean that the evidence will never change or be altered; to be consistently reliable, evidence must be continuously and consistently and systematically identified, evaluated, and selected.

For evidence-based medicine (EBM) content to correctly, accurately, and reliably be labeled as evidence-based, the following steps are required:

1. Systematically identifying all applicable evidence
2. Systematically selecting the best available evidence from that identified
3. Systematically evaluating the selected evidence (critical appraisal)
4. Objectively reporting the relevant findings and quality of the evidence
5. Synthesizing multiple evidence reports
6. Deriving overall conclusions and recommendations from the evidence synthesis
7. Changing the conclusions when new evidence alters the best available evidence

Editors use these seven steps to consider medical literature for inclusion in DynaMed, an online clinical EBM resource for clinicians at the point-of care. But simply stating that seven steps are needed isn’t enough to be assured that the best available evidence is being presented. Each step needs definition, clarification, and process. Outlining the editors’ process through these steps will help illustrate what is required behind the scenes for clinicians to practice EBM.

To systematically identify all applicable evidence in Step 1, an extensive set of current literature is monitored daily. Systematic literature surveillance (SLS) is conducted using more than 500 journals directly or indirectly through many journal review services and other sources of systematic evidence evaluation.

For each source monitored, each issue is reviewed cover-to-cover. All entries are considered because information in letters to the editor, editorials, and “reporting from the literature” pages may contain reports of new research that would be otherwise unidentified if relying exclusively on abstracts posted with traditional research articles.

When adding a new topic or critically revising an existing topic, PubMed Clinical Queries is used to provide systematic searches for identification of the best available evidence. In addition, numerous sources are searched for evidence-based reviews (such as Cochrane Database of Systematic Reviews), for guidelines (such as National Guideline Clearinghouse), and for traditional reviews.

Editors then move to the second step in the process. Each article is assessed for clinical relevance and each relevant article is further assessed for validity relative to existing content. The most valid articles are summarized, the summaries are integrated with content, and overview statements and outline structure are updated based on the overall evidence synthesis.

Determining clinical relevance is the first consideration in systematically selecting the best available evidence from that identified. The relevance of medical information is different for every user. DynaMed is used in clinical care by practitioners with a wide range of experience and interests, and is also used in medical education. When adding information, the editors consider several questions to determine relevance.

Does this information have a direct bearing on patient-oriented outcomes? Patient-oriented outcomes are outcomes that affect quality of life without extrapolation. Examples include mortality, incidence of myocardial infarction, and presence and severity of pain. These are also called clinical outcomes. Disease-oriented outcomes are used as surrogate markers for monitoring the effects of interventions ultimately intended to affect patient-oriented outcomes. Examples include cholesterol concentration, blood pressure and bone mineral density. Patients are only interested in these outcomes as a means for affecting clinically significant outcomes such as mortality or fracture incidence. Because DynaMed is primarily a clinical tool for use during patient care, patient-oriented outcomes information is considered relevant and included. Patient-oriented evidence is given priority over disease-oriented evidence, with disease-oriented evidence entered only if it adds substantially new information.

In the absence of patient-oriented evidence, might this information be useful in clinical decision-making? Much of medical knowledge is still lacking in terms of patient-oriented outcomes research. Clinical decisions based on extrapolated disease-oriented evidence are not proven to be appropriate. However, clinicians still need to make decisions in situations where patient-oriented evidence is not yet available. Disease-oriented evidence is considered relevant for inclusion in situations where patient-oriented evidence is lacking. Individual clinicians will have to determine if this information is considered relevant within their practice. When disease-oriented evidence is not presented as such in the supporting reference, commentary will be added and may appear as “patient-oriented outcomes not assessed,” “clinical outcomes not assessed,” or specific commentary pointing out problems with extrapolating the information to clinical care.

Is this information part of a clinical controversy? In situations where the evidence does not clearly support or refute a clinical fact, opposing views are presented. DynaMed is not designed to resolve clinical controversies and strives to present information with as little “inappropriate” bias as possible. The inherent bias towards patient-oriented outcomes is considered appropriate. Information that questions “standard” approaches and has a potential bearing on patient-oriented outcomes is considered relevant for inclusion.

Is this information that is of unique interest due to popularity? Some medical information is not clinically relevant but widely publicized. Summarization of this type of information (often with commentary) is relevant to users if it is likely that clinicians will be asked about it during clinical encounters. It is important for physician and patient education to point out where this type of information is not clinically applicable.

Clinically relevant articles must be assessed to determine the scientific validity of conclusions and facts presented before consideration for use. Conducting critical appraisal for all articles would be wasteful if these articles would not make a change to the existing knowledge base.

Easily identifiable study features (e.g., study method, sample size) are compared with existing studies in current content to determine if new articles potentially represent the best available evidence. Articles that do not provide relevant information with validity that meets or exceeds the existing content are excluded at this stage.

In Step 3, editors have found that abstracts in research publications often do not accurately reflect the methodologic quality and results found in full-text articles. Article summaries in other publications often do not accurately reflect the methodologic quality and results found in full-text articles.

Full-text evaluation of articles is required for:

• Any article rated as Level 1 [likely reliable] evidence or Grade A recommendation [consistent high-quality evidence] (1)
• Any article potentially ratable as Level 1 or Grade A based on abstract-only information; full-text evaluation is necessary to provide lower levels or grades
• Any article for which definition of absolute magnitude of effect and/or detailed description of interventions or exposures are necessary, regardless of level of evidence
• Any article which represents the most important guidance for a topic, regardless of level of evidence

Reports used for updating content represent the best available evidence for the specific content under consideration. Evidence may be labeled in one of three levels: Level 1 (likely reliable) evidence; Level 2 (mid-level) Evidence; Level 3 (lacking direct) Evidence) (1). Articles that potentially warrant the highest evidence ratings undergo complete critical appraisal using methods established in the Users’ Guides to Evidence-based Practice from the Evidence-Based Medicine Working Group (2). If serious methodological shortcomings are discovered (sufficient to affect clinically relevant results), then the evidence is labeled as mid-level evidence and the shortcomings are described.

When reporting the evidence, editors consider all of the following as they go through Step 4

• Were all relevant outcomes reported in the original article?
• What are the most relevant outcomes to report in the topic?
• For relevant outcomes, what is the magnitude of effect? This may be represented by absolute rates and number needed to treat (NNT) or harm (NNH) abbreviations, or by absolute differences in continuous variables (e.g., mean decrease in 1.3 points on 0–10 visual analog pain scale).
• Were the findings clinically significant?
• In the case of no statistically significant differences, were the findings robust enough to rule out clinically significant difference?
• Are there any methodologic limitations sufficient to alter reliability of clinical conclusions?In Step 5, evidence-based summarization of articles is necessary, but insufficient for a point-of-care reference. Evaluating individual evidence reports requires synthesizing multiple evidence reports.

Addition, deletion, and organization of information within content is done with consideration of levels of evidence. When multiple articles are present on the same topic, preference for inclusion and organization is based on the quality of methodology, e.g., preference given to data derived from randomized controlled trials over data from prospective observational studies, which is given preference over retrospective studies, which is given preference over anecdotal reports. When data of lesser quality does not add any substantially new or different information, this data is then deleted from content.

Moving to Step 6, deriving overall conclusions and recommendations from the evidence synthesis is required for a comprehensive point-of-care reference. Multiple evidence reports of similar quality are organized such that the overall conclusions quickly provide a synthesis of the best available evidence.

In DynaMed, treatment overviews (the ultimate synthesis of evidence for a clinical topic) are based upon all of the available evidence in the treatment section, and selection of the most important concepts. As new topics are created and existing topics are critically revised, treatment overviews are explicitly linked directly to the supporting evidence synthesis.

The final step in DynaMed’s evidence-based methodology is changing conclusions when new evidence alters the best available evidence. This step is crucial because new evidence is published every day. Having new evidence summaries handled separately from reviewed content in a manner requiring the clinician to search in two locations to synthesize the entire story would make finding the best available evidence more difficult.

As soon as new evidence is evaluated using the six steps governing systematic processing, it is added to the appropriate topic(s) in context. This process allows immediate and comprehensive access to the best available evidence as it occurs.

In conclusion, while it is true that the definition of “evidence-based” can be confusing and is often misused, when the term is correctly used, and the evidence in an EBM content source is accurately and consistently derived from the best available evidence, clinicians can use that content source to make decisions that enable them to provide the best care to their patients at the point-of-care.

Resources:

1. Definitions and sources available at www.ebscohost.com/dynamed/levels.php.
2. Available from the Centre for Health Evidence at www.cche.net/usersguides/main.asp.

Visit DynaMed Online

DynaMed, a leading point-of-care clinical reference tool, is available to all healthcare professionals at www.ebscohost.com/dynamed. Subscribers to the site are provided with the most up-to-date, evidence-based information gathered from over 500 medical journals and evidence review databases that will assist them in making the best clinical decisions when it is needed most. Updated daily, the data presented on DynaMed’s website is thoroughly reviewed for scientific relevance and validity, and then integrated with existing content to produce the best available evidence on various health-related topics. Residents, medical students, practicing physicians, and medical scholars looking for answers to complex clinical questions are invited to subscribe to this site.