1.0 DESCRIPTION and BACKGROUND
1.1 EBM is characterized as use of evidence to resolve clinical problems
on diagnosis and therapy. The evidence is from high quality randomized clinical trials. All relevant trials are considered
and the evidence is analyzed systematically to assess its validity, precision, relevance, and usefulness. This yields the
‘best evidence’ that is applied to solution of a specific clinical problem.
1.2 EBM involves systematic review that starts with defining a clinical
problem requiring a solution. This is followed by a systematic search for the relevant
evidence. The evidence is then summarized and is appraised critically before formulating clinical decisions.
1.3 EBM empowers the physician to search for evidence that is most
relevant to the specific clinical situation he is dealing with instead of relying on clinical practice guidelines that were
developed in the institution. It thus enables individualization and customization
of clinical decision making. It enables the physician to deal with primary sources of evidence instead of relying on secondary
1.4 In the past 15 years EBM has changed the practice of medicine.
This was mainly due to developments in information technology that allowed busy clinicians to access research information
easily and quickly. There are now many computerized data bases containing a lot of published and unpublished evidence from
many countries and centers. It was also helped by increasing interest of clinicians in analyzing and interpreting research
1.5 Before introduction of EBM, it was observed that drug use in many cases did not have strong scientific evidence
to back it up. Some patients received unneeded therapy and procedures whereas others received inadequate or appropriate treatment.
Physicians in their clinical decision making depended on selective and personal knowledge of the literature, personal or institutional
experience, practice guidelines or clinical protocols, intuition, unsystematic personal experience, and patho-physiological
reasoning. They also used secondary evidence from textbooks and not primary research publications. This happened despite availability
of a lot of published evidence in the medical literature based on randomized clinical trials indicating the best treatments.
This behavior could be excused if there was no research evidence or if the evidence was contradictory or unclear. It could
not be excused with the existence of high quality research evidence.
1.6 EBM evolved since the early 1990s to address these problems. It has resulted in major changes in physician
behavior and perhaps improved patient care. EBM enables integration of research directly into clinical practice. It closes
the gap between scientific evidence and actual clinical practice. It is essentially
translation of research evidence into patient care. In a sense EBM is not new, scientific medicine has always been evidence-based
though in the past the evidence was limited and was of questionable quality. The only new dimension in EBM is systematic and uniform use of evidence directly from research findings.
2.0 LITERATURE SEARCH
2.1 Literature search is made easy by availability of many medical databases
easily accessible on-line for free of for a small fee. Documents of medical importance are usually journal articles, books,
technical reports, or theses. The sources of on-line documents are MEDLINE, on-line journals, on-line books, on-line technical
reports, on-line theses and dissertations.
STATING THE PROBLEM
2.2 The first step in the search is to define the clinical problem clearly.
We should take time to do this well. The problem should be written down in the form of a question. The first step is to identify
the variables of interest such as outcome, exposure, confounder, intermediate, and effect modifying variables. The variables
are used together with other relevant key words to identify relevant articles from the data bases. Care should be taken to
include the main key words in the formulation of the question because these key words will be the basis for the computer search.
LIST THE SEARCH KEY WORDS
2.3 Generally the most important key words should be in the question. However there are related secondary key words
that refine the search. Usually search programs help identify these secondary key words on entry of the primary key words.
Authors of articles often give the key words that they consider important for retrieval and this is very useful. We also need
to be creative with regard to the language of the key words. We can include the selected key words as well as their synonyms,
antonyms, homographs, and homonyms. The thesaurus can help in the retrieval because it gives synonyms and antonyms of words.
Homographs are words that have the same spelling but different meanings. Homonyms are words with the same sound but different
spellings. There are about 250-300 common grammatical words that account for 50% of any text such as the, of, and, to a, in
etc. These have to be excluded from queries with little loss of efficiency. Stemming algorithms remove the ends of words and
leave only the roots.
2.4 Criteria must be set for what articles to include or exclude. These include definition of what effect measures
are used and the populations covered. It is best if there is as much homogeneity in the population as is practical.
2.5 The search is submitted as a query. A query is a short document
used to retrieve larger documents by use of Boolean logic (AND, OR, NOT). The search can be limited by type of journal, date
of publication, type of publication (review, randomized controlled trial, meta analysis, practice guidelines), Age, gender,
language, and special subsets (cancer, AIDS, nursing etc). The retrieval must be reviewed. Despite careful selection of key
words, many titles that appear are not relevant and must be eliminated. This can be a tedious exercise if the topic is popular
and hundreds of titles are retrieved.
3.0 OBTAINING THE BEST EVIDENCE
3.1 There is a gradation in the quality of evidence used in EBM. The best is the randomized clinical trials (RCT).
Where this is not available, systematic reviews of RCTs can be used. Resort can also be to observational studies and other
types of studies and observations.
RANDOMIZED CONTROLLED TRIALS (RCT)
3.2 The prospective randomized double blind controlled trial is the gold standard of evidence. It can be carried
out in 2 forms. A study comparing drug to placebo is of higher quality than a study comparing a drug to another drug.
3.3 Systematic Review of several RCTs can yield useful evidence. Systematic reviews can
be carried out in 2 ways. Use of meta-analysis is of higher quality that qualitative review by an expert in the field.
3.4 Review articles written by discipline leaders were the most popular method of combining
findings from various studies. They however had two serious draw-backs. They were subjective and therefore prone to error
and bias. The reviewer was free to make decisions on what data and conclusions to emphasize. Secondly they dealt with summarization
of conclusions without looking at the data on which the conclusions were based.
3.5 Meta-analysis has become popular with the proliferation of studies on particular subjects.
Clinicians would like to have some form of consensus or summary of the findings of various studies. Meta analysis enables computation of an odds
ratio or risk ration for a larger number of study subjects thus enabling picking up statistical significance that would be
missed if analysis was based on small individual studies. Many clinical trials especially with invasive intervention cannot
recruit enough patients in one center to reach statistical significance. Meta analysis also enables
study of variation across several population subgroups since it involves several individual studies carried out in various
countries and populations. Meta analysis makes the process of reviewing several studies with view to
reaching a general conclusion very transparent because it is based on quantitative assessments.
3.5 Observational studies such as cohort studies, case control studies, point prevalence studies can be used in
the absence of good RCTs. There are situations when these studies are superior to poorly conducted RCTs.
In the absence of clinical or observational studies, resort can be to other types of evidence of lower quality
on the premise that any evidence is better than no evidence at all. In N-of-one trial one patient is used as his own control.
He is observed before and after taking a given trial medication. Non-randomized trials or non-controlled trials can be used
but care should be taken in their interpretation because of potential bias due to confounding and selection. Studies that
rely on historical controls are not as reliable as those using contemporary controls. The worst forms of evidence are retrospective
analyses, non systematic reviews, and case reports or anecdotal observations
4.0 CRITICAL READING AND ASSESSMENT OF JOURNAL ARTICLES
4.1 Before a journal article can be used in EBM, it must be read critically
and assessed as to validity. Only a select few that fulfill given criteria of validity
should be used. Each section of the article must be analyzed for the following common problems.
4.2 The main problem of the title
is irrelevance to the body of the article. Problems of the abstract are failure
to show the focus of the study and to provide sufficient information to assess the study (design, analysis, and conclusions).
Problems of the introduction are failures of the following: stating the reason
for the study, reviewing previous studies, indicating potential contribution of the present study, giving the background and
historical perspective, stating the study population, and stating the study hypothesis.
4.3 Problems of study design
are the following: going on a fishing expedition without a prior hypothesis, study design not appropriate for the hypothesis
tested, lack of a comparison group, use of an inappropriate comparison group, the Berkson fallacy, selection of cases and
controls from different populations, and sample size not big enough to answer the research questions.
4.4 Problems in data
collection are: missing data due to incomplete coverage, loss of information due
to censoring and loss to follow-up, poor documentation of data collection, and methods of data collection inappropriate to the study design.
4.5 Problems of data analysis
are failures in the following: not stating type of hypothesis testing (p value or confidence interval), use of the wrong statistical
tests, drawing inappropriate conclusions, use of parametric tests for non-normal data, multiple comparisons or multiple significance
testing, assessment of errors, assessment of normality of data, using appropriate scales and tests, using the wrong statistical
formula, and confusing continuous and discrete scales.
4.6 Problems in reporting results
are: selective reporting of favorable results, numerators without denominator, inappropriate denominators, numbers that do
not add up, tables not labeled properly or completely, numerical inconsistency (rounding, decimals, and units), stating results
as mean +/- 2SD for non-normal data, stating p values as inequalities instead of the exact values, missing degrees of freedom
and confidence limits.
Problems of the conclusion
are failures in the following: repeating the results section, discussion of the consistency of conclusions with the data and
the hypothesis, extrapolations beyond the data, discussing short-comings and limitations of the study, evaluation of statistical
conclusions in view of testing errors, assessment of bias (misclassification, selection, and confounding), assessment of precision
(lack of random error), and assessment of validity (lack of systematic error).
Validity both internal
and external is involved in EBM. While reviewing an article we should first consider internal validity. Internal validity
is achieved when the study is internally consistent and the results and conclusions reflect the data. External validity is
generalizability (i.e. how far can the findings of the present study be applicable to other situations) and is achieved by
several independent studies showing the same result.
4.7 Abuse of statistics is by incomplete and inaccurate documentation
of results as well as selection of a favorable rate and ignoring unfavorable ones. This is done by 'playing' either with the
numerator or the denominator. The scales of numerators and denominators can be made artificially wider or narrower giving
false and misleading impressions. Statistical results are misleading in the following situations: (a) violating the principle
of parsimony, (b) study objective unclear and not reflected in the study hypothesis (c) fuzzy, inconsistently, and subjective
definitions (of cases, non-cases, the exposed, the non-exposed, comparison groups, exposure, method of measurement), (f) incomplete
information on response rates and missing data.
5.0 META ANALYSIS
5.1 Meta analysis refers to methods used to combine data from more
than one study to produce a quantitative summary statistic.
5.2 Data collection is carried out by abstracting information from the articles on a standardized
data abstract form with standard outcome, exposure, confounder, or effect modifying variables. The ideal is to abstract raw
data and reanalyze it in a standard way to enable combination of several studies to get one summary effect measure. In practice
the raw data is not available and only effect measures (odds ratio and risk ratio) are available.
5.3 The first step is to display the effect measures with their 95% confidence limits
to get a general idea of their distribution before proceeding to compute summary measures. The summary effect measure, OR
or b, is computed from the effect measures of individual studies. The combined effect measure is then statistically
adjusted for confounding, selection, and misclassification biases.
DIFFICULTIES OF META ANALYSIS
5.4 Meta analysis is methodologically complex because of different study designs,
study analysis, and even different data quality. The major problems are: over-conclusion and bias (publication bias, selection
bias), and use of wrong methods. Over-conclusion arises when a conclusion is artifactual and is not supported by the aggregated
data. The results of meta-analysis based on published sources may not reflect the true situation because of existence of publication
bias. Positive findings are more likely to be published than negative ones. Studies carried out in academic or government
institutions are thought to be more credible and are therefore more likely to be published whereas studies by pharmaceutical
firms have a lower publication rates. Inadequate search for reports may lead to bias just as multiple publications of the
same study data leads to bias. Bias, conscious or unconscious, may occur in the selection of studies for analysis. In some
cases the methods used for meta-analysis are wrong or inappropriate.
READING RESULTS OF META ANALYSIS
5.5 The following questions should be considered when reading review or meta-analysis articles: are the methods
clearly stated?, was the search for articles comprehensive enough?, were the criteria for selecting articles for review stated?,
were the criteria objective and were they adhered to?, was there a possibility of bias in the selection of those articles?, was the methodologic quality of each article assessed?, were differences between studies
explained or were they just glossed over?, was the combination of results from
the primary studies appropriate?, and were the conclusions of the reviewer supported by data?
6.0 PROBLEMS AND ETHICAL ISSUES OF EBM
PHYSICIAN AUTONOMY AND FREEDOM
6.1 Before EBM the physician could make autonomous decisions and could not be questioned because they were independent
professionals. Under EBM the physician is guided to decisions by the analysis of evidence from published literature. There
is both loss of autonomy and also diminished responsibility. Yet the literature could be wrong or could have been done unethically.
BARRIERS TO EBM
6.2 Many physicians especially those with no training in research or quantitative analysis are skeptical about
EBM and its benefits. Some physicians may not have the time to research the literature. Many of them do not have the training
necessary to read and interpret the literature. In many cases the literature is not easily accessible.
FEAR OF LITIGATION
6.3 Some physicians would rather stick with established clinical guidelines and protocols rather than venture into
EBM for fear of litigation. In case of any medical mishap they can use the defense that they followed established guidelines.
If there is a mishap following use of EBM it is their individual interpretation and use of the evidence that is called into
COST ANALYSIS ISSUES
6.4 Economic considerations may also play a role in EBM. Some institutions may refuse to use therapies shown by
EBM evidence to be effective because they are more expensive.
ETHICAL ISSUES6.5 Use of EBM should
not be mechanical. There are ethical, social, and psychological aspects that have to be considered in choice of treatment.
Thus EBM is one of many contributing factors to making a clinical decision. The evidence from EBM may not be applicable to
the patient because it was collected from a population with different characteristics. Even if the patent is from the same
base population, he may have individual peculiarities that have to be considered.