Quick Picks
Why Critical Appraisal Matters
Services
Seminars
Books
Updates & Contact Info
Free Online Tools
Free Online Tutorial
Delfini Blog
Follow & Share...
Just-in-time Updates
Like Us Find Us |
DelfiniGram™: GET ON OUR UPDATE LIST
Volume
— Use
of Evidence
Quality of Care — From Variation to Improvement
Newest
03/09/2015: Biosimilars & FDA's New "Purple Book"
03/02/2015: Genomic Medicine Leaps Forward—More Drugs Targeting More Cancers
Contents
Go to DelfiniClick™
for all volumes. |
The
Volume of Inappropriate Care in the US
Could things be
getting worse in the area of clinical quality? We have seen
estimates of 20%-30% inappropriate care in the past, but in
the May/June 2004 issue of Health Affairs (Vo 23;3: 247-255),
Kerr et al. report that in a study of approximately 7,000 people
in 12 metropolitan areas in the United States, on average, residents
received only 50% to 60% of recommended care (recommended care
was based on 439 quality indicators across thirty conditions
and preventive care recommendations developed by RAND and validated
by four multi-specialty expert panels using a modified Delphi
method). And this doesn't even address the other side of inappropriate
care — overuse. We continue to believe the answer lies
in organizations creating a system of evidence-based solutions
for care.
Full text is available
to subscribers and can be obtained for a fee for non-subscribers
at http://www.healthaffairs.org/
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
More
On Therapies Becoming “Routine Clinical Practice”
Without Evidence & Why We Are Doing This Work: Radiofrequency
for the Treatment of Gastro-esophageal Reflux Disease Critical
Appraisal Example
04/01/08
Several years ago, we received a phone call from a medical director asking if we would do a limited review of radiofrequency for the treatment of gastro-esophageal reflux disease (GERD in the US; GORD in UK). He had received a request for coverage of this procedure by a physician and was concerned that there was not good scientific evidence supporting use of this procedure. His health plan policies required proof of efficacy.
We agreed. We contacted the requesting physician and asked him to recommend his best three studies for review. (He pleaded for more - we relented and said, “Okay, four.”) Of the 4 studies he sent us, only 1 was a randomized controlled trial (RCT), and so we had to surmise that if he had been given permission to send us even more, those additional studies, too, would have been observational.
Observational studies should not be used for addressing questions of efficacy of interventions with rare exception.[1,2] The only exception that is thus far agreed upon is when all-or-none results are achieved and are likely to be reliable, and this is extremely rare.
And one RCT generally will not be considered sufficient to declare “proof.” Even a meticulously designed, executed and reported RCT may still result in findings merely due to chance, not to mention that only through replicated studies might one discern the unfortunate occasional fraud. And this RCT could not even come close to being a candidate for definitive proof of anything. While thoughtfully designed by investigators more knowledgeable of research design than many we have seen, it was very small and had myriad problems as our critique—reproduced below—demonstrates.
Even the authors - very responsibly - themselves acknowledged that their study should only be considered preliminary and to have proved nothing (a point that the requesting physician seemed to have overlooked in his desire for a solution for this problem that he, himself, believed in).
And now, in The Lancet, Mehran Anvari[3] of St Joseph’s Healthcare, Hamilton ON, Canada, comments upon a recently published review which considers clinical trials from the past three decades on the use of endoluminal therapies for GERD/GORD.[4]
Anvari writes that the systematic reviewers found that “…no study of any techniques has provided adequate ‘scientific and clinical data on their safety, efficacy and durability to support the use of endoluminal therapies for GORD [gastro-oesophageal reflux disease] in routine clinical practice.’ Thus we could be left with a sense of pessimism about the future use of endoluminal therapies for this common disease, at a time when similar endoluminal approaches are being proposed for other common disorders, such as morbid obesity.”
He tells us that the reviewers, “…also highlight the surprising lack of well-designed randomised trials, even though many of the endoluminal therapies have been in clinical use for several years and many of the trials of these techniques have come from respected academic centres.” He goes on to say, “It almost seems that the standards established for the assessment of treatment efficacy in the control of GORD… were lowered [emphasis ours] when it came to assessing these approaches.”
He also presents the sobering news that, “As a result, many patients have undergone procedures that have since been abandoned and proven ineffective, and an unfortunate minority has had substantial side-effects or even died.”[5-9]
Indeed. And how many patients may have been told that research studies have shown benefits of this treatment? Isn’t it possible that some patients would not have elected to undergo this procedure if they had been told, what Anvari states—most correctly—in his commentary, “…up to this point, there is no clear evidence to support the use of any of the endoluminal therapies…in routine clinical practice.”
This is what we advocate:
- All health care professionals who care for patients need to recognize that the vast majority of published research in even the best medical journals is so flawed in design, execution and/or reporting, or is not clinically useful, that it should not be used to inform health care decision-making. Patients’ quality of care—and even their lives—depend upon this understanding. Of the approximately 12,000 articles published in the medical literature each week, it is our estimate, having evaluated thousands of published studies, that fewer than 10 percent of the published studies in health care are both valid and clinically useful, while others have estimated that the number is less than 5 percent.[10]
- And most users of the medical literature lack the skills to tell. The vast majority of physicians, clinical pharmacists and nurses fail our simple critical appraisal pre-test. (Read more shocking facts here.)
- It is incumbent upon every such professional to attain the skills to discern valid and clinically useful research from that which is not.
- This means that—
- Health care researchers need these skills before subjecting patients to medical research.
- Medical editors, peer reviewers and health care writers need these skills in order to avoid passing on the flood of misleading information. It is one thing to publish work that may legitimately raise hypotheses. We, personally, think it is perilous to publish work (and even perform research) that is clearly severely flawed.
- Publishers also need to change their editorial policies which frequently emphasize “discussion” over “reporting of methods.” Users need to see methodological details to determine whether they agree that the study is valid. It is not sufficient to tell a reader that a study was blinded. Readers need to evaluate the likelihood of success of blinding—and only details will do. Transparency is vital.
- Physicians, clinical pharmacists, nurse practitioners and others directing or influencing patient decision-making need to attain the skills to evaluate the health care information put before them.
- Leaders need to attend to this urgent need and provide the appropriate culture and work elements, such as structures, processes, methods, staff roles, skills and tools for evidence- and value-based practice.
- Professional societies, advocacy groups, experts writing reviews, systematic reviewers, guideline developers and others who create secondary studies and secondary sources need to develop these skills before producing “guidance.”
- Policy makers and payers need to be aware of the huge problems in the quality of health care information and attain a true understanding of the evidence before dictating sweeping changes with far reaching impact that may only contribute to the cost and quality problems in health care.
- Manufacturers need to gain an understanding of what makes for valid and clinically useful science and how it needs to be reported.
- Legal professionals need to become responsible and help raise the bar for evidence that is valid and clinically useful.
And much of it all starts in the schools. Academicians must gain these skills themselves. And they must strive to identify ways to successfully inculcate evidence-based practice knowledge and habits in their students.
While some would say if there is no evidence of benefit, don’t do it, we, personally, do not wish to tell others how to practice medicine. That is not what we see our role to be. No evidence of benefit is not the same thing as evidence of no benefit. And it is possible that something for which there is no evidence works. (In fact, some topics are so inherently difficult to study that we will never have good evidence—and this doesn’t mean that practitioners will stop trying to find solutions and offering options to patients.)
We do, of course, advocate that a science-based approach be taken in all possible instances. It is important to try and study—correctly—cause and effect for interventions. It is important to report findings responsibly. It is important that all users of such information have skills to evaluate science and that they do so.
Know and clearly state the evidence first! Then decisions can be made taking into account other factors.
However, it is our strong belief that patients should be provided with honest information to enable them to make choices that are in their best interest in accordance with their values and preferences. This requires that all working in health care decision-making acquire the appropriate skills. There are many, many resources available to help achieve this — including information we make freely available on our site from our own work and the work of others.
Our patients deserve this—and indeed—their lives depend upon it. And that is why we are doing this work.
Improvement
of gastroesophageal reflux symptoms after radiofrequency
energy: a randomized, sham-controlled trial. Corley DA,
Katz P, Wo JM, Stefan A, Patti M, Rothstein R, Edmundowicz
S, Kline M, Mason R, Wolfe MM. Gastroenterology. 2003
Sep;125(3):668-76. PMID 12949712
Review
& Critique by Sheri A. Strite & Michael E. Stuart
MD, Delfini Group, Original Review January 14, 2006 and
Subsequently Updated
Commentaries
in PubMed: See editorial by Kahrilas (Kahrilas
PJ. Radiofrequency energy treatment of GERD. Gastroenterology.
2003 Sep;125(3):970-3. No abstract available. PMID: 12949740).
Editorial urges caution until further studies become available.
Issues
with Study
Number
of People in Study (N): Small N—small N
may be especially prone to non-representation with population
at large.
Inclusions:
It appears that patients who may still have benefited
from proton pump inhibitors (PPI) and H2 blocker treatment
(i.e., those who had not failed medical therapy) were
allowed to participate in the study.
Randomization:
Sequential randomization method was used which is a weaker
method more prone to “gaming” especially when
concealment of allocation is not adequate. There is no
mention of effective concealment of allocation methods
being used which would prevent channeling patients to
desired groups, such as use of locked sealed-containers
or a 1-800 call-in center, as examples. One patient (active
treatment group) was determined to be ineligible post-randomization
(see discussion in Intention-to-Treat Analysis or ITT
analysis).
Baseline
Characteristics: Authors state that there
was a slight difference in age and PPI use between groups.
However, upon visual comparison the groups appear equal
for demographics and disease dimensions. Plus, this is
suggestive that randomization was successfully achieved.
Blinding:
Authors state study is double-blinded, however, no details
of blinding were reported excepting to state that staff
blinded to the treatment assignments performed subsequent
outcome assessments. The endoscopist opened the envelope
to determine assignment during the endoscopy. Of some
concern is that the authors state that that “subsequent
contact between the subject and the endoscopist was minimized…”,
but it was not subsequently prevented—See Intention-to-Treat
Analysis. The only outcome that remained statistically
significant through the toughest test was subjective (health-related
quality of life) which could have been influenced if patients
had any direct, indirect or even a subliminal idea they
had received radiofrequency energy treatment (RF). Authors
themselves state that they “cannot exclude a possible
bias from subjects guessing their treatment assignments
because of difference in post-treatment symptoms between
the active and sham groups.”
Statistical
Analysis: No analysis of statistics was performed
by Delfini.
Loss
of Data Points: Significant loss of data points.
However, authors performed sensitivity analysis using
ITT methods including an imputation method that put the
intervention through the toughest test. In that test,
health-related quality of life remained statistically
significant. However, there are no details reported of
the actual outcomes in this analysis—further see
comments that potential subsequent contact with endoscopist
could have affected subjective measurement of patients.
Analysis
Methods Used: Authors did not use Intention-to-Treat
(ITT) analysis as their primary analysis method. They
performed a completer analysis as the primary method—and
due to loss to follow-up, these
results are not dependable. The authors, however, also
did a sensitivity analysis that included a last-observation-carried
forward (LOCF) method plus an extreme-case analysis (see
below for ITT discussion). LOCF has been shown to be prone
to bias. [Reminder to readers: extreme-case analysis puts the intervention under study through the toughest
test: one assigns “failure” as an outcome
to missing values in the intervention group and “success”
to missing values in the comparison group. Statistical
significance is then calculated. If results are statistically
significant in an otherwise valid study, we can conclude
efficacy of the treatment.]
Intention-to-treat
Analysis & Imputation Methods: Authors state
that they performed sensitivity analyses on “all
patients” using LOCF and extreme-case analysis—putting
the intervention through the toughest test. It would be
preferable to see the numbers used and outcomes obtained
rather than summary statements; however, overall the authors
have demonstrated sophistication and a great deal of clarity
in reporting this study (including ITT methods)—indicating
knowledge with key methodologic principles.
Eligibility
was determined after randomization, further contributing
to patient loss—this, however, amounted to one patient,
and was 3.5 percent. Because the actual number of “all”
patients was not reported, it is not possible to tell
if this patient was included in the various ITT analyses.
However, this is less than 5 percent and so probably would
not significantly affect study results. Delfini reviewers
are most focused on results of sensitivity analysis as
these were ITT analyses. LOCF seems reasonable for imputing
missing values for this condition as a sensitivity analysis,
and extreme-case scenario reporting is always appropriate
as it presents the most severe challenge to test statistically
significant differences between groups. Authors report
statistically significant improvement in the primary outcomes
of heart burn symptoms (LOCF and extreme-case) and quality
of life (extreme-case). There were no differences in any
analysis in daily medication use or esophageal acid exposure
times—however, study was not be powered to detect
differences between groups in these outcomes, and so we
do not know if “no difference” between groups
is true or if they didn’t have enough people to
show a statistically significant difference if there was
one, in fact.
Mechanism
of Action: Per editorial, it is uncertain whether
the results are due to reduced gastroesophageal reflux
(paper suggests not) or are due to diminished esophageal
sensitivity.
Clinical
Significance: This study does not provide sufficient
evidence to conclude that radiofrequency ablation is effective
and safe. Because of the above-mentioned threats to validity
and the patient population, conclusions regarding symptom
relief and improved quality of life should be drawn cautiously.
Further confirmatory studies that include patients who
have not responded to H2-blockers and PPIs would be useful.
Undetected long term harms remain a possibility. Patients
should be aware that the medical evidence is sparse and
that uncertainty regarding the appropriate role of radiofrequency
therapy in the treatment of gastroesophageal reflux symptoms
remains.
Safety
- There were more adverse effects (ADEs) reported in the
active treatment group than in the sham group—no
p-values are reported. Specific ADEs were not determined
a priori which increases the likelihood that any reported
outcomes are due to chance.
- One
active treatment patient with an esophageal ulcer had
self-limited bleeding and had inpatient observation
overnight.
- A
patient in the sham group developed pneumonia and received
inpatient antibiotics; the pneumonia was thought to
be due to aspiration during the endoscopy.
- One
active treatment patient experienced increased bloating;
a nuclear medicine gastric emptying study showed worsening
of a pre-existing delayed gastric emptying disorder.
- Some
active and sham patients experienced temporary post-procedure
retrosternal discomfort that required oral analgesics.
These transient symptoms were not fully quantifiable
with the measurement instruments.
Other
- In
many respects, this is a very well done and very well
reported study. The authors clearly have understanding
of research excellence in design and execution.
- A
study such as this cannot be sufficient to “prove”
a therapy’s efficacy. Proof is not possible with
main issues identified here including small N in a single study, questions
about maintenance of blinding, plus a need for assurance
that analyses were truly done as intention-to-treat
with all randomized patients especially given the subjective
measures of heartburn and health-related quality of
life. Furthermore, even in this research setting, the
reported benefits are minimal and extrapolating benefits
to clinical practice should be done with caution. Without
confirmatory studies, radiofrequency therapy for GERD
symptoms should be considered unproven. However, for
a small study, this is a very thoughtfully-designed,
apparently well-executed study as reported. It is suggestive
that there may be benefits from this procedure. Further
research is clearly needed to determine if this treatment
can help patients avoid more invasive and risky procedures.
|
References
1. Delfini Primer: Problems with Case Series
2. Delfini Primer: Problems with the Use of Observational Studies to Draw Cause and Effect Conclusions About Interventions
3. Anvari, M. Endoscopic treatments for gastro-oesophageal reflux disease. [Comment] The Lancet. Vol 371. March 22, 2008. 965-966
4. Fry LC, Mönkemüller K, Malfertheiner P. Systematic review. Endoluminal therapy for gastro-oesophageal refl ux disease: evidence from clinical trials. Eur J Gastroenterol Hepatol 2007; 19: 1125–39.
5. Pleskow D, Rothstein R, Lo S, et al. Endoscopic full-thickness plication for the treatment of GERD: a multicenter trial. Gastrointest Endosc 2004; 59: 163–71.
6. Noh KW, Loeb DS, Stockland A, Achem SR. Pneumomediastinum following enteryx injection for the treatment of gastroesophageal refl ux disease. Am J Gastroenterol 2005; 100: 723–26.
7. Wong RF, Davis TV, Peterson KA. Complications involving the mediastinum after injection of Enteryx for GERD. Gastrointest Endosc 2005; 61: 753–56.
8. Tintillier M, Chaput A, Kirch L, Martinet JP, Pochet JM, Cuvelier C. Esophageal abscess complicating endoscopic treatment of refractory gastroesophageal refl ux disease by enteryx injection: a fi rst case report. Am J Gastroenterol 2004; 99: 1856–58.
9. Tuebergen D, Rijcken E, Senninger N. Esophageal perforation as a complication of EndoCinch endoluminal gastroplication. Endoscopy 2004; 36: 663–65.
10. Ebell M. An introduction to information mastery, July 15, 1988. http://www.poems.msu.edu/InfoMastery/default.htm. Accessed December 21, 2007.
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message.
|
Reversal of Established Practices—Unringing the Bell
01/24/2012
Many healthcare practices become established before there is sufficient evidence to conclude that the intervention is beneficial and that benefits outweigh harms. Reversal of established practices has been recently discussed by Prasad, Gall, Cifu and Ioannidis [1, 2]. Abandoning a practice because evidence demonstrates that the practice is not beneficial is an important issue and frequently is very slow to happen.
Prasad defines reversal as the phenomenon of a new trial—superior to predecessors because of better design, increased power, or more appropriate controls—contradicting current clinical practice [2]. Prasad and colleagues provide a very useful window through which we can see from yet another vantage point the problem of overuse of unproven healthcare interventions and the difficulty of withdrawal. These investigators provide numerous examples of recent reversals where practiced has changed—hormone replacement therapy to prevent second myocardial infarctions (MIs) in women (although there has been renewed used of HRT in younger women based on what appears to us to be evidence at high risk-of-bias), routine use of flecainide and encainide to treat premature ventricular contractions (PVCs) after MI, use of pulmonary artery catheters in most situations, etc. However, in other instances, withdrawal is either slow to occur or absent— e.g., percutaneous coronary intervention (PCI) for many patients with stable coronary artery disease, vertebroplasty for compression fractures of the spine, etc. One particularly troubling example is failure to abandon late (> 24hours after MI) PCI of obstructed coronary arteries in the setting of acute myocardial infarction. The authors found that there was no significant decline in adjusted monthly rates of PCI after the publication of the Occluded Artery Trial (OAT) and revised guidelines, thus suggesting a lack of “reversal” of medical practice [3, 4].
The authors performed a review of publications in the New England Journal of Medicine [3 archives]. They evaluated all original articles published in the NEJM in 2009 and found that 13% of the publications constituted reversal [2]. Examples of the 16 reversals contradicting current medical practice from their review include—
- Intensive Versus Conventional Glucose Control in Critically Ill Patients
- Vertebroplasty for Painful Osteoporotic Vertebral Fractures
- Revascularization Versus Medical Therapy for Renal-Artery Stenosis
- Extended-Release Niacin or Ezetimibe and Carotid Intima–Media Thickness
“‘Confidence in physiologic models’ as the prime reason to adopt a practice initially” was the most common precondition for reversal. This should serve as yet another warning that clinicians should wait for valid RCTs and enough evidence to conclude that benefits outweigh harms in most cases before adopting new interventions. The information provided by Prasad et al provides further support to taking a conservative stance on benefits reported in observational studies and high risk-of-bias clinical trials. And even then, patients are likely to be exposed to ineffective and harmful interventions. If we can’t unring a bell, let’s at least stop ringing it when reversals become apparent in the medical literature.
References
1. Prasad V, Cifu A, Ioannidis JP. Reversals of established medical practices: evidence to abandon ship. JAMA. 2012 Jan 4;307(1):37-8. PubMed PMID: 22215160.
2. Prasad V, Gall V, Cifu A. The frequency of medical reversal. Arch Intern Med. 2011 Oct 10;171(18):1675-6. Epub 2011 Jul 11. PubMed PMID: 21747003.
3. Hochman JS, Lamas GA, Buller CE, et al; Occluded Artery Trial Investigators. Coronary intervention for persistent occlusion after myocardial infarction. N Engl J Med. 2006 Dec 7;355(23):2395-407. Epub 2006 Nov 14. PubMed PMID: 17105759.
4. Moscucci M. Medical reversal, clinical trials, and the "late" open artery hypothesis in acute myocardial infarction. Arch Intern Med. 2011 Oct 10;171(18):1643-4. Epub 2011 Jul 11. PubMed PMID: 21747001.
5. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124. Epub 2005 Aug 30. PubMed PMID: 16060722.
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
Variations
in Experts' Recommendations
For more than two decades, observers like Jack Wennberg have pointed out geographic variations in health care practices and recommendations in the United States. David Eddy has elegantly demonstrated extreme variation in experts’ recommendations for health care interventions. And yet, many clinicians rely on expert opinion as their primary source of information. EBM emphasizes the need to critically appraise information sources for validity and clinical relevance as a way of dealing with such variation.
Recently Schaafsma FG, Verbeek JH, Hulshof CT, van Dijk FJ have reported on how expert opinion differs from information gathered from an evidence-based approach (Caution required when relying on a colleague's advice; a comparison between professional advice and evidence from the literature. BMC Health Serv Res. 2005 Aug 31;5(1):59. PMID: 16131405)
The authors presented 12 occupational medicine cases to 14 occupational medicine specialists from differing geographical areas in the Netherlands. The authors also searched the literature and critically appraised the evidence on the 12 problems. Questions regarding management were stated so that specialists could answer with yes/no responses. [Example: For a 36-year old caretaker at a secondary school with a lateral ankle ligament rupture treated with tape for three weeks, is it safe to resume work? Evidence-based answer: Yes]
The authors found that 53% (95% confidence interval 42% to 65%) of all professional advice was not in line with the research literature. In 18 cases (24%), professional advice explicitly referred to up-to-date research literature as their source of information. These cases were substantially less incorrect (17%) than advice that had not mentioned the literature as a source (65%). The authors conclude that less than half of the given professional advice by experts to a practical occupational health problem was in line with evidence from the research literature.
This article points out the need for caution in accepting information from “experts” without evaluating the actual evidence from the published medical literature.
Full text
of this article is available at --
http://www.biomedcentral.com/content/pdf/1472-6963-5-59.pdf
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message.
|
Variations
in Care Costs and Care Quality: Evidence (Lack of), Quality
of Care and Physician Revenue
06/01/09
Thanks to our friend,
Edward A. Walker, MD, MHA, Professor, Department of Psychiatry
and Behavioral Sciences, Cheryl M. Scott / Group Health Cooperative
Professor of Health Administration & Director, UW Healthcare
Leadership Development Alliance, who alerted us to an excellent
and thought-provoking article published in the New Yorker in
their Annals of Medicine column on June 1, 2009:
“The
Cost Conundrum: What a Texas town can teach us about health
care”
by Atul Gawande
Gawande takes a
journey throughout the US to ostensibly answer the question,
“Why is health care so expensive in McAllen County, Texas,
when the explanations do not seem to be population-dependent,
nor the answers better resulting care?” And at the end
of his investigation, he concludes that “…the damning
question we have to ask is whether the doctor is set up to meet
the needs of the patient, first and foremost, or to maximize
revenue.”
Some key quotes
from Gawande:
“…having
a system that does so much to misalign [needs of the patient
to physician revenues] has proved disastrous. As economists
have often pointed out, we pay doctors for quantity, not quality.
As they point out less often, we also pay them as individuals,
rather than as members of a team working together for their
patients. Both practices have made for serious problems…"
"Imagine
that, instead of paying a contractor to pull a team together
and keep them on track, you paid an electrician for every
outlet he recommends, a plumber for every faucet, and a carpenter
for every cabinet. Would you be surprised if you got a house
with a thousand outlets, faucets, and cabinets, at three times
the cost you expected, and the whole thing fell apart a couple
of years later? Getting the country’s best electrician
on the job (he trained at Harvard, somebody tells you) isn’t
going to solve this problem. Nor will changing the person
who writes him the check.”
Highly compelling
reading addressing one of the most vital problems we face. The
full text is available here.
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message.
|
Variations
in Clinicians' Estimates of Pretest Probability
Newer EBM texts, when addressing the areas of diagnosis and screening, frequently state that diagnostic tests that produce large changes from pretest to post-test probabilities are very useful clinically.
The teaching goes like this: Likelihood ratios can be utilized along with practitioners’ clinical estimate of the pretest probability of disease, incorporating the results of the diagnostic tests and then deriving an individual patient’s probability of having a disease or condition.
Phelps and Levitt point out (Acad Emerg Med. 2004 Jun;11(6):692-4.) that this approach to generating post-test disease probabilities has never been validated and that the approach may result in very inconsistent results between clinicians.
In a cross-sectional cohort study of emergency and internal medicine residents and faculty, the authors presented clinical vignettes and asked clinicians to estimate the likelihood of common diseases based on history and physical exam findings. No lab or imaging results were provided. They reported that the mean pretest probability estimates of disease ranged from 42% (95% confidence interval [95% CI] = 36.6% to 47.4%) to 77% (95% CI = 72.9% to 81.1%). The smallest difference in pretest probability magnitude for a single vignette was 70% (range 30-100%; interquartile range [IQR] 64-80%), whereas the largest was 95% (range 3-98%; IQR 30-60%).
Their conclusions were that there is wide variability in clinicians’ estimates of pretest probability of disease and that post-test disease estimates are also likely to be inconsistent.
Because of such wide variability, we prefer to concentrate on more traditional statistics derived from the classical 2x2 table such as sensitivity, specificity, positive predictive value and negative predictive value and skip the hassle of estimating pretest probability and using nomograms or tables to generate post-test probabilities.
Abstract
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
Underuse
of Proven Interventions
One of the main reasons for using valid, relevant evidence in health care is to more accurately predict outcomes from various interventions and thus be equipped to make informed choices. For example, it is very useful to know the evidence for outcomes by risk category for cardiovascular disease. The higher an individual’s risk for a cardiovascular event, the greater the benefit from interventions such as lipid-lowering with statins.
Because of increased benefits, we would expect patients who are at the highest baseline risk for CV events to receive the most aggressive lipid-lowering treatment. Ko et al. (JAMA.2004;291:1864-1870), however, have demonstrated that the prescription of statins in Canadian patients older than 66 years of age with a history of CV disease or diabetes mellitus decreased progressively as baseline CV risk increased.
In their database study—
- Only 19% of
the 75,617 patients received statins for secondary prevention;
- In patient
66-74 years of age, the adjusted probabilities for receiving
statin Rx was 37.7% (95% CI 37.3-38.2), 26.7% (26.4-27.1),
and 23.4% (22.8-23.9) in the categories of low, intermediate
and high baseline risk, respectively.
Why would such
a small percentage of elderly patients with CV disease or diabetes
be prescribed lipid-lowering drugs? Why would the likelihood
that physicians prescribe statin therapy be inversely correlated
with CV risk?
Comment: This is
yet one more study demonstrating the huge problem of underuse
of proven interventions in health care. However, this is apparently
the first article pointing out the treatment-risk paradox and
is worth reading. The authors suggest that the treatment risk
paradox in these elderly patients might be explained by:
- Clinicians
not understanding tradeoffs between benefits and harms of
statins—clinicians may believe that the results of RCTs
of lipids cannot be applied to patients with comorbidites
on the grounds that these patients may experience fewer benefits
and greater harms than the study subjects;
- Physicians
may prejudge patients with co-morbidities—they may believe
that such patients will not adhere to lipid-lowering treatment
regimens; and/or,
- Physicians
may be inattentive to preventive care measures in elderly
patients with chronic conditions.
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
Class
Effect: Caution Urged
Pharmacy and Therapeutics committees and clinicians are frequently faced with issues regarding substitution for agents in a specific drug class. CD Furburg and BM Psaty, using ace-inhibitors (ACEIs) as an example, make a number of interesting points in an article entitled, “Should evidence-based proof of drug efficacy be extrapolated to a “class of agents”? (Circulation 2003;108:2608-2510). Below is a summary of some key points made by the authors of the article and also some points made by the editorialists, EM Antman and JJ Ferguson (Circulation 2003;108:2604-2607).
Generic substitution is the act of dispensing a different brand or an unbranded drug product that is the same chemical entity and meets US Food and Drug Administration criteria for bioequivalence (e.g., generic lisinopril in place of brandname Prinivil [Merck & Co.] or Zestril [Zeneca Pharmaceuticals]).
Therapeutic substitution is the dispensing of an alternate chemical entity for the original drug prescribed by the physician from the same general therapeutic class (e.g., a physician orders enoxaparin, but the hospital pharmacy dispenses dalteparin; a physician writes a prescription for ramipril, and the pharmacy dispenses moexipril).
The HOPE trial, using ramipril 10mg, reported the following relative risk reductions in patients with vascular disease or diabetes and one additional risk factor:
- 16% reduction
over placebo in all cause mortality
- 20% reduction
in MI
- 32% reduction
in stroke
However, the evidence is less robust for some of the other ACEIs:
- There was no
decrease in ischemic events in subjects who had undergone
angioplasty and took Quinapril 20mg when compared to placebo
over 3 years
- In the PROGRESS
trial, there was no difference in stroke or coronary events
in the Perindopril 4 mg and placebo groups. (However, in the
EUROPA trial using 8 mg, there was a reduction in coronary
mortality and morbidity of 20%.)
- 4 ACEIs have
not been shown to reduce mortality or morbidity—2 agents
have not been tested and 2 have not shown a reduction in clinical
events.
Does it make sense to declare all of the ACEIs equivalent? What about differing doses that have not been tested?
Furburg and Psaty point out that all 7 drugs have FDA approval for treating hypertension, heart failure and LV dysfunction. All 7 have been shown to lower BP and improve hemodynamic measurements. However, only 4 of these drugs—enalapril, captopril, ramipril and trandolopril—have been shown to improve long term survival or clinical outcomes compared to placebo in large, RCTs.
They point out that:
- The actions
specific to an individual agent may add to, subtract from
or have a neutral effect on the safety and efficacy attributed
to the common class actions of a drug class.
- Concluding
that equipotency exists rests on surrogate measures of efficacy.
- Equipotency
is difficult to establish in the absence of relevant outcome
data.
- The term “class
effect” has never been defined from a regulatory perspective.
- The class effect
concept is often overinterpreted to mean that all agents in
a class are interchangeable.
- Me-too drugs
can be produced without the risk and expense of long term
morbidity and mortality trials.
- Regulatory
agencies do not accept extrapolations, and FDA regulations
prevent manufacturers from making claims for unapproved indications.
- Safety is an
issue to consider.
Given all of the above, the authors state that efforts should be made to use drugs with the best-documented outcomes and in full doses. Further, the use of me-too drugs without adequate documentation should be restricted. Comparability should be established in trials where two agents in a class are compared.
The editorialists list the following minimal criteria they would require for defining a class effect of a drug:
- A clearly defined
biological target or pathway.
- Comparable
efficacy demonstrated for multiple agents within the class
(with multiple randomized, controlled clinical trials for
each agent).
- Absence of
convincing evidence that there is a member of the class that
does not have comparable clinical benefit to that of other
agents within the class.
They also recommend
considering the following before deciding whether drugs within
a class are interchangeable:
- The absolute
and relative degree of benefit for each agent;
- The clinical
circumstances in which benefit has been demonstrated;
- The extent and
the depth of the evidence in favor of a particular agent;
- The safety profile
and tolerability of agents within a class;
- The cost of
the alternatives, viewed not only in terms of cost per quality
life-year saved versus placebo or an established control,
but also in terms of cost per quality life-year lost in the
case of less expensive, but also less effective, alternatives;
- The specific
details of the inclusion/exclusion criteria and the exact
management protocols used in the individual supporting clinical
trials; and,
- The particular
subgroups that show benefit (or lack of benefit) for agents
within the class.
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
Is “Biologics Versus Biosimilars” A Different Story Than Brand Names Versus Generics?
02/16/2012
In 1984, the Drug Price Competition and Patent Term Restoration Act (Hatch-Waxman Act) created a shortened new drug application pathway for generics by eliminating the need for preclinical and clinical studies if bioequivalence—i.e., the rate and extent of absorption—could be demonstrated. The FDA defines a generic drug as follows: “A generic drug is identical—or bioequivalent—to a brand name drug in dosage form, safety, strength, route of administration, quality, performance characteristics and intended use.”[1] For many drugs, the process was straightforward and payers hoped to reduce expense by purchasing generics.
Although the FDA frequently uses the term “biologic drug,” we have been unable to find an official FDA definition for “biologic drug.” The FDA instead separately defines “biological product” and “therapeutic biological product.” These are the current FDA definitions—Biological Product: “Biological products include a wide range of products such as vaccines, blood and blood components, allergenics, somatic cells, gene therapy, tissues, and recombinant therapeutic proteins. Biologics can be composed of sugars, proteins, or nucleic acids or complex combinations of these substances, or may be living entities such as cells and tissues. Biologics are isolated from a variety of natural sources—human, animal, or microorganism—and may be produced by biotechnology methods and other cutting-edge technologies. Gene-based and cellular biologics, for example, often are at the forefront of biomedical research, and may be used to treat a variety of medical conditions for which no other treatments are available.” The FDA then defines a therapeutic biological product as follows: “A therapeutic biological product is a protein derived from living material (such as cells or tissues) used to treat or cure disease.”[2] Thus, biologics are large-molecule medications produced by living cells. They are produced by using recombinant DNA technology to direct protein synthesis within cells which can then be used to produce medications.
The Patient Protection and Affordable Care Act (Affordable Care Act), signed into law by President Obama on March 23, 2010, amends the Public Health Service Act (PHS Act) to create an abbreviated licensure pathway for biological products that are demonstrated to be “biosimilar” to or “interchangeable” with an FDA-licensed biological product. This pathway is provided in the part of the law known as the Biologics Price Competition and Innovation Act (BPCI Act). Under the BPCI Act, a biological product may be demonstrated to be “biosimilar” if data show that, among other things, the product is “highly similar” to an already-approved biological product. [3] Biosimilars, also called “follow-on biologics,” are in many ways analogous to generics, but are not generic drugs. In Canada they are aptly labeled “subsequent entry biologics (SEBs).” [4] We like the clarity of the Canadian term, but Americans will hear the terms “biologics” and “biosimilars.”
Biologics and biosimilars are large protein molecules. Manufacturing a biosimilar drug poses numerous complexities not seen in the manufacturing of “small molecules” created as generic drugs. A company wishing to manufacture a biosimilar does have access to the commercial biologic product, but does not have access to the biologic cell line used by manufacturerof the reference biologic or details of the manufacturing process such as how fermentation and purification were carried out. This lack of information (and therefore the lack of identical manufactured cellular material), together with the molecular and structural complexity of large biologic proteins makes for immense complexity for the companies who will be creating biosimilars. The bottom line is that even though a biosimilar has the same recombinant DNA sequencing, small differences in structure or chemistry due to the processes and chemicals used in the culture, purification, storage, etc. may result in differences in efficacy, safety and immunological outcomes.
The Health Care Reform Act has outlined a process for abbreviated approval for biosimilars. The FDA has conducted public meetings and published requirements for a drug to be considered biosimilar, emphasizing clinical studies demonstrating that the product is “highly similar” to the reference product, although minor differences are allowed if the differences do not result in clinically meaningful safety, purity and potency differences. In addition, interchangeable biological products may be substituted at the pharmacy level without the intervention of a healthcare provider. [3]
The strength of evidence necessary to result in a decision of biosimilarity and interchangeability remains unclear. It is also currently unclear if separate evidence for each indication will be required. Currently The European Medicines Agency (EMA) has outlined important elements and considerations required to support the designation of biosimilarity in comparison with existing biologics along with fairly detailed scientific guidance documents on biosimilar medicines.[5] Key requirements include similar pharmacokinetics and pharmacodynamics in humans and the demonstration of similar clinical efficacy and safety of the biosimilar compared to that of the reference biologic. Several biosimilar agents have already been licensed in Europe. The American College of Rheumatology has developed a concise position paper which takes a patient-centered approach to biosimilars. The paper is available online. [6] Key points are—
-
Biologics are proteins produced by living cells, including monoclonal antibodies, soluble receptors, receptor antagonists, novel molecules derived by genetic engineering and other types of proteins that can be used to treat human diseases.
-
Biosimilars may represent a cost-saving alternative to reference biologics.
-
Currently the FDA has not provided details of the kind of testing required to demonstrate sufficient similarity in efficacy and safety for approval.
-
A similar production process does not ensure that the biosimilar is functionally equivalent to a reference biologic. Extensive human testing will be required.
-
Even though the FDA has been establishing standards for licensure to ensure the safety and effectiveness of biosimilars and issued a guidance in February 2012 [7], the FDA has not approved a biological product as biosimilar or interchangeable.
-
Several companies are developing biosimilar products and will almost certainly submit applications for licensure under the new law. It is not yet known when the first biosimilar will be on the U.S. market.
Comment? Click to go to our Blog. At the end of that page, you can Leave a Reply.
References
1.Generics.
http://www.fda.gov/Drugs/ResourcesForYou/Consumers/BuyingUsingMedicineSafely/
UnderstandingGenericDrugs/ucm144456.htm. Accessed 2/1/12.
2. FDA definitions.
http://www.fda.gov/Drugs/informationondrugs/ucm079436.htm#B
3. BPCI Act. http://www.fda.gov/Drugs/DevelopmentApprovalProcess/
HowDrugsareDevelopedandApproved/ApprovalApplications/
TherapeuticBiologicApplications/Biosimilars/ucm241719.htm. Accessed 2/1/12.
4. Subsequent Entry Biologics (SEBs).
http://www.hc-sc.gc.ca/dhp-mps/brgtherap/applic-demande/
guides/seb-pbu/notice-avis_seb-pbu_2010-eng.php. Accessed 2/1/12
5.BiosimilarsEMA.
http://www.ema.europa.eu/ema/index.jsp?curl=pages/special_topics/
document_listing/document_listing_000318.jsp&murl=menus/
special_topics/special_topics.jsp&mid=WC0b01ac0580281bf0. Accessed 2/1/12.
6. American College of Rheumatology. http://www.rheumatology.org/practice/
clinical/position/biosimilars.pdf. Accessed 2/1/12.
7. Biosimilars FDA Guidance
http://www.fda.gov/downloads/Drugs/
GuidanceComplianceRegulatoryInformation/
Guidances/UCM291134.pdf.
Accessed 06/05/2015.
Other Sources
8. Colbert RA, Cronstein BN. Biosimilars: the debate continues. Arthritis Rheum. 2011 Oct;63(10):2848-50. doi: 10.1002/art.30505. PubMed PMID: 21702015.
9. Reichert JM, Beck A, Iyer H. European Medicines Agency workshop on biosimilar monoclonal antibodies: July 2, 2009, London, UK. MAbs. 2009 Sep-Oct;1(5):394-416. Epub 2009 Sep 25. PubMed PMID: 20065643.
10. Kozlowski S, Woodcock J, Midthun K, Sherman RB. Developing the nation's biosimilars program. N Engl J Med. 2011 Aug 4;365(5):385-8. PubMed PMID: 21812668.
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
Genomic Medicine Leaps Forward—More Drugs Targeting More Cancers
03/02/2015
We, like others, have been watching to see how genetic information will improve health outcomes (genomic medicine). Recently we encountered two pieces worth reading. The first is the NCI Molecular Analysis for Therapy Choice Program (MATCH) which will conduct small, phase II trials that will enroll adults with advanced solid tumors and lymphomas whose tumors are no longer responding to standard therapy and have begun to grow. Subjects will receive drugs targeting specific genetic abnormalities common across cancers. What is unique is that DNA sequencing will be used to identify individuals whose tumors of various types have specific genetic abnormalities that may respond to selected targeted drugs. Study arms (baskets) are created by cancer type, and multiple drugs can be studied. Details are available at— http://www.cancer.gov/clinicaltrials/noteworthy-trials/match.
The second piece titled, "A Faster Way to Try Many Drugs on Many Cancers," by Gina Kolata and published in the New York Times (http://www.nytimes.com/2015/02/26/health/fast-track-attacks-on-cancer-accelerate-hopes.html?_r=0) provides examples of some of the clinical trials with basket designs, often referred to as "basket trials" because patients are also grouped by genetic abnormality rather than cancer type.
Delfini Comment
These trials will rely on surrogate markers (progression free survival and response rates), but may be useful if effect sizes are large. Investigators are interested in these trials because they can be done rapidly and are not constrained by many of the requirements of RCTs. You can quickly get the idea of the basket trial designs by looking at the first link above and the FDA site below. The FDA appears to be supportive of these initiatives and has created a PowerPoint slide deck with additional information about basket trials,including specific cancers and drugs at— http://www.fda.gov/downloads/Drugs/NewsEvents/UCM423361.pdf.
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
Biosimilars & FDA's New "Purple Book"
03/09/2015
As many of you know the FDA's nickname for the "Approved Drug Products with Therapeutic Equivalence Evaluations" is the "Orange Book." And now for the newest color—purple. The FDA has now nicknamed its lists of biosimilar and interchangeable biological products licensed by FDA under the Public Health Service Act (the PHS Act) the "Purple Book."
The Patient Protection and Affordable Care Act (Affordable Care Act), signed into law by President Obama on March 23, 2010, amended the Public Health Service Act (PHS Act) to create an abbreviated licensure pathway for biological products that are demonstrated to be "biosimilar" to, or "interchangeable," with an FDA-licensed biological product. A biological product may be demonstrated to be "biosimilar" if data show that, among other things, the product is "highly similar" to an already-approved biological product [1].
Delfini Comment
More information about biosimilars, the Purple Book and the first FDA approved biosimilar product—Zarxio—is available at the following FDA site below [2].
For those wishing more background information see our previous blog, Is "Biologics Versus Biosimilars" A Different Story Than Brand Names Versus Generics? Available at http://delfini.org/blog/?p=100.
References
- BPCIAct. http://www.fda.gov/Drugs/ DevelopmentApprovalProcess/ HowDrugsareDevelopedandApproved/ ApprovalApplications/TherapeuticBiologicApplications/Biosimilars/ucm241719.htm Accessed 3/9/15 http://www.fda.gov/Drugs/
- DevelopmentApprovalProcess/ HowDrugsareDevelopedandApproved/ ApprovalApplications/TherapeuticBiologicApplications/Biosimilars/default.htm Accessed 3/9/15
Click here to share this page. If you are at an entry URL (#title), copy URL, then click "share" button to paste into body text of your message. |
........
CONTACT DELFINI |
At DelfiniClick™
DelfiniClick™
|
Read Our Blog...
Menu........
Use of our website implies agreement to our Notices. Citations for references available upon request.
Home
Best of Delfini
What's New
Blog
Seminars
Services
Delfini Group Publishing
Resources
Sample Projects
Notices
About
Us & Our Work
Testimonials
Other
Site Search
Contact
Info/Updates
........
Quick
Navigator to Selected Resources
.......................
|