Evidence Based Health Care Quality Help: Quality of Care

Volume — Use of Evidence:
Quality of Care — From Variation to Improvement

newest
01/24/2012: Reversal of Established Practices—Unringing the Bell

Contents

Go to DelfiniClick™ for all volumes.

The Volume of Inappropriate Care in the US

Could things be getting worse in the area of clinical quality? We have seen estimates of 20%-30% inappropriate care in the past, but in the May/June 2004 issue of Health Affairs (Vo 23;3: 247-255), Kerr et al. report that in a study of approximately 7,000 people in 12 metropolitan areas in the United States, on average, residents received only 50% to 60% of recommended care (recommended care was based on 439 quality indicators across thirty conditions and preventive care recommendations developed by RAND and validated by four multi-specialty expert panels using a modified Delphi method). And this doesn't even address the other side of inappropriate care — overuse. We continue to believe the answer lies in organizations creating a system of evidence-based solutions for care.

Full text is available to subscribers and can be obtained for a fee for non-subscribers at http://www.healthaffairs.org/

More On Therapies Becoming “Routine Clinical Practice” Without Evidence & Why We Are Doing This Work: Radiofrequency for the Treatment of Gastro-esophageal Reflux Disease Critical Appraisal Example
04/01/08

Several years ago, we received a phone call from a medical director asking if we would do a limited review of radiofrequency for the treatment of gastro-esophageal reflux disease (GERD in the US; GORD in UK). He had received a request for coverage of this procedure by a physician and was concerned that there was not good scientific evidence supporting use of this procedure. His health plan policies required proof of efficacy.

We agreed. We contacted the requesting physician and asked him to recommend his best three studies for review. (He pleaded for more - we relented and said, “Okay, four.”) Of the 4 studies he sent us, only 1 was a randomized controlled trial (RCT), and so we had to surmise that if he had been given permission to send us even more, those additional studies, too, would have been observational.

Observational studies should not be used for addressing questions of efficacy of interventions with rare exception.[1,2] The only exception that is thus far agreed upon is when all-or-none results are achieved, and this is extremely rare.

And one RCT generally will not be considered sufficient to declare “proof.” Even a meticulously designed, executed and reported RCT may still result in findings merely due to chance, not to mention that only through replicated studies might one discern the unfortunate occasional fraud. And this RCT could not even come close to being a candidate for definitive proof of anything. While thoughtfully designed by investigators more knowledgeable of research design than many we have seen, it was very small and had myriad problems as our critique — reproduced below — demonstrates.

Even the authors - very responsibly - themselves acknowledged that their study should only be considered preliminary and to have proved nothing (a point that the requesting physician seemed to have overlooked in his desire for a solution for this problem that he, himself, believed in).

And now, in The Lancet, Mehran Anvari[3] of St Joseph’s Healthcare, Hamilton ON, Canada, comments upon a recently published review which considers clinical trials from the past three decades on the use of endoluminal therapies for GERD/GORD.[4]

Anvari writes that the systematic reviewers found that “…no study of any techniques has provided adequate ‘scientific and clinical data on their safety, efficacy and durability to support the use of endoluminal therapies for GORD [gastro-oesophageal reflux disease] in routine clinical practice.’ Thus we could be left with a sense of pessimism about the future use of endoluminal therapies for this common disease, at a time when similar endoluminal approaches are being proposed for other common disorders, such as morbid obesity.”

He tells us that the reviewers, “…also highlight the surprising lack of well-designed randomised trials, even though many of the endoluminal therapies have been in clinical use for several years and many of the trials of these techniques have come from respected academic centres.” He goes on to say, “It almost seems that the standards established for the assessment of treatment efficacy in the control of GORD… were lowered [emphasis ours] when it came to assessing these approaches.”

He also presents the sobering news that, “As a result, many patients have undergone procedures that have since been abandoned and proven ineffective, and an unfortunate minority has had substantial side-effects or even died.”[5-9]

Indeed. And how many patients may have been told that research studies have shown benefits of this treatment? Isn’t it possible that some patients would not have elected to undergo this procedure if they had been told, what Anvari states — most correctly — in his commentary, “…up to this point, there is no clear evidence to support the use of any of the endoluminal therapies…in routine clinical practice.”

This is what we advocate:

All health care professionals who care for patients need to recognize that the vast majority of published research in even the best medical journals is so flawed in design, execution and/or reporting, or is not clinically useful, that it should not be used to inform health care decision-making. Patients’ quality of care — and even their lives — depend upon this understanding. Of the approximately 12,000 articles published in the medical literature each week, it is our estimate, having evaluated thousands of published studies, that fewer than 10 percent of the published studies in health care are both valid and clinically useful, while others have estimated that the number is less than 5 percent.[10]
And most users of the medical literature lack the skills to tell. The vast majority of physicians, clinical pharmacists and nurses fail our simple critical appraisal pre-test. (Read more shocking facts here.)
It is incumbent upon every such professional to attain the skills to discern valid and clinically useful research from that which is not.
This means that —
- Health care researchers need these skills before subjecting patients to medical research.
- Medical editors, peer reviewers and health care writers need these skills in order to avoid passing on the flood of misleading information. It is one thing to publish work that may legitimately raise hypotheses. We, personally, think it is perilous to publish work (and even perform research) that is clearly severely flawed.
- Publishers also need to change their editorial policies which frequently emphasize “discussion” over “reporting of methods.” Users need to see methodological details to determine whether they agree that the study is valid. It is not sufficient to tell a reader that a study was blinded. Readers need to evaluate the likelihood of success of blinding — and only details will do. Transparency is vital.
- Physicians, clinical pharmacists, nurse practitioners and others directing or influencing patient decision-making need to attain the skills to evaluate the health care information put before them.
- Leaders need to attend to this urgent need and provide the appropriate culture and work elements, such as structures, processes, methods, staff roles, skills and tools for evidence- and value-based practice.
- Professional societies, advocacy groups, experts writing reviews, systematic reviewers, guideline developers and others who create secondary studies and secondary sources need to develop these skills before producing “guidance.”
- Policy makers and payers need to be aware of the huge problems in the quality of health care information and attain a true understanding of the evidence before dictating sweeping changes with far reaching impact that may only contribute to the cost and quality problems in health care.
- Manufacturers need to gain an understanding of what makes for valid and clinically useful science and how it needs to be reported.
- Legal professionals need to become responsible and help raise the bar for evidence that is valid and clinically useful.
- And much of it all starts in the schools. Academicians must gain these skills themselves. And they must strive to identify ways to successfully inculcate evidence-based practice knowledge and habits in their students.

While some would say if there is no evidence of benefit, don’t do it, we, personally, do not wish to tell others how to practice medicine. That is not what we see our role to be. No evidence of benefit is not the same thing as evidence of no benefit. And it is possible that something for which there is no evidence works. (In fact, some topics are so inherently difficult to study that we will never have good evidence — and this doesn’t mean that practitioners will stop trying to find solutions and offering options ot patients.)

We do, of course, advocate that a science-based approach be taken in all possible instances. It is important to try and study — correctly — cause and effect for interventions. It is important to report findings responsibly. It is important that all users of such information have skills to evaluate science and that they do so.

Know and clearly state the evidence first! Then decisions can be made taking into account other factors.

However, it is our strong belief that patients should be provided with honest information to enable them to make choices that are in their best interest in accordance with their values and preferences. This requires that all working in health care decision-making acquire the appropriate skills. There are many, many resources available to help achieve this — including information we make freely available on our site from our own work and the work of others.

Our patients deserve this — and indeed — their lives depend upon it. And that is why we are doing this work.

Improvement of gastroesophageal reflux symptoms after radiofrequency energy: a randomized, sham-controlled trial. Corley DA, Katz P, Wo JM, Stefan A, Patti M, Rothstein R, Edmundowicz S, Kline M, Mason R, Wolfe MM. Gastroenterology. 2003 Sep;125(3):668-76. PMID 12949712

Review & Critique by Sheri A. Strite & Michael E. Stuart MD, Delfini Group, Original Review January 14, 2006 and Subsequently Updated

Commentaries in PubMed: See editorial by Kahrilas (Kahrilas PJ. Radiofrequency energy treatment of GERD. Gastroenterology. 2003 Sep;125(3):970-3. No abstract available. PMID: 12949740). Editorial urges caution until further studies become available.

Issues with Study

Number of People in Study (N): Small N—small N may be especially prone to non-representation with population at large.

Inclusions: It appears that patients who may still have benefited from proton pump inhibitors (PPI) and H2 blocker treatment (i.e., those who had not failed medical therapy) were allowed to participate in the study.

Randomization: Sequential randomization method was used which is a weaker method more prone to “gaming” especially when concealment of allocation is not adequate. There is no mention of effective concealment of allocation methods being used which would prevent channeling patients to desired groups, such as use of locked sealed-containers or a 1-800 call-in center, as examples. One patient (active treatment group) was determined to be ineligible post-randomization (see discussion in Intention-to-Treat Analysis or ITT analysis).

Baseline Characteristics: No p-values were reported for baseline characteristics, and authors state that there was a slight difference in age and PPI use between groups. However, upon visual comparison the groups appear equal for demographics and disease dimensions. Plus, this is suggestive that randomization was successfully achieved.

Blinding: Authors state study is double-blinded, however, no details of blinding were reported excepting to state that staff blinded to the treatment assignments performed subsequent outcome assessments. The endoscopist opened the envelope to determine assignment during the endoscopy. Of some concern is that the authors state that that “subsequent contact between the subject and the endoscopist was minimized…”, but it was not subsequently prevented—See Intention-to-Treat Analysis. The only outcome that remained statistically significant through the toughest test was subjective (health-related quality of life) which could have been influenced if patients had any direct, indirect or even subliminal idea they had received radiofrequency energy treatment (RF). Authors themselves state that they “cannot exclude a possible bias from subjects guessing their treatment assignments because of difference in post-treatment symptoms between the active and sham groups.”

Statistical Analysis: No analysis of statistics was performed by Delfini.

Loss of Data Points: Significant loss of data points. However, authors performed sensitivity analysis using ITT methods including an imputation method that put the intervention through the toughest test. In that test, health-related quality of life remained statistically significant. However, there are no details reported of the actual outcomes in this analysis—further see comments that potential subsequent contact with endoscopist could have affected subjective measurement of patients.

Analysis Methods Used: Authors did not use Intention-to-Treat (ITT) analysis as their primary analysis method. They performed a completer analysis as the primary method—and because they had over 5 percent loss to follow-up, these results are not dependable. The authors, however, also did a sensitivity analysis that included a last-observation-carried forward (LOCF) method plus an extreme-case analysis (see below for ITT discussion). LOCF has been shown to be prone to bias. [Reminder to readers: extreme-case analysis is when puts the intervention under study through the toughest test: one assigns “failure” as an outcome to missing values in the intervention group and “success” to missing values in the comparison group. Statistical significance is then measured. If results are statistically significant in an otherwise valid study, we can conclude efficacy of the treatment.]

Intention-to-treat Analysis & Imputation Methods: Authors state that they performed sensitivity analyses on “all patients” using LOCF and extreme-case analysis—putting the intervention through the toughest test. It would be preferable to see the numbers used and outcomes obtained rather than summary statements; however, overall authors have demonstrated sophistication and a great deal of clarity in reporting this study (including ITT methods)—indicating knowledge with key methodologic principles.

Eligibility was determined after randomization, further contributing to patient loss—this, however, amounted to one patient, and was 3.5 percent. Because the actual number of “all” patients was not reported, it is not possible to tell if this patient was included in the various ITT analyses. However, this is less than 5 percent and so probably would not significantly affect study results. Delfini reviewers are most focused on results of sensitivity analysis as these were ITT analyses. LOCF seems reasonable for imputing missing values for this condition as a sensitivity analysis, and extreme-case scenario reporting is always appropriate as it presents the most severe challenge to statistically significant differences between groups. Authors report statistically significant improvement in the primary outcomes of heart burn symptoms (LOCF and extreme-case) and quality of life (extreme-case). There were no differences in any analysis in daily medication use or esophageal acid exposure times—however, study was not be powered to detect differences between groups in these outcomes, and so we do not know if “no difference” between groups is true or if they didn’t have enough people to show a statistically significant difference if there was one, in fact.

Mechanism of Action: Per editorial, it is uncertain whether the results are due to reduced gastroesophageal reflux (paper suggests not) or are due to diminished esophageal sensitivity.

Clinical Significance: This study does not provide sufficient evidence to conclude that radiofrequency ablation is effective and safe. Because of the above-mentioned threats to validity and the patient population, conclusions regarding symptom relief and improved quality of life should be drawn cautiously. Further confirmatory studies that include patients who have not responded to H2-blockers and PPIs would be useful. Undetected long term harms remain a possibility. Patients should be aware that the medical evidence is sparse and that uncertainty regarding the appropriate role of radiofrequency therapy in the treatment of gastroesophageal reflux symptoms remains.

Safety:

There were more adverse effects (ADEs) reported in the active treatment group than in the sham group—no p-values are reported. Specific ADEs were not determined a priori which increases likelihood that any reported outcomes are due to chance.
One active treatment patient with an esophageal ulcer had self-limited bleeding and had inpatient observation overnight.
A patient in the sham group developed pneumonia and received inpatient antibiotics; the pneumonia was thought to be due to aspiration during the endoscopy.
One active treatment patient experienced increased bloating; a nuclear medicine gastric emptying study showed worsening of a pre-existing delayed gastric emptying disorder.
Some active and sham patients experienced temporary post-procedure retrosternal discomfort that required oral analgesics. These transient symptoms were not fully quantifiable with the measurement instruments.

Other:

In many respects, this is a very well done and very well reported study. The authors clearly have understanding of research excellence in design and execution.
A study such as this cannot be sufficient to “prove” a therapy’s efficacy. Proof is not possible with main issues such as small N in a single study, questions about maintenance of blinding, plus a need for assurance that analyses were truly done as intention-to-treat with all randomized patients especially given the subjective measures of heartburn and health-related quality of life. Furthermore, even in this research setting, the reported benefits are minimal and extrapolating benefits to clinical practice should be done with caution. Without confirmatory studies, radiofrequency therapy for GERD symptoms should be considered unproven. However, for a small study, this is a very thoughtfully-designed, apparently well-executed study as reported. It is suggestive that there may be benefits from this procedure. Further research is clearly needed to determine if this treatment can help patients avoid more invasive and risky procedures.

References
1. Delfini Primer: Problems with Case Series

2. Delfini Primer: Problems with the Use of Observational Studies to Draw Cause and Effect Conclusions About Interventions

3. Anvari, M. Endoscopic treatments for gastro-oesophageal reflux disease. [Comment] The Lancet. Vol 371. March 22, 2008. 965-966

4. Fry LC, Mönkemüller K, Malfertheiner P. Systematic review. Endoluminal therapy for gastro-oesophageal refl ux disease: evidence from clinical trials. Eur J Gastroenterol Hepatol 2007; 19: 1125–39.

5. Pleskow D, Rothstein R, Lo S, et al. Endoscopic full-thickness plication for the treatment of GERD: a multicenter trial. Gastrointest Endosc 2004; 59: 163–71.

6. Noh KW, Loeb DS, Stockland A, Achem SR. Pneumomediastinum following enteryx injection for the treatment of gastroesophageal refl ux disease. Am J Gastroenterol 2005; 100: 723–26.

7. Wong RF, Davis TV, Peterson KA. Complications involving the mediastinum after injection of Enteryx for GERD. Gastrointest Endosc 2005; 61: 753–56.

8. Tintillier M, Chaput A, Kirch L, Martinet JP, Pochet JM, Cuvelier C. Esophageal abscess complicating endoscopic treatment of refractory gastroesophageal refl ux disease by enteryx injection: a fi rst case report. Am J Gastroenterol 2004; 99: 1856–58.

9. Tuebergen D, Rijcken E, Senninger N. Esophageal perforation as a complication of EndoCinch endoluminal gastroplication. Endoscopy 2004; 36: 663–65.

10. Ebell M. An introduction to information mastery, July 15, 1988. http://www.poems.msu.edu/InfoMastery/default.htm. Accessed December 21, 2007.

Reversal of Established Practices—Unringing the Bell

January 24, 2012

Many healthcare practices become established before there is sufficient evidence to conclude that the intervention is beneficial and that benefits outweigh harms. Reversal of established practices has been recently discussed by Prasad, Gall, Cifu and Ioannidis [1, 2]. Abandoning a practice because evidence demonstrates that the practice is not beneficial is an important issue and frequently is very slow to happen.

Prasad defines reversal as the phenomenon of a new trial—superior to predecessors because of better design, increased power, or more appropriate controls—contradicting current clinical practice [2]. Prasad and colleagues provide a very useful window through which we can see from yet another vantage point the problem of overuse of unproven healthcare interventions and the difficulty of withdrawal. These investigators provide numerous examples of recent reversals where practiced has changed—hormone replacement therapy to prevent second myocardial infarctions (MIs) in women (although there has been renewed used of HRT in younger women based on what appears to us to be evidence at high risk-of-bias), routine use of flecainide and encainide to treat premature ventricular contractions (PVCs) after MI, use of pulmonary artery catheters in most situations, etc. However, in other instances, withdrawal is either slow to occur or absent— e.g., percutaneous coronary intervention (PCI) for many patients with stable coronary artery disease, vertebroplasty for compression fractures of the spine, etc. One particularly troubling example is failure to abandon late (> 24hours after MI) PCI of obstructed coronary arteries in the setting of acute myocardial infarction. The authors found that there was no significant decline in adjusted monthly rates of PCI after the publication of the Occluded Artery Trial (OAT) and revised guidelines, thus suggesting a lack of “reversal” of medical practice [3, 4].

The authors performed a review of publications in the New England Journal of Medicine [3 archives]. They evaluated all original articles published in the NEJM in 2009 and found that 13% of the publications constituted reversal [2]. Examples of the 16 reversals contradicting current medical practice from their review include—

Intensive Versus Conventional Glucose Control in Critically Ill Patients
Vertebroplasty for Painful Osteoporotic Vertebral Fractures
Revascularization Versus Medical Therapy for Renal-Artery Stenosis
Extended-Release Niacin or Ezetimibe and Carotid Intima–Media Thickness

“‘Confidence in physiologic models’ as the prime reason to adopt a practice initially” was the most common precondition for reversal. This should serve as yet another warning that clinicians should wait for valid RCTs and enough evidence to conclude that benefits outweigh harms in most cases before adopting new interventions. The information provided by Prasad et al provides further support to taking a conservative stance on benefits reported in observational studies and high risk-of-bias clinical trials. And even then, patients are likely to be exposed to ineffective and harmful interventions. If we can’t unring a bell, let’s at least stop ringing it when reversals become apparent in the medical literature.

References

1. Prasad V, Cifu A, Ioannidis JP. Reversals of established medical practices: evidence to abandon ship. JAMA. 2012 Jan 4;307(1):37-8. PubMed PMID: 22215160.

2. Prasad V, Gall V, Cifu A. The frequency of medical reversal. Arch Intern Med. 2011 Oct 10;171(18):1675-6. Epub 2011 Jul 11. PubMed PMID: 21747003.

3. Hochman JS, Lamas GA, Buller CE, et al; Occluded Artery Trial Investigators. Coronary intervention for persistent occlusion after myocardial infarction. N Engl J Med. 2006 Dec 7;355(23):2395-407. Epub 2006 Nov 14. PubMed PMID: 17105759.

4. Moscucci M. Medical reversal, clinical trials, and the "late" open artery hypothesis in acute myocardial infarction. Arch Intern Med. 2011 Oct 10;171(18):1643-4. Epub 2011 Jul 11. PubMed PMID: 21747001.

5. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124. Epub 2005 Aug 30. PubMed PMID: 16060722.

Note: Sharing in Facebook does not transmit full URL, only delfini.org. Email will share this page, but not this specific entry. To share this page and/or this entry, copy and paste the URL into the body of your message.

Variations in Experts' Recommendations

For more than two decades, observers like Jack Wennberg have pointed out geographic variations in health care practices and recommendations in the United States. David Eddy has elegantly demonstrated extreme variation in experts’ recommendations for health care interventions. And yet, many clinicians rely on expert opinion as their primary source of information. EBM emphasizes the need to critically appraise information sources for validity and clinical relevance as a way of dealing with such variation.

Recently Schaafsma FG, Verbeek JH, Hulshof CT, van Dijk FJ have reported on how expert opinion differs from information gathered from an evidence-based approach (Caution required when relying on a colleague's advice; a comparison between professional advice and evidence from the literature. BMC Health Serv Res. 2005 Aug 31;5(1):59. PMID: 16131405)

The authors presented 12 occupational medicine cases to 14 occupational medicine specialists from differing geographical areas in the Netherlands. The authors also searched the literature and critically appraised the evidence on the 12 problems. Questions regarding management were stated so that specialists could answer with yes/no responses. [Example: For a 36-year old caretaker at a secondary school with a lateral ankle ligament rupture treated with tape for three weeks, is it safe to resume work? Evidence-based answer: Yes]

The authors found that 53% (95% confidence interval 42% to 65%) of all professional advice was not in line with the research literature. In 18 cases (24%), professional advice explicitly referred to up-to-date research literature as their source of information. These cases were substantially less incorrect (17%) than advice that had not mentioned the literature as a source (65%). The authors conclude that less than half of the given professional advice by experts to a practical occupational health problem was in line with evidence from the research literature.

This article points out the need for caution in accepting information from “experts” without evaluating the actual evidence from the published medical literature.

Full text of this article is available at --
http://www.biomedcentral.com/content/pdf/1472-6963-5-59.pdf

Variations in Care Costs and Care Quality: Evidence (Lack of), Quality of Care and Physician Revenue
06/01/09

Thanks to our friend, Edward A. Walker, MD, MHA, Professor, Department of Psychiatry and Behavioral Sciences, Cheryl M. Scott / Group Health Cooperative Professor of Health Administration & Director, UW Healthcare Leadership Development Alliance, who alerted us to an excellent and thought-provoking article published in the New Yorker in their Annals of Medicine column on June 1, 2009:

“The Cost Conundrum: What a Texas town can teach us about health care”
by Atul Gawande

Gawande takes a journey throughout the US to ostensibly answer the question, “Why is health care so expensive in McAllen County, Texas, when the explanations do not seem to be population-dependent, nor the answers better resulting care?” And at the end of his investigation, he concludes that “…the damning question we have to ask is whether the doctor is set up to meet the needs of the patient, first and foremost, or to maximize revenue.”

Some key quotes from Gawande:

“…having a system that does so much to misalign [needs of the patient to physician revenues] has proved disastrous. As economists have often pointed out, we pay doctors for quantity, not quality. As they point out less often, we also pay them as individuals, rather than as members of a team working together for their patients. Both practices have made for serious problems…"

"Imagine that, instead of paying a contractor to pull a team together and keep them on track, you paid an electrician for every outlet he recommends, a plumber for every faucet, and a carpenter for every cabinet. Would you be surprised if you got a house with a thousand outlets, faucets, and cabinets, at three times the cost you expected, and the whole thing fell apart a couple of years later? Getting the country’s best electrician on the job (he trained at Harvard, somebody tells you) isn’t going to solve this problem. Nor will changing the person who writes him the check.”

Highly compelling reading addressing one of the most vital problems we face. The full text is available here.

Variations in Clinicians' Estimates of Pretest Probability

Newer EBM texts, when addressing the areas of diagnosis and screening, frequently state that diagnostic tests that produce large changes from pretest to post-test probabilities are very useful clinically.

The teaching goes like this: Likelihood ratios can be utilized along with practitioners’ clinical estimate of the pretest probability of disease, incorporating the results of the diagnostic tests and then deriving an individual patient’s probability of having a disease or condition.

Phelps and Levitt point out (Acad Emerg Med. 2004 Jun;11(6):692-4.) that this approach to generating post-test disease probabilities has never been validated and that the approach may result in very inconsistent results between clinicians.

In a cross-sectional cohort study of emergency and internal medicine residents and faculty, the authors presented clinical vignettes and asked clinicians to estimate the likelihood of common diseases based on history and physical exam findings. No lab or imaging results were provided. They reported that the mean pretest probability estimates of disease ranged from 42% (95% confidence interval [95% CI] = 36.6% to 47.4%) to 77% (95% CI = 72.9% to 81.1%). The smallest difference in pretest probability magnitude for a single vignette was 70% (range 30-100%; interquartile range [IQR] 64-80%), whereas the largest was 95% (range 3-98%; IQR 30-60%).

Their conclusions were that there is wide variability in clinicians’ estimates of pretest probability of disease and that post-test disease estimates are also likely to be inconsistent.

Because of such wide variability, we prefer to concentrate on more traditional statistics derived from the classical 2x2 table such as sensitivity, specificity, positive predictive value and negative predictive value and skip the hassle of estimating pretest probability and using nomograms or tables to generate post-test probabilities.

The abstract of this article can be found at:

Go to http://www.pubmed.gov and enter or copy and paste 15175211 in the search field.

Underuse of Proven Interventions

One of the main reasons for using valid, relevant evidence in health care is to more accurately predict outcomes from various interventions and thus be equipped to make informed choices. For example, it is very useful to know the evidence for outcomes by risk category for cardiovascular disease. The higher an individual’s risk for a cardiovascular event, the greater the benefit from interventions such as lipid-lowering with statins.

Because of increased benefits, we would expect patients who are at the highest baseline risk for CV events to receive the most aggressive lipid-lowering treatment. Ko et al. (JAMA.2004;291:1864-1870), however, have demonstrated that the prescription of statins in Canadian patients older than 66 years of age with a history of CV disease or diabetes mellitus decreased progressively as baseline CV risk increased.

In their database study:

Only 19% of the 75,617 patients received statins for secondary prevention;
In patient 66-74 years of age, the adjusted probabilities for receiving statin Rx was 37.7% (95% CI 37.3-38.2), 26.7% (26.4-27.1), and 23.4% (22.8-23.9) in the categories of low, intermediate and high baseline risk, respectively.

Why would such a small percentage of elderly patients with CV disease or diabetes be prescribed lipid-lowering drugs? Why would the likelihood that physicians prescribe statin therapy be inversely correlated with CV risk?

Comment: This is yet one more study demonstrating the huge problem of underuse of proven interventions in health care. However, this is apparently the first article pointing out the treatment-risk paradox and is worth reading. The authors suggest that the treatment risk paradox in these elderly patients might be explained by:

Clinicians not understanding tradeoffs between benefits and harms of statins—clinicians may believe that the results of RCTs of lipids cannot be applied to patients with comorbidites on the grounds that these patients may experience fewer benefits and greater harms than the study subjects;
Physicians may prejudge patients with co-morbidities—they may believe that such patients will not adhere to lipid-lowering treatment regimens; and/or,
Physicians may be inattentive to preventive care measures in elderly patients with chronic conditions.

Class Effect: Caution Urged

Pharmacy and Therapeutics committees and clinicians are frequently faced with issues regarding substitution for agents in a specific drug class. CD Furburg and BM Psaty, using ace-inhibitors (ACEIs) as an example, make a number of interesting points in an article entitled, “Should evidence-based proof of drug efficacy be extrapolated to a “class of agents”? (Circulation 2003;108:2608-2510). Below is a summary of some key points made by the authors of the article and also some points made by the editorialists, EM Antman and JJ Ferguson (Circulation 2003;108:2604-2607).

Generic substitution is the act of dispensing a different brand or an unbranded drug product that is the same chemical entity and meets US Food and Drug Administration criteria for bioequivalence (e.g., generic lisinopril in place of brandname Prinivil [Merck & Co.] or Zestril [Zeneca Pharmaceuticals]).

Therapeutic substitution is the dispensing of an alternate chemical entity for the original drug prescribed by the physician from the same general therapeutic class (e.g., a physician orders enoxaparin, but the hospital pharmacy dispenses dalteparin; a physician writes a prescription for ramipril, and the pharmacy dispenses moexipril).

The HOPE trial, using ramipril 10mg, reported the following relative risk reductions in patients with vascular disease or diabetes and one additional risk factor:

16% reduction over placebo in all cause mortality
20% reduction in MI
32% reduction in stroke

However, the evidence is less robust for some of the other ACEIs:

There was no decrease in ischemic events in subjects who had undergone angioplasty and took Quinapril 20mg when compared to placebo over 3 years
In the PROGRESS trial, there was no difference in stroke or coronary events in the Perindopril 4 mg and placebo groups. (However, in the EUROPA trial using 8 mg, there was a reduction in coronary mortality and morbidity of 20%.)
4 ACEIs have not been shown to reduce mortality or morbidity—2 agents have not been tested and 2 have not shown a reduction in clinical events.

Does it make sense to declare all of the ACEIs equivalent? What about differing doses that have not been tested?

Furburg and Psaty point out that all 7 drugs have FDA approval for treating hypertension, heart failure and LV dysfunction. All 7 have been shown to lower BP and improve hemodynamic measurements. However, only 4 of these drugs—enalapril, captopril, ramipril and trandolopril—have been shown to improve long term survival or clinical outcomes compared to placebo in large, RCTs.

They point out that:

The actions specific to an individual agent may add to, subtract from or have a neutral effect on the safety and efficacy attributed to the common class actions of a drug class.
Concluding that equipotency exists rests on surrogate measures of efficacy.
Equipotency is difficult to establish in the absence of relevant outcome data.
The term “class effect” has never been defined from a regulatory perspective.
The class effect concept is often overinterpreted to mean that all agents in a class are interchangeable.
Me-too drugs can be produced without the risk and expense of long term morbidity and mortality trials.
Regulatory agencies do not accept extrapolations, and FDA regulations prevent manufacturers from making claims for unapproved indications.
Safety is an issue to consider.

Given all of the above, the authors state that efforts should be made to use drugs with the best-documented outcomes and in full doses. Further, the use of me-too drugs without adequate documentation should be restricted. Comparability should be established in trials where two agents in a class are compared.

The editorialists list the following minimal criteria they would require for defining a class effect of a drug:

A clearly defined biological target or pathway.
Comparable efficacy demonstrated for multiple agents within the class (with multiple randomized, controlled clinical trials for each agent).
Absence of convincing evidence that there is a member of the class that does not have comparable clinical benefit to that of other agents within the class.

They also recommend considering the following before deciding whether drugs within a class are interchangeable:

The absolute and relative degree of benefit for each agent;
The clinical circumstances in which benefit has been demonstrated;
The extent and the depth of the evidence in favor of a particular agent;
The safety profile and tolerability of agents within a class;
The cost of the alternatives, viewed not only in terms of cost per quality life-year saved versus placebo or an established control, but also in terms of cost per quality life-year lost in the case of less expensive, but also less effective, alternatives;
The specific details of the inclusion/exclusion criteria and the exact management protocols used in the individual supporting clinical trials; and,
The particular subgroups that show benefit (or lack of benefit) for agents within the class.