Use of Evidence:
Applying the Evidence

newest
03/04/08: Messaging Scripts for Implementing Clinical Practice Change at On the Same Page™: Clinician + Patient Communications »

Contents      

• Guidelines & Effectiveness of Implementation »
• Oregon Preferred Drug List »
• Successful Evidence-based QI Project: Diabetes Management at Dreyer Medical Clinical »
• External Validity: Case of the Carotid Stent »
• Also see On the Same Page™ for applying the evidence through patient-centered resources »

Guidelines & Effectiveness of Implementation

Tremendous effort goes into the development of high quality clinical guidelines and effective disease management programs. But, there is much uncertainty about how to effectively implement the final product — the evidence-based clinical improvements. In this systematic review, Weingarten et al report that clinicians do change their practice based on provider education efforts, among other strategies. Click to learn more about improvements in care directed at clinicians and patients.

Link to the Abstract (Full Text Available)

BMJ 2002;325:925 ( 26 October )

Weingarten S. et al. Interventions used in disease management programmes for patients with chronic illness — which ones work? Meta-analysis of published reports.

http://bmj.com/cgi/content/abstract/325/7370/925

Oregon Preferred Drug List: Conference Report

"Adding Value to the Cost Equation with Prescription Drugs: The Oregon Experience" — Tuesday, February 25th, Seattle Washington

Mike was on a panel, including many notables. Keynote address was by John Kitzhaber, MD, Former Governor, State of Oregon. The conference was a great success. Read about the program at —

http://seattletimes.nwsource.com/cgi-bin/ PrintStory.pl?document_id=134641620&
zsection_id=268448406&slug= prescription26m&date=20030226

Successful Evidence-based QI Project: Diabetes Management at Dreyer Medical Clinical
Example provided by Rami Rihani, PharmD, Director of Pharmacy

Delfini Introduction
Measuring clinical improvements is complex. One of the most important, frequently misunderstood issues is that cause and effect relationships can only be drawn with reasonable certainty from valid experiments (RCTs). However, if we have valid evidence from RCTs that an intervention leads to improved clinical outcomes, it is then reasonable to use process measures to evaluate the success of our evidence-based clinical improvement project.

Generally, we advise people to measure — not health status outcomes — but to perform a process measurement to evaluate the success of application of the intervention. In other words, we advise people to measure the success of implementation of the clinical improvement. For example, if we are trying to ensure patients get a beta-blocker post-MI, we would recommend looking to see if prescriptions increased for hospitalized MI patients — not to measure whether patient survival was improved. This is because observational data, such as information extracted from databases, can be highly prone to confounding. If health status outcomes are measured, then we advise people to ensure that there is a sufficient understanding of all those utilizing the data that conclusions drawn from observational data can be misleading. In the above example, if patient survival decreased, there could be many explanations.

However, if a health status outcome is measured, and if the before/after change is dramatic, it is reasonable to hypothesize that our project has been successful. For example…

Problem
Many diabetics have difficulty achieving a HbA1c <7.0. Frequently diabetics are told their HbA1cs are too high but active medication change is not aggressively pursued.

Evidence-based QI Project: A quality improvement group at Dreyer Medical Clinic developed a disease management initiative using PharmDs to actively titrate dosages of insulin and other drugs based on the Intermountain Health Care (IHC) diabetes management protocol. The process is as follows:

• Primary care physician (PCP) refers patient to the diabetes management program;
• PharmD aggressively titrates medication based on IHC protocol;
• PharmD monitors for safety and efficacy of medication interventions in collaboration with the PCP

Outcomes

Delfini Commentary
There was a significant improvement in the percent of patients achieving goal HbA1c and LDL associated with this project.

It is reasonable to believe that the clinical improvement project was successful. Using outcomes data from the UK Prospective Diabetes Study 35 (1), the QI team estimates that since inception, the disease management initiative resulted in the prevention of —

• four diabetes related deaths and
• nine microvascular events (defined as renal failure, death from renal failure, retinal photocoagulation, or vitreous hemorrhage)

1. Stratton, I,M., Adler, A.I., et al, Association of glycaemia with macrovascular and microvascular complications of type 2 diabetes (UKPDS 35): prospective observation study. BMJ 2000; 321; 405-12.

The lead article in the Oct. 7, 2004 issue of The New England Journal of Medicine, (Yadav JS, Wholey MH, Kuntz RE, et al. Protected Carotid-Artery Stenting versus Endarterectomy in High-Risk Patients N Engl J Med 2004;351:1493-501 — PMID: 15470212 — abstract), known as the SAPPHIRE trial, is an RCT with what appears to be adequate randomization/concealment of allocation, similar baseline characteristics of subjects, adequate blinding and with intention to treat analysis comparing carotid stenting using an embolic protection device to endarterectomy in patients with symptomatic carotid stenosis of at least 50 percent or asymptomatic stenosis of at least 80 percent and at least one high risk factor such as recurrent stenosis after endarterectomy e.g., age>80, cardiac disease or other significant medical problems. More than 20% of the patients in both treatment groups were patients with recurrent stenosis after endarterectomy.

The primary end point was a composite of death, stroke or MI within 30 days after the intervention, or death or ipsilateral stroke between 31 days and 1 year. The study appeared to be a sufficiently powered equivalence study. Study team members included both a surgeon and an interventionist who could prevent randomization of enrolled subjects if, in their judgment, the procedures could not be performed safely in these high risk patients. 747 patients were enrolled and 334 underwent randomization. The primary endpoint occurred in 20 patients in the stenting group (12.2 percent) and in 32 in the endarterectomy group (20.1%), yielding a P value of 0.004 for non-inferiority. At one year, revascularization was repeated in 0.6 percent of the stent patients and 4.3 percent of the endarterectomy patients; P=0.04. The authors conclude that in patients with severe carotid-artery stenosis and coexisting conditions, carotid stenting with an emboli protection device is not inferior to carotid endarterectomy.

Question
What are the biggest threats to validity in this study (based on the above information)?

Our Answer
There is a huge problem with selection bias and external validity in this study. More than 20% of the patients in both treatment groups were patients with recurrent stenosis after endarterectomy which may bias the results towards stenting. 55% of enrolled study patients were excluded by the physician team raising further concerns about selection bias and issues surrounding highly selected patients.