Quick Picks
Why Critical Appraisal Matters
Services
Seminars
Books
Updates & Contact Info
Free Online Tools
Free Online Tutorial
Delfini Blog
Follow & Share...
Just-in-time Updates
Like Us  Find Us |
DelfiniGram™: Get on our quarterly update list
CASE STUDY: CRITICAL APPRAISAL AND SPECIAL ATTRITION ANALYSIS
Along with colleagues from Amgen, an abstract of ours was accepted by ISPOR (International Society For Pharmacoeconomics and Outcomes Research) for their Clinical Outcomes Studies Poster Presentations at their 20th Annual International Meeting. Among the many instructive lessons from this project includes an approach to evaluate the potential for attrition bias when attrition is high. Read on...
At ISPOR 20th Annual Internation Meeting
May 19, 2015
Delfini at the ISPOR 20th Annual International Meeting.
Michael E. Stuart MD & Sheri Ann Strite.
PROJECT BACKGROUND
We were invited by Amgen to perform a critical appraisal of three randomized controlled pivotal trials, Stopeck 10, Fizazi 11 and Henry 11, comparing the efficacy of denosumab versus zoledronic acid (ZA) for the prevention of sekeletal-related events (SREs), as well as a subsequent combined integrated analysis, Lipton 12, to help evaluate the validity of the research findings.
Funding for this review was provided by Amgen, Inc. Delfini's focus is on reliability of evidence and clinical usefulness of outcomes from a patient perspective and, as reviewers, Delfini had complete control over the entire review. Our review was based on the published trials as well as supplementary information as described in our abstract, reproduced below, which was accepted by ISPOR.
We found the three pivotal trials and the integrated analysis to be of high-quality evidence and at low risk of bias and chance effects based on a number of factors (see poster). Because attrition was high, as is often the case in oncology trials, we also conducted a special analysis to evaluate whether attrition was likely to result in attrition bias. Based on our analysis, with the exception of possible impacts of reduced study sample size (e.g., potential for insufficient power), we concluded that attrition bias was improbable.
Amgen contacted us subsequent to our review to explore preparing an abstract together for submission to ISPOR. That abstract was accepted, resulting in a poster presenation for ISPOR by Delfini authors along with Amgen staff, Vidya Setty Beckman and Katarina Öhrling [Strite 15].
DELFINI COMMENTARY
SREs
Many readers will be interested in the important topic of prevention of skeletal-related events for various cancers. Details, at a high level, of our critical appraisals and conclusions are available via the published abstract and poster.
Attrition
Of particular interest to many critical appraisers is how to approach studies with a high degree of attrition.
A key point is that attrition does not necessarily result in attrition bias. It may simply result in a smaller sample size. This can make the study more prone to sample size issues such as reduced power, but it might not otherwise distort study results. Meaning, there might not be any difference in study outcomes experienced by completers as compared to what outcomes might have occurred had more patients completed the trial.
Keep in mind that there are 4 potential realities:
- Intervention A > Intervention B
- Intervention A < Intervention B
- Intervention A = Intervention B
- Confounded intervention trumps
Some helpful questions to ask are—
- What would be required for any scenario to be true?
- Were those conditions met?
Therefore, our key question in exploring attrition became, "Considering attrition, what would be required for it to be false that denosumab is superior in efficacy to ZA, and were those conditions met?"
In this case study, a myriad of factors (see poster) supported the high quality of these studies, thus protecting against various biases that could otherwise distort results.
Key among these included—
- No confounding treatments included in the analysis;
- Unlikely imbalanced groups or treatment of groups due to likely successful randomization, effective concealment of allocation and likely success of blinding, as supported by blinding methods plus review of safety data suggesting that adverse events would be unlikely to unblind participants;
- Very high degree of adherence;
- Very low incidence of protocol violations;
- Balance in numbers and reasons for discontinuations in each of the many categories reported; and,
- Patterns through similar outcomes across studies.
We had data on reported SREs for both completers and discontinueds in each group. Therefore, given the above, it seems reasonable to surmise that only an imbalance of undetected SREs between the groups could overturn the direction of the efficacy results. However, due to all the study strengths that protect against treating the groups differently, undetected SREs would be likely to be equal between groups, resulting in no change in outcomes. For us, that was all we needed to conclude that the efficacy results are likely to be truly in favor of denosumab.
DELFINI CONCLUSION
Attrition bias is the bias category for which there is the least helpful and conclusive information available of all the research we have studied on the distorting effects of bias. We think that this inconclusivity is due to the fact that attrition does not always equate with attrition bias, but has the potential for being neutral (with the exception of potential issues related to study size, in the context of the specific study in question), whereas other study biases, such as lack of blinding, do not share this neutrality and generally put a study at a high risk of bias depending upon the specific situation.
Therefore, we think this case study may be a useful example in helping other critical appraisers think about how to explore factors in other studies in which attrition is high.
........
Side Exploration
Even though we were comfortable with our analysis and conclusion, we decided to do another hypothetical exploration just for the sake of doing so and because we are conservative.
Important note: we consider this hypothetical exploration unnecessary given the above, and it was undertaken only as a theoretical and conservative exercise.
For denosumab not to be superior to ZA, subjects in the ZA group who discontinued the study would have had to experience a sizable decrease in SRE rates compared with subjects in the ZA group who remained on-study.
How could this be? It would seem that this could only happen if ZA subject were treated differently than denosumab subjects—resulting in a cohort of ZA patients who were selectively removed from the study or encouraged to withdraw from the study. But various factors above (effective blinding) make it unlikely that this could be.
Bottom Line
It is unlikely that subjects in either the ZA or denosumab groups who discontinued would have had a better treatment response than subjects who remained on-study in each group. Even if subjects who discontinued would have had a better treatment response than those who
remained on-study, the rates would likely be similar due to balance in numbers and reasons for discontinuation. This would result in no change to the reported differences between the treatment groups.
This side exploration lends further support to our conclusion that the efficacy results are likely to be truly in favor of denosumab.
........
PUBLISHED ABSTRACT (return to top)
Objectives
Rigorous critical appraisal of clinical trials to assess bias and chance effects, which can distort trial results, can help evaluate the validity of research findings. In a pre-specified integrated analysis of three phase 3 pivotal trials in patients with bone metastases secondary to breast cancer, prostate cancer, and other solid tumors or multiple myeloma (N=5,723), denosumab was reported to be superior to ZA for the prevention of SREs, with statistically and clinically significant differences. Delfini Group performed critical appraisals of the three individual pivotal trials and the integrated analysis.
Methods
Published trials (Lipton et al, EJC, 2012; Stopeck et al, JCO, 2010; Henry et al, JCO, 2011, Fizazi et al, Lancet, 2011) were analyzed along with supplementary information from study protocols, clinical study reports, and prescribing information. The trials were evaluated for any potential threats to study validity, such as selection bias,
performance bias, and assessment bias, as well as the likelihood for chance effects instead of true effects. A detailed analysis of attrition (discontinuation or loss to follow up), which is a frequent issue in oncology studies, was also conducted to assess the presence of attrition bias.
Results
The integrated analysis and three pivotal trials were found to be of high-quality evidence and at low risk of bias and chance effects. Important quality features included a robust randomization process, high likelihood of patients remaining balanced and blinded throughout the study, as well as a high degree of adherence to assigned treatments. These and other factors, such as balance in co-interventions and in rates and reasons for discontinuation, make bias from attrition unlikely.
Conclusions
The critical appraisal confirmed that the results of the integrated analysis and three pivotal trials were robust, with denosumab providing clinically meaningful benefit in patients with bone metastases from advanced cancer.
Poster is available for viewing or dowload here.
References
- Fizazi K, Carducci M, Smith M, Damião R, Brown J, Karsh L, Milecki P, Shore N, Rader M, Wang H, Jiang Q, Tadros S, Dansey R, Goessl C. Denosumab versus zoledronic acid for treatment of bone metastases in men with castration-resistant prostate cancer: a randomised, double-blind study. Lancet. 2011 Mar 5;377(9768):813-22. Epub 2011 Feb 25. PubMed PMID: 21353695; PubMed Central PMCID: PMC3090685.
- Henry DH, Costa L, Goldwasser F, Hirsh V, Hungria V, Prausova J, Scagliotti GV, Sleeboom H, Spencer A, Vadhan-Raj S, von Moos R, Willenbacher W, Woll PJ, Wang J, Jiang Q, Jun S, Dansey R, Yeh H. Randomized, double-blind study of denosumab versus zoledronic acid in the treatment of bone metastases in patients with advanced cancer (excluding breast and prostate cancer) or multiple myeloma. J Clin Oncol. 2011 Mar 20;29(9):1125-32. Epub 2011 Feb 22. PubMed PMID: 21343556.
- Lipton A, Fizazi K, Stopeck AT, Henry DH, Brown JE, Yardley DA, Richardson GE, Siena S, Maroto P, Clemens M, Bilynskyy B, Charu V, Beuzeboc P, Rader M, Viniegra M, Saad F, Ke C, Braun A, Jun S. Superiority of denosumab to zoledronic acid for prevention of skeletal-related events: a combined analysis of 3 pivotal, randomised, phase 3 trials. Eur J Cancer. 2012 Nov;48(16):3082-92. doi: 10.1016/j.ejca.2012.08.002. Epub 2012 Sep 10. PubMed PMID: 22975218.
- Stopeck AT, Lipton A, Body JJ, Steger GG, Tonkin K, de Boer RH, Lichinitser M, Fujiwara Y, Yardley DA, Viniegra M, Fan M, Jiang Q, Dansey R, Jun S, Braun A. Denosumab compared with zoledronic acid for the treatment of bone metastases in patients with advanced breast cancer: a randomized, double-blind study. J Clin Oncol. 2010 Dec 10;28(35):5132-9. Epub 2010 Nov 8. PubMed PMID: 21060033.
- Strite SA, Stuart ME, Beckman V, Öhrling K. Comparing the efficacy of denosumab versus zoledronic acid (ZA) for prevention of skeletal-related events (SREs): a critical appraisal of three pivotal trials. ISPOR (International Society For Pharmacoeconomics and Outcomes Research) 20th Annual International Meeting—Clinical Outcomes Studies Poster Presentation. May 19, 2015.
........
CONTACT DELFINI  |
At Home
Read Our Blog...
Menu........
Use of our website implies agreement to our Notices. Citations for references available upon request.
Home
Best of Delfini
What's New
Blog
Seminars
Services
Delfini Group Publishing
Resources
Sample Projects
Notices
About
Us & Our Work
Testimonials
Other
Site Search
Contact
Info/Updates
........
Quick
Navigator to Selected Resources
.......................
|
