Downloaded from http://esmoopen.bmj.com/ on November 13, 2016 - Published by group.bmj.com
Open Access Review
ESMO - Magnitude of Clinical Benefit
Scale V.1.0 questions and answers
N I Cherny,1 R Sullivan,2 U Dafni,3 J M Kerst,4 A Sobrero,5 C Zielinski,6
M J Piccart,7 J Bogaerts,8 J Tabernero,9 N J Latino,10 E G E de Vries11
To cite: Cherny NI, ABSTRACT ▸ When governments are the payer, as is
Sullivan R, Dafni U, et al. The ESMO Magnitude of Clinical Benefit Scale common in many European countries, we
ESMO - Magnitude of Clinical (ESMO-MCBS) is a standardised, generic, validated tool
Benefit Scale V.1.0 questions
envisage that the ESMO-MCBS will assist
to stratify the magnitude of clinical benefit that can be in the Health Technology Assessment
and answers. ESMO Open
2016;1:e000100.
anticipated from anticancer therapies. The ESMO-MCBS process. This is described in our paper1
doi:10.1136/esmoopen-2016- is intended to both assist oncologists in explaining the
where we write: “Grading derived from
000100 likely benefits of a particular treatment to their patients
as well as to aid public health decision makers’ prioritise
the ESMO-MCBS provides a backbone for
therapies for reimbursement. From its inception the value evaluations for cancer medicines.
▸ Prepublication history and Medicines and therapies that fall into the
additional material for this
ESMO-MCBS Working Group has invited questions and
critiques to promote understanding and to address ESMO-MCBS A+B for curative therapies
paper is available online. To
view these files please visit misunderstandings regarding the nuanced use of the and 4+5 for non-curative therapies should
the journal online scale, and to identify shortcomings in the scale to be be highlighted for accelerated assessment
(http://dx.doi.org/10.1136/ addressed in future planned revisions and updates. The of value and cost-effectiveness. While a
esmoopen-2016-000100). ESMO-MCBS V.1.0 has attracted many questions high ESMO-MCBS score does not auto-
regarding its development, structure and potential matically imply high value (that depends
Received 12 August 2016 applications. These questions, together with responses
Revised 31 August 2016 on the price), the scale can be used to
from the ESMO-MCBS Working Group, have been edited
Accepted 1 September 2016 frame such considerations and can help
and collated, and are herein presented as a
supplementary resource.
public policymakers advance ‘accountabil-
ity for reasonableness’ in resource alloca-
tion deliberations.”
INTRODUCTION SCALE STRUCTURE AND CRITERIA
The ESMO Magnitude of Clinical Benefit
Scale (ESMO-MCBS) is a standardised, Will ESMO consider taking proportional
generic, validated tool to stratify the magni- benefit (ie, a percentage gain relative to
tude of clinical benefit that can be antici- current survival in orphan or difficult-to-treat
pated from anticancer therapies. The cancers) into account beyond focusing on
ESMO-MCBS is intended to both assist oncol- absolute overall survival (OS)/progression-free
ogists in explaining the likely benefits of a survival (PFS)? If proportionality were to be
particular treatment to their patients as well taken into account, then the percentage gain
as to help decision makers prioritise out- of OS for a drug in a rare or difficult-to-treat
standing new drugs for reimbursement. cancer would have quite a different benefit
rating. Equally some of the cancers for which
survival is already very good might have their
SCALE DEVELOPMENT rankings changed.
Answer
Which payers/reimbursement agencies were ▸ All of the quantitative elements of the
consulted on the use of this tool? How does scale, both for PFS and OS, take into
ESMO envisage payers using the account both absolute gain and hazard
ESMO-MCBS? ratio (HR). HR data takes proportionality
For numbered affiliations see Answer into account.
end of article.
▸ The scale has not been discussed with any ▸ The scale is stratified for diseases with
payer organisations. better and worse prognosis, that is, there
Correspondence to
▸ The scale was developed as a tool to is a different scoring of PFS for disease
N J Latino, ESMO Head
Office, via Luigi Taddei 4,
derive clear and unbiased evaluation of with PFS in the control arm >6 months or
6962 Viganello - Lugano, the magnitude of clinical benefit based <6 months, and for median OS
Switzerland; mcbs@esmo.org on published peer-reviewed data. >12 months or <12 months.
Cherny NI, et al. ESMO Open 2016;1:e000100. doi:10.1136/esmoopen-2016-000100 1
, Downloaded from http://esmoopen.bmj.com/ on November 13, 2016 - Published by group.bmj.com
Open Access
If the ESMO-MCBS is only to be applied to comparative QoL/grade 3–4 toxicities assessment
studies, what will that mean for breakthrough therapies,
Mark with X
licenced on single arm data? if relevant
Answer
▸ The grading of single arm data will be incorporated Was quality of life (QoL) evaluated as
secondary outcome?
into version 1.1 of the ESMO-MCBS.
Does secondary end point QoL show
Evaluation form 1—for new approaches to adjuvant improvement
therapy or new potentially curative therapies grade A (for Are there statistically significantly less
grade B and C analogously): What is the specific meaning grade 3–4 toxicities impacting on daily
of ‘in studies without mature survival data’? Assuming we well-being*
have survival data at 3 years but the improvement is 4%. *This does not include alopecia and myelosuppression, but rather
At the same time, we have an improvement in disease- chronic nausea, diarrhoea, fatigue, etc.
free survival (DFS) of HR=0.60. What would then be the
Adjustments
correct grading? Grade A (both conditions connected by
A. Downgrade 1 level if there is one or more of the
OR) or grade B (condition on DFS only matters if 3 years
above incremental toxicities associated with the new
survival data are missing)?
drug
Answer
B. Upgrade 1 level if improved QoL or if less grade 3–4
▸ In general, once mature survival data becomes avail-
toxicities that bother patients are demonstrated
able, the DFS, which is a surrogate indicator of sur-
C. When OS as secondary end point shows improve-
vival, becomes redundant and in the example given
ment, it will prevail and the new scoring will be
here the ESMO-MCBS score would be B.
carried out according to form 2a
▸ However, in rare instances in which DFS was the
D. Downgrade 1 level if the drug ONLY leads to
primary outcome and where there was major cross-
improved PFS and QOL assessment does not demon-
over because of early analysis leading to an unblind-
strate improved QoL
ing of the randomisation (as happened in some of
the trastuzumab trials) the DFS scoring would prevail. Final, toxicity and QoL adjusted, magnitude clinical
benefit grade
How is the statistical significance of effects incorporated
4 3 2 1
in your method? Since it is not explicitly mentioned in the
text of the article, can we assume that a statistically
Highest magnitude clinic benefit grade that can be achieved
significant result (ie, p value <0.05) for a primary end grade 4.
point is a general requirement for a positive grading?
Answer ▸ In PFS studies in which QoL is evaluated, a positive
▸ YES. In the section ‘eligibility for application of the outcome demonstrating either improved QoL or
ESMO-MCBS’ the article1 is very specific: ‘The delayed deterioration in QoL provides further sup-
ESMO-MCBS can be applied to comparative outcome portive evidence regarding the significance of the
studies evaluating the relative benefit of treatments PFS advantage reported.
using outcomes of survival, quality of life (QoL), sur- – If improved QoL OR delayed deterioration in QoL
rogate outcomes for survival or QoL (disease-free is observed in QoL evaluation (statistically signifi-
interval, event-free survival (EFS), time to recurrence, cant), then the score can be upgraded by 1 point.
PFS and time to progression) or treatment toxicity in – If however, PFS improvement is not accompanied
solid cancers. Eligible studies can have either a rando- by OS advantage and evaluation of QoL does not
mised or comparative cohort design or a confirm that QoL was either improved or deterior-
meta-analysis which report statistically significant ation is delayed, this essentially devalues the PFS
benefit from any one or more of the evaluated and scores are reduced by 1 point.
outcomes’. ▸ In our field testing there were many instances of scores
being upgraded to 4 based on both high PFS score and
Can you advise if there is any upgrading of the improved QoL. There were three instances where PFS
ESMO-MCBS if QoL is evaluated as a secondary outcome scores were downgraded because PFS was not associated
(eg, first question listed below is marked)? Or is there with either statistically significant improvement in OS or
only an upgrading of 1 point if QoL is improved or there is a significant positive effect on QoL.
less toxicity (eg, only if second or third question listed ▸ The rationale for this approach is that PFS is, in
below is marked)? general, an unreliable surrogate for both OS and
Answer QoL improvement, and secondary outcomes can
▸ YES. Upgrading for improved QoL is incorporated in either lend veracity to the findings or indicate that
to the evaluation of both OS studies (form 2a) and they are of lesser clinical significance to patient
PFS studies (form 2b illustrated below) outcomes.
2 Cherny NI, et al. ESMO Open 2016;1:e000100. doi:10.1136/esmoopen-2016-000100
