The E9(R1) “Addendum on Estimands and Sensitivity Analyses in Clinical Trials,” adopted by the International Conference on Harmonization (ICH) in November 2019,¹ introduced the “estimand framework” for clinical trials. The goal of this framework is to ensure clarity in the descript ion of treatment effects by incorporating in the effect’s definition not only the endpoint and the population level summary but also the so-called intercurrent events, such as use of rescue or prohibited medications, withdrawal of study treatment due to adverse events (AEs) or disease progression, leaving the study due to consent withdrawal, etc.
In addition to this guideline and the accompanying training slides,² several papers with a tutorial-like approach on estimands were published in recent years, including Ratitch et al.,³ Keene et al.⁴ and Clark et al. ⁵ A paper by Fletcher et al.⁶ reports the work of the Estimand Implementation Working Group, formed in October 2019 and sponsored by the European Federation of Statisticians in the Pharmaceutical Industry and the European Federation of Pharmaceutical Industries and Associations. The paper by Fletcher et al. shares the pharmaceutical industry experience in implementing the estimand framework in the first two years since the final guidance on estimands became available. Key lessons learned, emerging best practices, and points to consider on strategies for implementing the new estimand framework are provided.
The number of scientific publications emerging on estimands is increasing year over year; a PubMed search for “estimand” conducted in October 2023 resulted in 555 hits. However, the current discussion on estimands mainly focuses on efficacy evaluation in individual clinical studies.
There is little discussion and no specific guideline on estimands in the context of safety assessments and integrated analyses. It is reasonable to predict that to fully evaluate the benefit/risk of a new medicine, it will become critical to use estimands in the integrated summaries of efficacy (ISE) and safety (ISS).
The ICH E9⁷ states that “any statistical procedures used to combine data across trials should be described in detail” and that “attention should be paid […] to the proper modelling of the various sources of variation.” The use of meta-analysis techniques is encouraged, particularly the techniques based on individual patient data (pooled analyses) from the studies to be combined.
The white paper ‘‘Considerations on the Use of Estimands Within the Integrated Summary of Effectiveness and Integrated Summary of Safety, and Benefit/Risk Analysis’’ describes the state of the art in the use of estimands in the field of ISE, ISS, and benefit/risk analyses.
On the ISE side, the use of pooled analysis techniques is well established. There is an expectation that the increasingly common use of pre-defined estimands in the individual clinical trials to be combined in an ISE will alleviate an important source of heterogeneity. In fact, before implementing the estimand framework, pooled analyses were used to combine clinical studies based on the sam e endpoint definition, with little or no attention to intercurrent events that could have been managed in different ways, leading to a heterogeneous impact on
the analysis results. Through the estimands, the intercurrent events are incorporated in the treatment effect definition, opening the space for more transparent and hopefully homogeneous pooled analyses.
The combined studies might have implemented different strategies to take care of intercurrent events, for example, some studies may have used the treatment strategy and others the hypothetical or the composite strategies. One of the objectives of an ISE might be defining a target estimand implementing a common
strategy for all studies. An example of a primary efficacy estima nd based on the treatment policy strategy, reported in the statistical analysis plan of an ISE on a new in vestigational medicinal product (IMP) for asthma, is described below.
Difference between Investigational Medicinal Products (IMP) and placebo in the change from baseline t o Week 24 of forced expiratory volume (FEV)₁ regardless of adherence (i.e., whether the patient is able to remain on treatment) and whether additional or alternative medication is used. This estimand is specified by the following four attributes:
A table providing the details for the actual implementation of this target estimand at the study level would be needed as well as a method to account for missing data in the statistical analysis.
In some cases it is not possible to implement a common target estimand. This could happen in the example above because in some of the studies, patients prematurely stopped the study treatment and/or patients
taking alternative medications were withdrawn from the study. In this case a reasonable approach could be to group the studies based on the primary estimand originally d efined at the study level. Another alternative might be the identification of treatment effects in strata based on potential experience of intercurrent events. Estimands based on Principal Stratum strategies might be useful for this purpose, i.e., to estimate the treatment effect in the subset of patients in whom the intercurrent event( s) of interest would (or would not) occur, for example, the subset of patients who would not use rescue medication regardless of which treatment arm they were assigned.
On the safety side, some publications on estimands are available, including Unkel et al.⁸ and Wang et al.⁹ Using the estimand terminology, common practice evaluates the safety of the treatment using an estimand based on the While on Treatment strategy, which includes the AEs until discontinuation of treatment (often plus a pre-defined follow-up time interval, typically one month) and requires the collection of AE data up to this time. Such simple analyses may be biased by different observational times (both within and between studies) and differential occurrence of intercurrent events.
Competing risks for targeted AEs add another layer of complexity. For example, as explained in Unkel et al., in a trial involving patients with type 2 diabetes and an elevated risk of cardiovascular disease, patients treated with the active treatment may be shown to have a much lower risk of cardiovascular events but a greater risk of amputation than those who received placebo. In such a situation, the actual risk of amputation might be difficult to determine in the placebo group because of the muc h larger risk of mortality, which precludes the observation of amputation. In this specific example, if the objective is the estimation of the amputation risk, cardiovascular mortality should be explicitly treated as an important intercurrent event and an adequate strategy for its evaluation should be implemented. The paper by Unkel et al. presents, within the framework of
estimands, statistical methods to analyze AEs and recommendations for which estimators should be used for the estimands described.
Accounting for these factors in the ISS may be very difficult wit hout having addressed these complicated aspects at the individual study level. Problems and possible solutions in the meta-analyses of AE data are addressed in Unkel et al.
A paper by Ren et al.¹⁰ initiates the discussion on benefit/risk analysis under the estimand framework. Among the other considerations, it is important to recognize upfront the relationship between efficacy and safety estimands. For example, Ratitch et al.¹¹ note that efficacy estimands may incorporate intercurrent events that could reflect tolerability of treatment or key safety considerations. This may lead to counting these risks twice in the benefit/risk analysis.
The paper by Fletcher et al. highlights what else needs to be done to continue the journey in embedding the estimand framework in clinical trials. Use of estimands in the context of ISE is already possible and should be encouraged. More work is needed when planning the individual clinical studies, both at the theoretical and practical level, to implement the estimand framework in the areas of ISS and benefit/risk analysis. Deepening the discussion on estimands for the ISS and benefit/risk analysis is definitely on the “to do list.”
Subscribe to our newsletter for the latest news, events, and thought leadership