Alira Health

Considerations on the Use of Estimands Within the Integrated Summary of Effectiveness and Integrated Summary of Safety, and Benefit/Risk Analysis

14 minute read Download

Estimand Framework: Introduction

The E9(R1) “Addendum on Estimands and Sensitivity Analyses in Clinical Trials,” adopted by the International Conference on Harmonization (ICH) in November 2019,¹ introduced the “estimand framework” for clinical trials. The goal of this framework is to ensure clarity in the descript ion of treatment effects by incorporating in the effect’s definition not only the endpoint and the population level summary but also the so-called intercurrent events, such as use of rescue or prohibited medications, withdrawal of study treatment due to adverse events (AEs) or disease progression, leaving the study due to consent withdrawal, etc.

In addition to this guideline and the accompanying training slides,² several papers with a tutorial-like approach on estimands were published in recent years, including Ratitch et al.,³ Keene et al.⁴ and Clark et al. ⁵ A paper by Fletcher et al.⁶ reports the work of the Estimand Implementation Working Group, formed in October 2019 and sponsored by the European Federation of Statisticians in the Pharmaceutical Industry and the European Federation of Pharmaceutical Industries and Associations. The paper by Fletcher et al. shares the pharmaceutical industry experience in implementing the estimand framework in the first two years since the final guidance on estimands became available. Key lessons learned, emerging best practices, and points to consider on strategies for implementing the new estimand framework are provided.

The number of scientific publications emerging on estimands is increasing year over year; a PubMed search for “estimand” conducted in October 2023 resulted in 555 hits. However, the current discussion on estimands mainly focuses on efficacy evaluation in individual clinical studies.

There is little discussion and no specific guideline on estimands in the context of safety assessments and integrated analyses. It is reasonable to predict that to fully evaluate the benefit/risk of a new medicine, it will become critical to use estimands in the integrated summaries of efficacy (ISE) and safety (ISS).

The ICH E9⁷ states that “any statistical procedures used to combine data across trials should be described in detail” and that “attention should be paid […] to the proper modelling of the various sources of variation.” The use of meta-analysis techniques is encouraged, particularly the techniques based on individual patient data (pooled analyses) from the studies to be combined.

The white paper ‘‘Considerations on the Use of Estimands Within the Integrated Summary of Effectiveness and Integrated Summary of Safety, and Benefit/Risk Analysis’’ describes the state of the art in the use of estimands in the field of ISE, ISS, and benefit/risk analyses.

Jump to a section:

Use of Estimands in the ISE

On the ISE side, the use of pooled analysis techniques is well established. There is an expectation that the increasingly common use of pre-defined estimands in the individual clinical trials to be combined in an ISE will alleviate an important source of heterogeneity. In fact, before implementing the estimand framework, pooled analyses were used to combine clinical studies based on the sam e endpoint definition, with little or no attention to intercurrent events that could have been managed in different ways, leading to a heterogeneous impact on
the analysis results. Through the estimands, the intercurrent events are incorporated in the treatment effect definition, opening the space for more transparent and hopefully homogeneous pooled analyses.

The combined studies might have implemented different strategies to take care of intercurrent events, for example, some studies may have used the treatment strategy and others the hypothetical or the composite strategies. One of the objectives of an ISE might be defining a target estimand implementing a common
strategy for all studies. An example of a primary efficacy estima nd based on the treatment policy strategy, reported in the statistical analysis plan of an ISE on a new in vestigational medicinal product (IMP) for asthma, is described below.

Difference between Investigational Medicinal Products (IMP) and placebo in the change from baseline t o Week 24 of forced expiratory volume (FEV)₁ regardless of adherence (i.e., whether the patient is able to remain on treatment) and whether additional or alternative medication is used. This estimand is specified by the following four attributes:

  • Treatment: treatment condition of interest is IMP regardless of whether the patient continues with it until week 24 and whether additional or alternative medication is used; alternative treatment condition is placebo regardless of whether the patient continues with it until week 24 and whether additional or alternative medication is used.
  • Population: patients with asthma.
  • Variable: change from baseline to week 24 in FEV1. How to account for intercurrent events: already covered in treatment attribute.
  • Population-level summary for the variable: difference between treatments in mean changes from baseline at week 24.

A table providing the details for the actual implementation of this target estimand at the study level would be needed as well as a method to account for missing data in the statistical analysis.

In some cases it is not possible to implement a common target estimand. This could happen in the example above because in some of the studies, patients prematurely stopped the study treatment and/or patients
taking alternative medications were withdrawn from the study. In this case a reasonable approach could be to group the studies based on the primary estimand originally d efined at the study level. Another alternative might be the identification of treatment effects in strata based on potential experience of intercurrent events. Estimands based on Principal Stratum strategies might be useful for this purpose, i.e., to estimate the treatment effect in the subset of patients in whom the intercurrent event( s) of interest would (or would not) occur, for example, the subset of patients who would not use rescue medication regardless of which treatment arm they were assigned.

Use of Estimands in the ISS

On the safety side, some publications on estimands are available, including Unkel et al.⁸ and Wang et al.⁹ Using the estimand terminology, common practice evaluates the safety of the treatment using an estimand based on the While on Treatment strategy, which includes the AEs until discontinuation of treatment (often plus a pre-defined follow-up time interval, typically one month) and requires the collection of AE data up to this time. Such simple analyses may be biased by different observational times (both within and between studies) and differential occurrence of intercurrent events.

Competing risks for targeted AEs add another layer of complexity. For example, as explained in Unkel et al., in a trial involving patients with type 2 diabetes and an elevated risk of cardiovascular disease, patients treated with the active treatment may be shown to have a much lower risk of cardiovascular events but a greater risk of amputation than those who received placebo. In such a situation, the actual risk of amputation might be difficult to determine in the placebo group because of the muc h larger risk of mortality, which precludes the observation of amputation. In this specific example, if the objective is the estimation of the amputation risk, cardiovascular mortality should be explicitly treated as an important intercurrent event and an adequate strategy for its evaluation should be implemented. The paper by Unkel et al. presents, within the framework of
estimands, statistical methods to analyze AEs and recommendations for which estimators should be used for the estimands described.

Accounting for these factors in the ISS may be very difficult wit hout having addressed these complicated aspects at the individual study level. Problems and possible solutions in the meta-analyses of AE data are addressed in Unkel et al.

Benefit/Risk Analysis

A paper by Ren et al.¹⁰ initiates the discussion on benefit/risk analysis under the estimand framework. Among the other considerations, it is important to recognize upfront the relationship between efficacy and safety estimands. For example, Ratitch et al.¹¹ note that efficacy estimands may incorporate intercurrent events that could reflect tolerability of treatment or key safety considerations. This may lead to counting these risks twice in the benefit/risk analysis.

Conclusion

The paper by Fletcher et al. highlights what else needs to be done to continue the journey in embedding the estimand framework in clinical trials. Use of estimands in the context of ISE is already possible and should be encouraged. More work is needed when planning the individual clinical studies, both at the theoretical and practical level, to implement the estimand framework in the areas of ISS and benefit/risk analysis. Deepening the discussion on estimands for the ISS and benefit/risk analysis is definitely on the “to do list.”

How Alira Health Helps Sponsors With End-to-End ISE/ISS Support

  • Clarification of how each study will be used in the ISE/ISS
  • Strategic statistical support to write the integrated analysis plan, including definition of sub-populations and grouping of trials, including interaction with health authorities
  • Gap analysis to evaluate availability and CDISC compliance of study data to be included in integrated analyses or submitted in eCTD Module 5
  • Writing of the study data standardization plan (SDSP) as required for pre-NDA discussions with the FDA
  • Defending integrated analysis plan and SDSP during Type C or pre-NDA meetings with FDA or scientific advice meetings with EMA
  • Programming, validation, documentation of SDTM and ADaM pooled datasets
  • Coding harmonization to the latest WHODrug and MedDRA dictionary versions
  • Conversion of legacy data into CDISC format
  • Statistical analyses, including reconciliation with the clinical study reports
  • Expert medical writing services and statistical support for the development of eCTD Modules 2 and 5

References

  1. ICH E9 (R1) addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials (accessed 27 July 2021): E9-R1_Step4_Guideline_2019_ 1203.pdf
    https://www.ema.europa.eu/en/documents/scientific-guideline/ich-e9-r1-addendum-estimands-and-sensitivity-analysis-clinical-trials-guideline-statistical-principles-clinical-trials-step-5_en.pdf
  2. Training slides on the ICH E9 (R1) addendum.
    https://ich.org/page/efficacy-guidelines
  3. Ratitch B et al. Defining efficacy estimands in clinical trials: examples illustrating ICH E9 (R1) guidelines. Ther Innov Regul Sci. 2020;54(2):370–84. 4.
    https://doi.org/10.1007/s43441-019-00065-7
  4. Keene DW et al. Why ITT analysis is not always the answer for estimating treatment effects in clinical trials. Contemp Clin Trials. 2021;108:106494.
    https://doi.org/10.1016/j.cct.2021.106494
  5. Clark TP et al. Estimands: Bringing clarity and focus to research questions in clinical trials. BMJ Open. 2022;12(1).
    https://doi.org/10.1136/bmjopen-2021-052953
  6. Fletcher C et al. Marking 2 Years of New Thinking in Clinical Trials: The Estimand Journey. Therapeutic Innovation & Regulatory Science (2022) 56:637–650.
    https://doi.org/10.1007/s43441-022-00402-3
  7. ICH Topic E 9 Statistical Principles for Clinical Trials. Note for Guidance on Statistical Principles for Clinical Trials (EMA/CPMP/ICH/363/96).
    https://www.ema.europa.eu/en/ich-e9-statistical-principles-clinical-trials-scientific-guideline
  8. Unkel S et al. On estimands and the analysis of adverse events in the presence of varying follow-up times within the benefit assessment of therapies. Pharmaceutical Statistics. 2019;18:166–183. 
    https://doi.org/10.1002/pst.1915Digital Object Identifier (DOI)
  9. Ren X et al. Estimand in benefit-risk assessment. J Biopharm Stat 2023 Jul 4;33(4):452-465. 
    https://doi.org/10.1080/10543406.2023.2170396
  10. Wang X et al. Application of estimand framework in ICH E9 (R1) to safety evaluation. J Biopharm Stat. 2023 Jul 4;33(4):476-487.
    https://doi.org/10.1080/10543406.2023.2189452
  11. Ratitch B et al. Choosing Estimands in Clinical Trials: Putting the ICH E9(R1) Into Practice. Therapeutic Innovation & Regulatory Science, vol. 54, no. 2, pp. 324-341.
    https://doi.org/10.1007/s43441-019-00061-x
        Welcome to Alira Health. This site is best viewed in Chrome, Microsoft Edge, or Firefox.