Fenfluramine for Treating Dravet Syndrome: An Evidence Review Group Perspective of a NICE Single Technology Appraisal
grid.412966.e0000 0004 0480 1382Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre+ (MUMC+), P. Debyelaan 25, PO Box 5800, 6202 AZ Maastricht, The Netherlands
grid.416017.50000 0001 0835 8259Center of Economic Evaluation and Machine Learning, Trimbos Institute (Netherlands Institute of Mental Health and Addiction), Utrecht, The Netherlands
grid.450936.d0000 0004 0450 3334Kleijnen Systematic Reviews Ltd, York, UK
grid.5012.60000 0001 0481 6099Care and Public Health Research Institute (CAPHRI), , Maastricht University, Maastricht, The Netherlands
Adelphi Values, Adelphi Mill, Bollington, UK
Abstract
Fenfluramine, tradename Fintepla®, was appraised within the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process as Technology Appraisal 808. Within the STA process, the company (Zogenix International) provided NICE with a written submission and a mathematical health economic model, summarising the company’s estimates of the clinical effectiveness and cost-effectiveness of fenfluramine for patients with Dravet syndrome (DS). This company submission (CS) was reviewed by an evidence review group (ERG) independent of NICE. The ERG, Kleijnen Systematic Reviews in collaboration with Maastricht University Medical Centre, produced an ERG report. This paper presents a summary of the ERG report and the development of the NICE guidance. The CS included a systematic review of the evidence for fenfluramine. From this review the company identified and presented evidence from two randomised trials (Study 1 and Study 1504), an open-label extension study (Study 1503) and ‘real world evidence’ from a prospective and retrospective study. Both randomised trials were conducted in patients up to 18 years of age with DS, whose seizures were incompletely controlled with previous anti-epileptic drugs. A Bayesian network meta-analysis was performed to compare fenfluramine with cannabidiol plus clobazam. There was no evidence of a difference between any doses of fenfluramine and cannabidiol in the mean convulsive seizure frequency (CSF) rate during treatment. However, fenfluramine increased the number of patients achieving ≥ 50% reduction in CSF frequency from baseline compared to cannabidiol. The company used an individual-patient state-transition model (R version 3.5.2) to model cost-effectiveness of fenfluramine. The CSF and convulsive seizure-free days were estimated using patient-level data from the placebo arm of the fenfluramine registration studies. Subsequently, a treatment effect of either fenfluramine or cannabidiol was applied. Utility values for the economic model were obtained by mapping Pediatric Quality of Life Inventory data from the registration studies to EuroQol-5D-3L Youth (EQ-5D-Y-3L). The company included caregiver utilities in their base-case, as the severe needs of patients with DS have a major impact on parents and caregivers. There were several key issues. First, the company included caregiver utilities in the model in a way that when patients in the economic model died, the corresponding caregiver utility was also set to zero. Second, the model was built in R statistical software, resulting in transparency issues. Third, the company assumed the same percentage reduction for convulsive seizure days as was estimated for CSF. Fourth, during the final appraisal committee meeting, influential changes were made to the model that were not in line with the ERG’s preferences (but were accepted by the appraisal committee). The company’s revised and final incremental cost effectiveness ratio (ICER) in line with committee preferences resulted in fenfluramine dominating cannabidiol. Fenfluramine was recommended as an add-on to other antiepileptic medicines for treating seizures associated with DS in people aged 2 years and older in the National Health Service (NHS).
Key Points for Decision Makers
| The largest uncertainties in this appraisal were caused by inclusion of caregiver disutilities and estimating convulsive seizure days based on convulsive seizure frequency, and due to transparency issues in the implementation of the model in software package R. |
| During the final appraisal committee meeting, influential changes were made to the model that were not in line with the evidence review group’s preferences (but that were accepted by the appraisal committee). Amongst others, these changes included an increase in the dosage of cannabidiol and a decrease in the dosage of fenfluramine based on real-world evidence. |
| The company’s revised and final ICER in line with appraisal committee preferences resulted in fenfluramine dominating cannabidiol. |
Introduction
Fenfluramine, tradename Fintepla®, was appraised within the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process as Technology Appraisal (TA) 808. Health technologies must be shown to be clinically effective and to represent a cost-effective use of National Health Service (NHS) resources to be recommended by NICE. Within the STA process, the company (Zogenix International) provided NICE with a written submission and a mathematical health economic model, summarising the company’s estimates of the clinical effectiveness and cost-effectiveness of fenfluramine for patients with Dravet syndrome (DS). This company submission (CS) was reviewed by an evidence review group (ERG) independent of NICE. The ERG, Kleijnen Systematic Reviews (KSR) in collaboration with Maastricht University Medical Centre, produced an ERG report [1]. After consideration of the evidence submitted by the company, comments by other stakeholders and the ERG report, the independent appraisal committee (AC) issued guidance on whether to recommend the technology by means of the Final Appraisal Determination (FAD) [2], to which an appeal can be made. This paper presents a summary of the ERG report and the development of the NICE guidance. Furthermore, it highlights important methodological issues that were identified that may help in future decision making. Full details of all appraisal documents (including the scope, CS, ERG report, consultee submissions, Appraisal Consultation Document (ACD) and FAD) are on the NICE website [3].
The Decision Problem
The population defined in the scope for this appraisal was “people with Dravet syndrome (DS) whose seizures are inadequately controlled by established clinical management without fenfluramine, which may include combinations of sodium valproate, topiramate, clobazam, stiripentol, levetiracetam, ketogenic diet, vagus nerve stimulation, or cannabidiol with clobazam” [4, 5]. The company addressed this population, i.e. at second-line, in the CS [5].
The intervention (fenfluramine hydrochloride (Fintepla®) in addition to current clinical management) was consistent with the scope issued by NICE [4]. Fintepla® is indicated for the treatment of seizures associated with DS as add-on therapy to other antiepileptic medicines in people aged at least 2 years [6, 7]. The starting dose is 0.1 mg/kg twice daily and it can be used with or without stiripentol. In contrast with cannabidiol, fenfluramine is licensed for use both with and without concomitant clobazam [7]. Additional details are:
- Patients not taking stiripentol:
- After 7 days, for patients who are tolerating fenfluramine, and require a further reduction of seizures, the dose can be increased to 0.2 mg/kg twice daily.
- After an additional 7 days, for patients who are tolerating fenfluramine, and require a further reduction of seizures, the dose can be increased to a maximum of 0.35 mg/kg twice daily (0.7 mg/kg/day).
- Do not exceed a total dose of 13 mg (6 mL) twice daily.
- Patients taking stiripentol:
- After 7 days, for patients who are tolerating Fintepla, and require a further reduction of seizures, the dose can be increased to 0.2 mg/kg twice daily (0.4 mg/kg/day).
- Do not exceed a total dose of 8.5 mg (4 mL) twice daily.
- When discontinuing fenfluramine, the dose should be decreased gradually.
The final NICE scope suggested that the comparators are combinations of sodium valproate, topiramate, clobazam, stiripentol, levetiracetam, ketogenic diet, vagus nerve stimulation and cannabidiol with clobazam [4]. The company considered that, in the absence of sufficient stiripentol data for a robust comparison, cannabidiol (with clobazam) was the most appropriate comparator [5]. However, the company also provided comparative analyses of fenfluramine as an add-on therapy to background antiepileptic drugs (AEDs) that included or excluded stiripentol, and in patients not taking concomitant clobazam [5].
The company stated that few patients might receive fenfluramine as a first-line add-on therapy in patients where clobazam or a clobazam-based regimen is undesired. They added that because clobazam is well established and clinicians managing people with DS are well experienced in the use of clobazam, they anticipated that clobazam would be considered as a first-line add-on in the vast majority of patients. Fenfluramine was therefore considered at second-line [8].
The primary outcome of the randomised trials was reduction in convulsive seizures, which was also used in the economic model, although seizure-free days was also used in the model [5].
Evidence Review Group (ERG) Critique of Decision Problem
Around 85% of people with DS can survive into adulthood [9]. However, although the decision problem did not specify any age restriction and the expected licenced indication for Fintepla® is for patients aged at least 2 years, neither of the key trials in the CS included adult patients. Given the lack of randomised evidence for adults, the company was asked to confirm that the CS only applies to children. The company responded that they believed that data from the open-label Study 1503, real-world evidence studies program, and the ongoing European expanded access demonstrated that both children and adults “experience significant and often similarly profound reductions in convulsive seizure frequency with fenfluramine treatment” [8]. However, there are few adults in these studies and none of them were randomised. Nevertheless, clinical experts consulted by the ERG agreed with the company.
The ERG considered that including caregiver quality of life (QoL) was appropriate [8]. It is also consistent with the previous NICE TA614 assessment of cannabidiol [10]. Including additional outcomes, for example, convulsive seizure-free days, appears to be appropriate. However, the ERG had concerns about the implementation in the economic model (see below). The ERG also noted that the link between convulsive seizure reduction and associated mortality reduction including Sudden Unexpected Death in Epilepsy (SUDEP) cannot be determined from the two trials given a treatment maintenance period of just 12 weeks. In addition, in TA614 the committee argued that “there is insufficient evidence to prove that cannabidiol prolongs life” [10].
Independent ERG Review
The ERG reviewed the clinical effectiveness and cost-effectiveness evidence. The ERG and NICE also sought clarification on issues in the CS, for which the company provided additional information [8, 11]. The ERG produced an ERG base case to assess the impact of alternative assumptions and parameter values by modifying the economic model submitted by the company. Sections 3.1–3.6 summarise the evidence presented in the CS and the ERG critique.
Clinical Effectiveness Evidence Submitted by the Company
The CS included a systematic review, which identified two randomised trials (Study 1 [12, 13] and Study 1504 [14, 15]), an open-label extension study (Study 1503 [16]) and ‘real world evidence’ from a prospective and retrospective study [17–19]. Both randomised trials were conducted in children with DS whose seizures were incompletely controlled with previous AEDs [12–15]. In Study 1504 all patients received stiripentol as concomitant treatment to fenfluramine whereas in Study 1 stiripentol was not permitted [12–15]. The results of these trials were that participants receiving fenfluramine had greater reductions in convulsive seizure frequency (CSF) per 28 days compared to placebo. Patients were also more likely to have 25%, 50% and 75% reductions in convulsive seizures. Furthermore, participants in fenfluramine groups had longer convulsive seizure-free intervals.
The company stated that “due to substantial differences in the assessment of convulsive seizure reduction endpoints in the stiripentol trials, and also the unclear risk of bias that limits the quality of the stiripentol trial evidence, we determined it is not feasible to conduct an ITC comparing add-on fenfluramine vs add-on stiripentol” [8].
A Bayesian NMA was performed to compare fenfluramine with cannabidiol plus clobazam as first-line add-on therapy. This used data from Study 1 and Study 1504 and two cannabidiol trials (GWPCARE1 part B and GWPCARE2) [12, 14, 20, 21]. There was no evidence of a difference between any doses of fenfluramine and cannabidiol in the mean CSF rate. However, fenfluramine increased the number of patients achieving ≥ 50% reduction in CSF frequency from baseline compared to cannabidiol.
Critique of Clinical Effectiveness Evidence and Interpretation
The ERG noted that, as per the company’s dosage instructions, fenfluramine is given at a lower dosage when combined with stiripentol and the randomised trials reflect this, which is why the ERG agreed with the company to not include both trials in the same NMA [12–15]. The ERG was concerned about the clinical heterogeneity of studies in the NMA regarding concomitant AEDs as the use of clobazam and stiripentol varied between studies.
Cost-Effectiveness Evidence Submitted by the Company
The company conducted a systematic literature review to identify and evaluate existing cost-effectiveness analyses/economic evaluations of fenfluramine or other pharmacological therapies used as add-on therapies to standard-of-care (SoC) anti-epileptic drugs in DS. The CS provided an overview of the included cost effectiveness, utility and resource use and costs studies. None of the identified studies evaluated the cost effectiveness of fenfluramine in patients with DS, and therefore were not directly generalisable to the NICE decision problem.
The company used an individual-patient state-transition model (R version 3.5.2), consisting of three health states: (1) alive, on treatment, (2) alive, treatment discontinued, and (3) death (Fig. 1). The individual patient approach was justified by the company to account for patient heterogeneity as well as by referring to the committee discussion of TA614 [10], where it was stated that “other approaches to modelling, such as discrete event simulation, may have been more appropriate to capture the benefits of different numbers of seizure-free days”. Due to insufficient data and to simplify the modelling approach, patients did not switch to subsequent treatments after treatment discontinuation, but instead returned to their baseline SoC. The company assumed based on the DISCUSS study that 58% of the population would receive concomitant stiripentol. The company calculated the cost effectiveness of fenfluramine separately for these two subpopulations and subsequently used these proportions to calculate a weighted average representing the cost effectiveness of the ‘merged’ population. This resulted in the comparison of fenfluramine+SoC to cannabidiol+clobazam+SoC in the base-case analyses.
The main source of evidence for fenfluramine (Study 1 and Study 1504) focused on DS patients aged ≥ 2 to ≤ 18 years, with convulsive seizures not completely controlled by current AEDs (≥ 4 convulsive seizures per 4-week period for 12 weeks prior to screening).
As per its licensed dosing, fenfluramine was modelled with a dose of 0.7 mg/kg/day (maximum of 26 mg/day) when used without stiripentol and 0.4 mg/kg/day (maximum of 17 mg/day) when used with stiripentol. Although the European Medicines Agency (EMA) has not proposed a ‘stopping rule’ for fenfluramine based on efficacy, treatment was stopped if there was insufficient improvement, i.e. any reduction in the number of convulsive seizures < 30% after 6 months after the start of the treatment. The comparator used in the model was cannabidiol with a dose of 12 mg/kg/day. The company argued that this was the only existing add-on therapy that has been appraised and accepted as clinically and cost effective in combination with clobazam. Ketogenic diet and vagus nerve stimulation (also mentioned in the final NICE scope) were not included in the model. The percentage of concomitant AEDs was retrieved from the fenfluramine registration studies, except for stiripentol and clobazam. The effects observed in Study 1504 (94% receiving stiripentol) and Study 1 (no stiripentol) were applied to the frequencies observed in the DISCUSS study. Concomitant clobazam use was applied to cannabidiol as per its licensed indication. The company excluded concomitant AEDs (given as add-on therapy to established clinical management) not listed in the final NICE scope from the analysis, because these are used in a minority of patients and used equally in patients who would receive fenfluramine and cannabidiol.
The analysis took an NHS and Personal Social Services perspective. Discount rates of 3.5% were applied to both costs and benefits. The model cycle length was four weeks with a lifetime time horizon.
The (relative) treatment effect was estimated based on CSF assuming that a reduction in convulsive seizure frequency leads to more convulsive seizure-free days, reduces the risk of SE, reduces the risk of mortality and improves patients’ and caregivers’ QoL.
The CSF and convulsive seizure-free days were estimated using patient-level data from the placebo arm of the fenfluramine registration studies. Subsequently, a treatment effect of either fenfluramine or cannabidiol was applied. The number of convulsive seizures was simulated in each cycle by bootstrapping the placebo arm of the fenfluramine registration studies (to reflect patient heterogeneity over time). The company assumed that for adults, the frequency of convulsive seizures was halved, and the number of convulsive seizure-free days was doubled. An indirect treatment comparison (ITC) was conducted using the fenfluramine registration studies and subgroup analyses of the cannabidiol registration studies. The considered outcome was the percentage change in CSF from baseline compared to placebo. The treatment effect derived from the ITC was then applied to the percent change in convulsive seizures per 28-day cycle from baseline for each individual. The company assumed the same percentage reduction for convulsive seizure days as for CSF. The estimated reductions were applied to the seizure frequencies of patients in the placebo arm (in both strategies). The company assumed no treatment waning effect for both fenfluramine and cannabidiol. The company did not incorporate non-convulsive seizures in the model due to measuring difficulties and consistency with TA614 [10]. The company assumed that status epilepticus (SE) was driven by convulsive seizures through estimating SE events as a proportion of convulsive seizures (0.17%) based on the fenfluramine registration studies. Total mortality included epilepsy-related SUDEP, SE and accidental mortality, and non-epilepsy-related mortality. Mortality estimates were retrieved from the published literature. Three types of treatment discontinuation (lack of efficacy, other discontinuation, and ongoing discontinuation) as well as a stopping rule (at 6 months after treatment initiation) were applied. After discontinuation, patients reverted to their baseline CSF.
Although treatment-emergent AEs were considered for inclusion into the economic model, the company judged that “the incidence of serious TEAEs was low and similar across the fenfluramine and placebo arms”. Therefore, although Study 1 reported 12.5% of patients with AEs leading to discontinuation, the company pragmatically assumed that the same would be true for cannabidiol (with clobazam), and AEs were excluded from the model.
Utility values for the economic model were obtained by mapping PedsQL data from the registration studies to EQ-5D-Y-3L using the Khan et al. algorithm [22]. The PedsQL and EQ-5D-Y-3L are QoL instruments developed to measure QoL in children. A linear mixed-effects regression model was used to predict utility scores in each model cycle. The company included caregiver utilities in their base case, as the severe needs of patients with DS have a major impact on parents and caregivers. EQ-5D-5L data were collected from caregivers in the registration studies and mapped onto EQ-5D-3L using the UK value set by van Hout et al. [23]. A linear panel regression model with fixed effects was used to predict the caregiver utility score in each model cycle. The economic model considered utilities of 1.8 caregivers per patient as accepted in TA614, which were removed when the patient died. Patient and caregiver utilities ranged from respectively 0.35 to 0.72 and 0.36 to 0.74.
Cost categories included in the model were treatment acquisition costs, monitoring costs and health state costs (ongoing and emergency resource use costs). Unit prices were based on the NHS reference prices, British National Formulary (BNF), and Personal Social Services Research Unit. Fenfluramine and other AED doses were weight dependent and thus changed over the patient’s lifetime. Average daily doses and costs were calculated for each cycle using the recommended BNF dose. Fenfluramine was offered with a confidential discount. Total concomitant SoC AEDs costs were calculated by weighting the daily AED cost by the percentage of patients receiving each concomitant AED in the fenfluramine trials. The assumed doses for SoC AEDs and rescue medications were based on the individual product doses from the BNF (March 2020), the draft Summary of Product Characteristics for fenfluramine, and the preferred AC dosing assumption for cannabidiol in NICE TA614 [10]. Drug costs were taken from the drug tariff price as reported in the March BNF, the Prescription Cost Analysis by Pharmacy and Appliance Contractors in England (November 2019), and expert opinion. Monitoring costs included echocardiograms, and health state costs included cost for the routine management of DS patients and for emergency care in case of severe seizures. Resource use estimates were elicited from physicians and nurses involved in the management and treatment of paediatric and adult patients with DS as reported in the UK Pathway research study. For the frequency of ongoing resource use, individuals in the model were categorized by age group and the average monthly number of convulsive seizures (‘low’, ‘medium’ and ‘high’ frequency of seizures). Emergency resource use was calculated based on the occurrence of SE events, which was calculated as a proportion of total seizures.
In the company’s base-case analysis, total life-years (LYs) and quality-adjusted life-years (QALYs) gained were larger for fenfluramine + SoC than for cannabidiol + clobazam + SoC. Incremental QALYs (0.93) were mainly driven by QALY gains resulting from caregiver utilities. Total costs were also higher for fenfluramine + SoC than for cannabidiol + clobazam + SoC. Incremental costs (£29,446) mainly resulted from higher treatment costs. The deterministic incremental cost-effectiveness ratio (ICER) amounted to £31,773 per QALY gained. The ICERs of the two separate models (based on Study 1 and Study 1504) vary greatly (£38,874 per QALY gained for Study 1 and £10,770 per QALY gained for Study 1504). In response to technical engagement and appraisal committee meetings, the company conducted various model implementations: using the merged population including people taking and not taking stiripentol, incorporating caregivers’ disutilities, and using real-world dosing evidence for fenfluramine and cannabidiol. The company’s revised and final ICER in line with AC preferences resulted in fenfluramine dominating cannabidiol (please note: this included a 50% stopping rule, which was in the end not preferred by the AC. The ICER including a 30% stopping rule, which was preferred by the AC, was not published and only used in the confidential analyses).
Critique of Cost-Effectiveness Evidence and Interpretation
Patients reverted to baseline seizure frequency rather than the placebo ‘on-treatment’ seizure frequency after treatment discontinuation. The ERG was concerned that removing the presumed placebo effect for discontinued patients while not removing it for patients on treatment would likely result in an overestimated treatment effect for being on treatment versus patients that discontinued treatment, as placebo effects may also be present in patients who are still on treatment. The ERG preferred that upon treatment discontinuation, patients reverted to the placebo seizure frequency as observed during the maintenance period of the trial, but this scenario analysis was not provided by the company.
Fenfluramine was licensed for use both with and without concomitant clobazam (unlike cannabidiol, which was only licensed with clobazam). However, the company considered cannabidiol as the only comparator, implying that the cost-effectiveness analyses are restricted to people receiving clobazam. When the focus is on the full marketing authorisation, comparators should not be restricted to cannabidiol as cannabidiol is not a recommended comparator for the full marketing authorisation. Moreover, given that cannabidiol is only relevant for a subgroup of the population, the cost effectiveness of the populations treated with and without concomitant clobazam should be considered separately. The company provided a scenario analysis assuming all patients in the fenfluramine arm receive clobazam (by increasing the modelled costs of clobazam), and thus assuming that the relative effectiveness is unaffected by concomitant clobazam. This scenario was implemented in the ERG analyses to reflect the concomitant clobazam population.
Based on TA614, a stopping rule was implemented for all patients whose seizure frequency did not reduce by at least 30% at 6 months, which had not been proposed by the EMA and was not found in the NICE scope [10]. Implementing the stopping rule in the cannabidiol arm only (i.e., the stopping rule was based on NICE TA614 for cannabidiol and had not been proposed by the EMA or the scope provided by NICE) increased the ICER from £31,773 per QALY to £63,268 per QALY. Due to transparency issues (it was unclear how the company implemented this analysis in the economic model) and time constraints, the ERG was unable to explore this issue in a scenario analysis.
Due to a lack of data on seizure-free days for cannabidiol, the company assumed the same percentage reduction as was estimated for CSF. Although there is an association between these two outcomes, the plausibility of assuming proportionality is unclear.
The ITC comparing the fenfluramine and cannabidiol registration studies showed that the CSF percentage reduction from baseline versus placebo was larger for fenfluramine than for cannabidiol, and the proportionality assumption therefore resulted in a larger reduction in convulsive seizure days for fenfluramine than for cannabidiol. Therefore, the ERG used the reduction in CSF × 0.4 to estimate the reduction in convulsive seizure days. This assumption is likely (still) favouring fenfluramine, and given that convulsive seizure-free days was the main QALY driver, the current assumptions may have resulted in an overly optimistic utility benefit for fenfluramine.
Mortality was assumed to be linked to CSF, which required multiple major assumptions that led to seemingly implausible estimates of the relative risk of SUDEP by annual seizure frequency. Given the strong assumptions the company was required to make, the significant challenges in providing empirical evidence to link mortality to CSF as well as the committee’s preferences for TA614, the ERG preferred to remove this link [10].
Caregiver utilities of 1.8 caregivers per patient were included irrespective of seizure frequency in the company’s base-case using a regression function based on caregivers of children and adolescents in the registration studies. This was not in line with TA614, in which caregiver utilities were only included for the two highest seizure frequency health states (> 8 to ≤ 25 and > 25 convulsive seizures a month). Furthermore, the ERG questioned whether the regression function based on caregivers of children and adolescents is also applicable to caregivers of adults, and considers the assumption of 1.8 caregiver per patient over a whole lifespan to be high. Moreover, in the two clinical trials, there were no differences between groups in change from baseline in caregiver QoL. The company provided a scenario analysis assuming zero caregiver utilities for individuals who had > 20 seizure-free days a month. This approach was not conservative for the health states with higher numbers of seizure-free days as these are performing worse because the higher caregiver utilities are not incorporated in the model. The ERG therefore instead assumed a utility of 0.74 (highest estimated utility by the company) for individuals with > 20 seizure-free days a month.
Caregiver utilities were set to zero when a patient died, which overestimated the impact of mortality given that the caregiver did not die together with the patient. This issue could have been solved by applying a caregiver disutility to the two worst health states in the model until a patient died rather than adding a caregiver utility. Although there is no clear guidance regarding how to best incorporate caregiver utilities, the ERG considered this approach to be more appropriate than the applied approach in the current STA and explored its impact in a scenario analysis.
Due to a lack of external data on fenfluramine discontinuation rates and mortality, the economic model lacked external validation on mortality and discontinuation assumptions. Furthermore, although the company referred to TA614 for several methodological assumptions, the CS lacked cross-validation to that appraisal on estimated outcomes of both models [10]. There were large discrepancies in total costs and QALY gains between TA614 and the current appraisal, which both resulted in a substantially lower ICER for cannabidiol compared to SoC [10]. The exact origin of these differences was unclear, and it was uncertain that the costs and QALY gains associated with cannabidiol were in line with TA614 [10]. Finally, there were several issues in the model that impacted usability and possibly threatened the internal validity and transparency of the model. Importantly, the company submitted a model that did not include the possibility to evaluate different scenarios within the same model. This hindered the validation of additional model changes (in later stages of the STA) and scenarios submitted by the company.
Additional Work Undertaken by the ERG
Based on all considerations highlighted in the ERG critique, the ERG defined a new base-case including various adjustments to the company’s base-case. This included the correction of errors in the bootstrapped patient data with convulsive seizure days and CSF, the SE mortality probability, the discontinuation probabilities, and the post-discontinuation CSF and convulsive seizure-free days. In addition, the ERG removed the CBD effect and cost from the comparator arm in the ERG base-case to assess the effect of only SoC as a comparator. As matters of judgement, the ERG presented their base-case for three subpopulations (i.e., no co-administered clobazam or stiripentol, co-administered clobazam without stiripentol, and co-administered clobazam with stiripentol), removed the link between convulsive seizures and mortality, assumed no changes in seizure frequency when patients aged, assumed a caregiver utility of 0.74 for > 20 seizure-free days a month, and used CSF × 0.4 to estimate the reduction in convulsive seizure days. The individual ERG adjustments had a large impact on the ICER, ranging from £19,863 per QALY gained to £162,886 per QALY gained in the merged population. The ERG base-case ICER for the merged population was £83,426 per QALY gained compared to cannabidiol, and £90,095 per QALY gained when comparing fenfluramine to SoC.
Model Changes After First and Second Appraisal Committee Meeting (ACM)
Following the ACM discussions on 4 March 2021 and the resulting preliminary negative recommendation for the use of fenfluramine in the treatment of seizures associated with DS, the company submitted additional information in May 2021. As a result, a second ACM was held on 20 January 2022, after which the company submitted additional information in March 2022. Lastly, a closed discussion ACM was held in April 2022.
In the revised model, the company amended the proportionality between change in CSF and change in seizure days by different proportional reduction in CSF. Moreover, upon discontinuing their index intervention, patients received a seizure profile equivalent to their ‘placebo effect’, rather than reverting patients back to their baseline seizure profile as previously implemented. In line with ERG preferences, the company assumed no change in seizures at adulthood.
Additional changes were made after the second ACM. The company increased the fenfluramine PAS discount and implemented additional changes to the model. Not all these changes had been requested by the AC or ERG, and included: (1) equalisation of short-term discontinuations for fenfluramine and cannabidiol (requested by ERG); (2) updating of long-term discontinuations for fenfluramine and cannabidiol; (3) increasing cannabidiol dose assuming real-world dosing: 15 mg/kg/day; (4) decreasing fenfluramine dose assuming real-world dosing: 0.32 mg/kg/day (with STP); 0.44 mg/kg/day (without STP); (5) fenfluramine: 50% stopping rule at 6 months; and (6) caregiver QoL applied as disutilities.
Conclusions of the ERG Report and Appraisal Consultation Phase
Both randomised trials were conducted in patients aged up to 18 years with DS, whose seizures were incompletely controlled with previous AEDs. Therefore, adults with DS were not fully represented in the CS. These trials also only had a 12-week treatment maintenance period, thus precluding an analysis of the effect on SUDEP and other deaths. As in the trials, fenfluramine is given at a lower dosage when combined with stiripentol, which is why the ERG agreed with the company to not include both trials in the same NMA.
The ERG base-case ICERs for the merged population were £83,426 and £90,095 per QALY gained compared to cannabidiol and SoC, respectively. Results compared to cannabidiol for the three subpopulations varied greatly, with ERG base-case ICERs ranging from £77,440 to £121,216 per QALY gained. These ICERs and the uncertainty related to the abovementioned issues resulted in substantial uncertainty about fenfluramine being cost-effective for patients with DS.
Key Methodological Issues
The inclusion of caregivers’ QALYs: the company included caregiver utilities in the total QALY calculation for both treatment arms in the model. However, the caregiver utility was set to zero when a patient in the economic model died. This overestimated the impact of mortality given that the caregiver did not die together with the patient and its assumed zero utility value was therefore a (relatively large) underestimation of reality. After ACM2, the company adopted an approach in which caregiver QoL was implemented by using disutilities by establishing a population norm utility value from which the trial-based utilities data from caregivers was subtracted. The company extracted a population norm utility of 0.91 for an adult aged 41 years. However, on average, patients in the fenfluramine arm live longer than patients in the CBD arm, and therefore keep the caregiver disutilities for a longer period of time. In conclusion, in the original approach, there was an additional benefit of living longer (i.e., removing caregiver utilities when a patient died) but in the new approach there was a ‘penalty’ for living longer (i.e., decreased caregiver QALYs due to longer caring time for the patient). This highlights a clear need for guidance on the inclusion of caregiver utilities in health economic models and NICE submissions in general.
The use of R as an alternative to build a health economic model: The model was built in statistical software package R. However, the way it was built lacked transparency and was very cumbersome. For example, a new model was built for every scenario or sensitivity analysis, resulting in more than 30 models for the ERG to check. In addition, the R-code of all corresponding scenario analyses lacked important annotations and contained a lot of redundant code. The use of R to code health economic models clearly has distinctive advantages over Microsoft Excel (i.e., currently the most frequently used software within NICE submissions), but it is important to emphasise that the level of transparency and details that are required in R-built models should not be underestimated. Furthermore, the use of different model versions for all separate analyses rather than one adaptable model file hinders the validation of individual adjustments made by the company. Although the ERG constructed a model including ‘model switches’ for its base-case analyses, the company neglected the implementation of switches by the ERG in later stages of the appraisal process.
Extrapolation of treatment effectiveness: The (relative) treatment effect was estimated based on CSF assuming that a reduction in CSF leads to more convulsive seizure-free days. The company assumed the same percentage reduction for convulsive seizure days as was estimated for CSF. Although it is evident that there is an association between these two outcomes, the plausibility of assuming proportionality is unclear. Furthermore, as also highlighted in the ACD, long-term treatment effect of fenfluramine while on treatment was uncertain as there are no such data beyond 3 years.
Changes during the last ACM not in line with ERG references: During the final ACM, changes were made to the model that were not in line with the ERG’s preferences (but that were accepted by the AC). As described above, these changes included an increase in the dosage of cannabidiol, a decrease in the dosage of fenfluramine (i.e., not in line with clinical trial evidence and a shift towards caregiver disutilities instead of caregiver utilities (although the ERG highlighted the lack of methodological guidance). The company provided additional references to justify the increased cannabidiol dose. The presented evidence reported average cannabidiol doses ranging from 13.3 mg/kg/day (no further details, Desai et al. [25]) to 18 mg/kg/day (median, D’Onforio et al. [24]) to 22 mg/kg/day (mean modal, 22 mg/kg/day). It should be noted that the referred studies were based on few participants and/or not conducted (solely) in the UK. For example, the studies of Scheffer et al. [26, 27] were based on the open-label extension study of the GWPCARE studies in which investigators could increase the cannabidiol dose to a maximum of 30 mg/kg/day if they considered it may be of benefit. It was therefore unclear to the ERG to what extent the presented results are applicable to UK clinical practice. The ERG also doubted the lower doses of fenfluramine. While lower doses of fenfluramine were supported by some of the presented evidence, it should be noted that the assumed dosages in the final model were not in line with the dosages used in the pivotal trials. These changes had a substantial impact on the ICER and disagreements remained between the AC and the ERG on influential model changes implemented by the company.
National Institute for Health and Care Excellence Guidance
Consideration of Clinical Effectiveness
The AC concluded that fenfluramine is an effective treatment for DS in the short term compared with placebo and that it might be superior to cannabidiol plus clobazam, but noted that there was high uncertainty given trial heterogeneity. During the ACM, clinical experts stated that a 30% reduction in seizures at 6 months is the minimum they would expect from a new treatment, and that it was unclear whether a 50% stopping rule at 6 months was a better threshold. Hence, the AC concluded that the stopping rule of at least 30% reduction in seizure frequency at 6 months was the most appropriate.
Consideration of Cost-Effectiveness
Several validity issues related to the model were raised at the first ACM. These were resolved during consultation, and it was concluded during the last ACM that the company’s model structure was appropriate for decision making, and that the results were valid. The AC highlighted that, because of the design of the model, the company provided the ERG with separate model files for scenario analyses, which resulted in longer time to validate the scenarios.
The AC noted that the cost-effectiveness estimates were substantially higher when stiripentol use was included. In the end, the committee considered that grouping the population based on stiripentol use may be artificial and not feasible for clinical practice, and concluded that the merged population is appropriate for decision making.
The AC concluded that modelling based on convulsive seizure-free days may be reasonable, but there are uncertainties in the relationship between CSF and seizure days. In response to the consultation, the company used a regression analysis to estimate the proportionality between the percentage change in CSF and the percentage change in convulsive seizure days.
In its base case, the company assumed that mortality is linked to CSF. During the first ACM, the AC requested scenario analyses testing different strengths of relationship between CSF and SUDEP, including analyses in which fenfluramine did not prolong life. The AC concluded that the strength of the relationship between CSF and mortality is not clear.
The company argued that inclusion of non-convulsive seizures likely would have improved fenfluramine’s cost effectiveness compared with standard-care drugs. However, the AC concluded that the impact of excluding non-convulsive seizures from the model is unclear.
During the last ACM, the company presented real-world evidence from studies that showed the average dosing of fenfluramine and cannabidiol. The AC concluded that using this real-world dosing evidence for fenfluramine and cannabidiol was appropriate for this appraisal.
The committee concluded that fenfluramine was associated with manageable AEs, although there was uncertainty in its modelling.
During the last ACM, the AC concluded that incorporating caregivers’ QoL was appropriate, but only by applying caregiver disutilities (large impact on the ICER). Although the committee noted that both approaches had limitations, it concluded that the disutility approach had more face validity because it did not result in the unexpected assumption that caregivers would die at the same time as the patient.
In this TA, the NICE AC recommended fenfluramine as an add-on to other antiepileptic medicines for treating seizures associated with DS in people aged 2 years and older in the NHS. This decision was based on DS having a substantial effect on the QoL of patients, and their families/caregivers and the conclusion that fenfluramine is clinically effective in reducing the number of convulsive seizures and that it may be more effective than cannabidiol plus clobazam. There were some uncertainties around the assumptions in the model. However, the AC considered the most plausible ICER for fenfluramine compared with cannabidiol plus clobazam was likely to be within the range normally considered an effective use of NHS resources.
Acknowledgements
The authors would like to acknowledge the clinical advice and expert opinion provided by: Prof. Dr. Marian Majoie, Department of Neurology, Academic Centre of Epileptology Kempenhaeghe Maastricht UMC+, The Netherlands; School for Mental Health and Neuroscience, Maastricht University Medical Centre, Maastricht, The Netherlands; School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands. Dr. Judith Verhoeven, Academic Center for Epileptology Kempenhaeghe/Maastricht University Medical Center, Heeze, The Netherlands.
Declarations
Funding
This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme. See the HTA programme website for further project information (http://www.hta.ac.uk). This summary of the ERG report was compiled after NICE issued the FAD. The views and opinions expressed herein are those of the authors and do not necessarily reflect those of NICE or the Department of Health.
Conflicts of interest
No conflicts of interest to declare.
Ethics approval
Not applicable.
Informed consent
Not applicable.
Data availability
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Code availability
Not applicable.