Has NICE made an error in their most recent HTA Methods guidance? And is there a simple fix?

A change in the methods recommended by the National Institute for Health & Care Excellence (NICE) in 2022 seems to have introduced an error in the collection of data regarding the measurement of health related quality of life. From our research, this is quite a substantial error which needs to be fixed. 

NICE make recommendations regarding the adoption of treatments in England. NICE assess the benefits of treatments in terms of the QALY (quality-adjusted life year), the single metric combining length of life and quality of life. NICE prefer that quality of life is measured using the EQ-5D completed by patients with the condition of interest [1]. However, it's not always possible to provide EQ-5D data across the time course of a chronic disease. It's even harder to provide such data in a rare or ultra-rare disease. To fill in these missing data points, modelling teams may rely on the literature or other data approaches.

Other ways to collect utility data are available, such as mapping studies and vignette studies. Vignette studies typically involve members of the public rating the severity of descriptions of a health state using a valuation task like time trade-off [2]. 

In the time trade-off method, members of the public read the vignettes and rate them against ‘Full health’ and ‘Dead’ in order to derive a utility. In rare diseases, NICE will accept data that has been collected from vignette studies. This is because they recognise the challenges of capturing data and also the fact that standard measures such as EQ-5D produce very noisy data in small samples.

NICE have changed their recommendations regarding these methods in the most recent guidance. They now recommend that the vignettes be rated using the EQ-5D, as well as — or rather than — the time trade-off [3]. So, in such an exercise, a member of the public reads the vignette, imagines what it would be like to be in such a state and then completes the EQ-5D as if they were in the state. 

Since this guidance has emerged, we have completed a number of studies (about 10) where we have asked people to rate vignettes using the EQ-5D approach and also used time trade-off. Every study shows a consistent pattern. 

The EQ-5D scores for the same health states are systematically lower than the TTO rating of the same state [4]. This study in vision loss is one example, but the findings are true in multiple disease areas [5]. The EQ-5D scores may be 0.10 or 0.15 lower than the corresponding TTO values. 

Such a difference could be an important determinant of the ultimate cost effectiveness (and therefore access for patients) for a new treatment. These two methods do not produce comparable data and are not interchangeable. We believe that this is something for NICE to look at again and consider whether EQ-5D is a suitable method for rating vignettes.