Head of Data and Analytics Julia Wilkins explores the issue of evaluation in a new digital world in her latest blog.
Emerging digital health technologies (DHTs) such as apps and wearable devices offer a wealth of opportunities for improving systems and treatments in the NHS, benefitting staff, patients and the public.
At ICHP, we’ve already worked with a number of organisations in the digital healthcare arena. For example, together with partners across North West London we have piloted three digital behaviour change programmes for patients with Type 2 Diabetes.
However, until recently there hasn’t been a standard set of evidential requirements for demonstrating the value of these new services or interventions. NICE have now published their evidence standards framework for digital health technologies, welcomed by both developers and commissioners alike. The standards will help both parties understand what good levels of evidence look like for DHTs. With developers following the framework, commissioners will be able to compare the cost and value of differing services or interventions, and make consistent decisions based on quality evidence.
The framework covers the three Es – efficacy (does it work in clinical trials?), effectiveness (does it work in clinical practice?) and efficiency (does it contribute to more efficient use of resources and offer value for money?). It also looks at how well the technology works and how much it costs compared to standard NHS practice.
There are three evidence tiers based on a risk classification (the higher the cost and risk, the higher the level of evidence required), ranging from technologies that simply provide services to the health and social care system, to those which support behaviour change and self-care, provide treatment or active monitoring, make calculations that affect the treatment or care provided, or guide and give diagnoses.
The evidence requirements
The framework calls for economic impact evidence, ranging from a simple budget impact analysis up to cost-consequence and cost-utility analysis. The level of analysis required varies depending on the level of financial commitment needed to deliver the new intervention or service.
It also outlines the ways developers can gather the effectiveness evidence required – including user testing, data demonstrating improvement in symptoms, and comparative outcomes for patients receiving other or no interventions, for example.
There are also some interesting questions around context – for example, whether the intended users are from a potentially vulnerable group (e.g. children or at-risk adults), how serious the consequences would be if the technology didn’t work as required, and whether the DHT is intended to be used with clinical support.
These are all areas that fall within our scope at ICHP, as our analysts and health economists work together to develop ‘wrap around’ evaluations that look at all aspects of a new intervention. By using integrated data we can look at the whole system impact (for example a new intervention in hospital may lead to decreased GP visits after discharge), offering a helicopter view over time and looking in addition at how patients got to the point of that intervention – all key elements in the new framework.
How we deliver the evidence needed
At ICHP we have an in-depth understanding of developing and working with the type of real-world evidence that the framework demands, in addition we also have access to the type of integrated data needed to demonstrate value across a whole health and care system. This includes access to Discover, a research database covering the North West London (NWL) population of 2.4 million patients.
The data which Discover uses is drawn from NWL’s Whole Systems Integrated Care dataset (WSIC) – one of the most powerful datasets in the UK, with linked coded data from primary, secondary, community, mental health and social care.
By accessing these systems we can flag a specific cohort of de-identified patients in Discover, work with clinicians across NWL to pilot an intervention, then carry out an evaluation with an in-built comparator drawn from the wider dataset.
At ICHP we are proud in our aim to help support in lowering the barriers to gathering the right evidence to support innovation and change across the whole healthcare system.
