Difference in differences analyses (DiD) can turn vast amounts of data into valuable insights which help health systems plan and evaluate change. Andi Orlowski, Deputy Director of Business Intelligence, explains:
Data can provide many helpful insights for health systems but one of the things we get asked to look at most often is the impact of an intervention over a period of time.
It is crucial for health trusts and integrated care systems (ICSs) to know whether the changes they are making are having a beneficial impact on patients and one of the best ways to do this is by using something known as a difference in differences (DiD) analysis.
A DiD analysis is a statistical technique commonly used in economics to estimate the effect of a specific intervention or treatment by comparing the changes in outcomes over time between a group that is enrolled in a programme (or treatment) and a group that is not.
Real life DiD analysis
I have recently finished a large-scale DiD analysis looking at the effects of new direct oral anti-coagulants (DOACs) on patients with atrial fibrillation (AF), the subsequent impact on stroke rates, and what this could mean for the NHS in the future. Evidence from clinical trials has shown that DOACs are as effective as warfarin at stopping stroke in AF patients, but they cost four to five times as much.
The new drugs were launched around the start of the decade and this coincided with a policy change in 2012, which said patients at risk should be prescribed an oral anticoagulant (OAC) to reduce their risk of an AF related stroke. This led to many people on aspirin being switched to an OAC.
Our study was designed to look at whether, in the real world, these drugs have had the same effect as warfarin and what the budget impact has been. Essentially, what has the impact of having these more expensive drugs in the system been and are they worth it?
Using a DiD analysis we studied data from two groups of patients over two time periods, to give a picture of the situation before and after the widespread introduction of DOACs.
The study used Hospital Episode Statistics, the electronic Prescribing and Costing Tool and the Quality and Outcomes Framework and took in more than 1.4 million patients with AF between 2011 and 2013 and 1.8 million patients with AF between 2015 and 2017.
It covered approximately 8,500 GP practices, 200 hospitals and 242 clinical commissioning groups (CCGs) and looked at the differences in outcomes for patients taking DOACs compared with those taking warfarin.
What we found is that there has been a profound effect on stroke rates, which have gone down by 13%, but there has also been a huge budget impact, with the cost of prescriptions rising by 983% which is an additional £700m.
However, the overall costs of all treatment – preventative, acute and post-stroke – only rose by 135%, leading us to conclude that, in the long term, health systems will save money because the cost of treating stroke will be reduced. Whilst the cost of the drug has gone up, if you offset this against the cost of stroke treatment, the overall cost per patient has gone down (although there are now many more people being treated!).
And the really interesting bit of our findings was that they showed, contrary to the evidence from clinical trials, that DOACs are not just as effective as warfarin, they could potentially be more effective.
We suspect that this is because in clinical trials people were taking their drugs correctly, adhering to the dosing regime and lifestyle changes required by warfarin etc, whereas in real life people might not always take them like this. Combine this with the fact that DOACs are easier to take than warfarin as there are no dietary requirements, which makes them an easier option for older, more frail patients (the demographic more likely to have strokes). The fact that DOACs are more likely to be prescribed for this higher risk group could also account for some of the difference.
Further explanation needed
Further explanation of our outcomes is needed before any definitive conclusions can be drawn but we believe this piece of work better helps us understand the future treatment of AF and helps to quantify the impacts of the use of DOACs in a way that can only be done using large tranches of data. We are writing a paper now to share these results more fully and are also starting a new piece of research using linked data in the hope to show what is causing these effects.
If you would like to know more about how DiD analyses could help your organisation plan and monitor healthcare change contact Andi Orlowski, Deputy Director of Business Intelligence
