In our last newsletter, we outlined the approaches taken by the various stats teams in the DfE to handling their data publications in light of recent changes to population estimates from the ONS.
The attached data table (see link below to download the full Microsoft Excel file) illustrates the impact of these changes to individual LAs; we hope it's a useful reference point when reviewing historical trend data which may be misleading at present.
There are two key risks to the current situation, with different data releases using different reference populations. The first is that you see a "hockey stick" curve to a historical rate trend line, indicating a rate change, where in fact the trend has been consistent but only the reference population has changed, altering the calculated rate where the actual activity in your area hasn't changed. This can require some explaining with senior service leads to ensure they understand what they're looking at.
The second is perhaps less likely but more difficult to spot: there's a similar risk that some significant change really has happened in your area, but the altered reference population has partially or wholly masked - or even reversed - the appearance of this change in your dataset, so that your rate trend line appears consistent.
There are several councils whose population estimates differ by as much as 10%, which is more than enough of a discrepancy to generate some very misleading trend data in key rates.
It's worth making clear here that we're not talking about actual population changes, which you do of course want to take into account in your analysis. We're talking about changes in the maths used to estimate the population in each local area, and how different DfE publications are currently using different estimate methods for different years' data.
We hope DfE will soon republish their historical data publications with corrections to accommodate the ONS changes to historical population estimates. This will bring each year's trend data back into alignment and make year-on-year comparison more accurate. When it happens, we'll update the D2I tools accordingly.
So, to briefly summarise the current state of play with regard to our tools:
The ChAT and Benchmarking Tool now contain the revised all ages population data, but this is used for very little, because most rates are calculated using the DfE-selected population figures. In order to maintain parity between our tools and DfE published figures, we don’t want to forcibly overwrite figures DfE has published in our tools.
The RIIA Benchmarking Tool is now using the latest available mid-year population estimates for any rates it calculates, to provide the most accurate possible in-year data.
Where rates are published by DfE, we have left those rates as published by DfE so as to avoid confusion when comparing our tools with DfE original sources. We appreciate this isn’t ideal, but we think it’s the best compromise.
For CLA data, the DfE CLA statistics team intend to revise their publication in light of the revised population estimates, including revising figures for recent years. When this revision is published, probably mid-December, we will incorporate it in the Benchmarking Tool and notify you.
For CIN data, the DfE CIN statistics team used the 2021 mid-year estimates for their publication, and have no current plan to revise their publication in light of the revised population estimates. They will review this following the ONS publication and communicate any future changes.
You can check how significantly your LA trend data is likely to be affected by these changes in population estimate by checking the “Benchmarking” tab of the ChAT (cells F929 and F930), or by reviewing the attached detailed data table.
We don’t yet know what this means for the “Outcomes” publication due in the new year, but we will tell you as soon as we do know.