The most significant weakness I can see in the Data to Insight toolkit is also a fundamental strength: all of our tools run in Microsoft Excel. This means that all of our colleagues, across 151 local authorities (LAs) in England and beyond, can use them without delay -- the tools work with data structures which every LA already produces, and run in software which every LA uses -- but it also means that we're sometimes limited to doing just what Excel can do, and nothing else. For good measure, our spreadsheets must work with the oldest supported version of Excel, because LAs can be slow to upgrade (some LAs also can't run macros or VBA code).
This has been a guiding principle for the project since the beginning; the stuff doesn't have to work miracles, it just has to work, so that it saves time and generates insight for local authorities. When the DfE publish new data (as they did last Thursday), we can bump that data into our Benchmarking Tool and get it out to LAs within a day or two (as we did last Friday), and nobody sitting in an LA performance office has to worry about installing this or subscribing to that in order to benefit from the work -- they can just start using it.
The world changes, though, and clever people invent better ways of doing things, or indeed they invent new things to do, and of course Excel isn't always the venue for that innovation. From a local authority perspective, often the arrival of new technical methods and software can feel intimidating, not because colleagues lack aptitude but because LAs as organisations like to make safe bets. When there are several BI tools on the market and a wide plurality of approaches to the new analytics, it's hard for any given LA to make a leap towards a new way of working, and almost impossible for every LA to leap in the same direction. The landscape is bewildering, and only in hindsight does it become obvious the direction things were moving. In five years' time, maybe we'll all be developing in BI Tool X, and maybe we won't; right now we've all got Excel, and then different LAs are making different experiments.
As much as we still love Excel, one thing we'd love to do with Data to Insight is bring people together from different LAs to explore some of those experiments in broader contexts. Last month a colleague showed me some work they did in Greater Manchester to prototype a data validation tool using Python, the basic idea being that they then bid for funding to build a complete data tool and provide it to LAs as a finished product. I think Data to Insight provides an opportunity to do it a little differently -- instead of them using their time to code something up on our behalf, could they help LAs across the country learn to write the data cleaning rules in Python ourselves? Could they help us build our own tool, learn to maintain it, and learn new transferable skills while we're at it?
I asked around, and within a week had a list of 40 people in different LAs who all wanted to get involved. The nice thing about Python, besides its utility and the reasonably straightforward initial learning curve, is that the cost of entry is at the level that LAs prefer. The nice thing about this project in particular is that it takes logic which LA colleagues already know intimately -- data validation for statutory returns -- and uses that as a basis for learning a programming language which could really help expand their analysis toolkits locally. The nice thing about Data to Insight is we can help colleagues in different LAs work together on projects like this, and benefit from each other's work and expertise. I'm sure this isn't the only example of a technology or technique which we could use together to find out if it fits in our shared toolkit.
So we're thinking about ways in which this network, which grew up around a set of common data tools developed in Excel, might help itself to outgrow some of those tools, or to redevelop them in other shapes, and what that process might look like. Lots of learning, certainly.
We're looking to get the Python project, or something like it, off the ground in the new year; hopefully from that we'll be able to gather some useful learning about what works and what doesn't. In the meantime, I'm eager to hear from people with thoughts about how this stuff might play out. What do you want to see? How best can we move things forward? What's going to get in the way, either locally or more broadly? And if you have a project in need of co-conspirators, do you want us to help you find those people?