Fundamentally, reform is a long game. Design and delivery take years; realising the impact may take a generation. In that context, reform needs to both endure and evolve. Continuity of sponsorship, leadership and resource are foundational. Successful reform must also be able to look through to the future it is creating, sensing and responding to markers of progress (or otherwise) along the way.
Reform balances the challenge of creating a sustainable new world while decommissioning the old one. For reform to be sustainable, it needs to look through the change to build the capabilities needed to be successful in the changed system. The current talent crunch is shining a light on some of our biggest gaps and the time it takes to build the workforces we need. Reform needs to build both the technical skills to operate the new system and the future of work capabilities.
Globally, we are more attuned to the possibility of known and unknown shocks than we have been in a century. The new worlds our reforms are building cannot merely be better than today’s, they need to be able to withstand and bounce back, and even improve and evolve in the face of shocks. Much is known about the qualities of adaptive and resilient systems, and these should be central to our reform objectives. For example, building capacity and capability into the system, creating the information pathways to learn and disseminate learning, and instilling agility in decision-making.
In reform, as with organisations, measuring impact provides insight into the effects and performance being delivered to individuals, whānau and communities. Reform programmes, and the systems they are reforming, are by their nature magnitudes more complex than individual initiatives. Implementing reform is likely to require highly interrelated and complex approaches. Outcomes may be hard to predict with certainty.
In this context it is useful to take a tiered approach to measuring results: at the initiative level to confirm we are doing the things we committed to, at the outcome level to confirm those things are delivering what we thought they would, and at the system level to inform us of the health of the system being reformed. At all three levels, our ways of knowing should be broader than the things that we can measure.
Outcome measures have their place, particularly where participants and citizens have clear shared definitions of success. Water quality, equitable access to healthcare and emissions can absolutely be measured (with baselines and progress charted). But cost/benefit analysis doesn't always fit with the types of outcomes we are looking for, or for the timeframes we are working in. Alongside quantifiable measures, we need broader ways to chart how mindsets are shifting, and how the things that we value, for example rangatiratanga, or the quality of relationships and partnerships, are being impacted and created.
System health might include how well the system is able to learn and evolve, how equitably distributed power and resources are, how adaptive it is to change in the system and how resilient it is.
“There is utility and logic in evaluating what we have done - but I don't buy quantification of benefits realisation. We can't measure love.” – Helmut Modlik
In the context of intergenerational change, we need markers of success built in from the outset of reforms to provide confidence in the reform itself. How well have we truly understood the root causes? How does our theory of change line up against our drivers? Do we have the key ingredients for success in place?
We also need early indicators of the impact we are seeking. It may be many years before we are able to confirm whether a system is more agile and responsive than it was before. However, we can look for indicators along the way that give us confidence: the diversity of participants in the system, or the nature and cycles of investment into people and infrastructure.
A broader set of measures and markers of success increases our understanding of what works and why, what is changing and how, and how the system is working. With a sensing capability and capacity in place to monitor those markers and to look for unexpected and unintended outcomes, reform programmes are armed with the insight to evolve their interventions, their theory of how change will be made and sustained, and even their analysis of the causes of performance and outcomes.
If successful reform sets the ambition of moving mountains, we need the people and the tools and together, we need the ability to plan for success, adjust for complexity and stay the course. We present our recommendations for doing just that.