Karlie Gray

What happens when the performance targets we chase take the place of the purpose we set out to serve? Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." A famous historical example is the "cobra effect". During the British Raj, the government, concerned about the number of venomous cobras, offered a bounty for every dead cobra. Initially, this worked well. Then people started breeding cobras for the bounty income. The reward programme was scrapped so the breeders set their snakes free & increased the cobra numbers. The pattern repeats across centuries & contexts. A modern example involved carbon credits. The United Nations set up a programme to pay companies for destroying HFC-23 (a potent greenhouse gas), a byproduct of refrigerant chemical manufacture. The payments were so lucrative that companies in some countries increased production of the refrigerant specifically to generate more of the HFC-23 they’d be paid to get rid of. Health & care has many “cobra effect” examples. The US Hospital Readmissions Reduction Program penalised hospitals for 30-day readmissions. Readmission rates fell. Retrospective analysis of eight million hospitalisations found that 30-day mortality rose for patients admitted with heart failure & pneumonia. The measure improved. Many patients deteriorated at home. This isn’t an argument against measurement. Every improvement initiative I’ve worked on depends on data. Goals give direction. Metrics help us learn, surface inequity & hold us to account for outcomes we say matter. The risk sits in the moment the proxy measurement quietly becomes the purpose. @Digitaltonto argues that leaders should manage for mission, not only for metrics. He points to many activists touting a “rule” from Erica Chenoweth's research that once a protest movement mobilizes 3.5% of the population, it achieves its goals within a year. Some uprisings mobilised more than 6% of a population & still failed. Others succeeded well below 3.5%. The number described what happened in movements that won. It did not manufacture the winning. Five ways to cobra-proof our metrics: 1) Map the whole system. The cobras were bred because the incentive was designed in isolation from the system it was meant to change. 2) Ask the gaming question before launch. Could a team deliver this number in full without delivering the outcome we care about? If the answer is yes, the metric is already fragile. 3) Pair every metric with a counter-metric. Readmissions with mortality; waiting times with clinical outcomes; activity with the experience of patients & colleagues. 4) Have conversations with teams that constantly bring us back to the bigger context. Stay focused on purpose so that the target doesn't become the proxy purpose. 5) Build fast feedback loops. Ask frontline teams & patients what the data is not showing. Distortion shows up in stories long before it shows up in a dashboard. The work of a change leader is to keep the purpose more vivid than the number. When everyone can see clearly what we are here to do, the measure is more likely to stay in service of it. Links: gregsatell.substack.com/p/th… by @Digitaltonto and scienceinsights.org/what-is-… by Science Insights. Second graphic by @sketchplanator