How do you measure the impact of a social change petition platform?
By Danny Hutley
At the Change.org Foundation, we want to continuously learn about and deepen the impact that our teams have. But we have often found that efforts to measure the impact we have tend to fall short. When we oversimplify impact as a set of quantifiable platform metrics, we falsely equate growth or engagement with impact. When we go deep into the individual stories of impact that petitions have, we find it difficult to apply lessons from these case studies to decisions on strategy.
I believe that many organisations face this same struggle because we have not had the hard conversations about what we actually mean by ‘impact’. Complicating this, we find it difficult to let go of business school concepts that obsess over quantifiable indicators, scale and return on investment. This approach simply does not work for ‘impact’.
This year the Change.org Foundation is trying a different approach to impact measurement.
We have defined impact through our organisation’s ‘role’ in society, created a People Power Index and for the impact side of our strategy, we have replaced goals with learning and evaluation processes.
How does this work?
First – we had to do the hard work of defining our impact in society. It’s not enough to say that we are an ‘impact organisation’. British Petroleum also has an impact on society—though a disastrously negative one for the climate. To fix this, we pushed ourselves to clearly answer these questions:
- What is our critique of power and decision-making in the world?
- What is our vision for how this can be different?
- What do we think will drive this shift in power?
- What is our unique role in this? (acknowledging that we are not the only ones who will drive change)
There are several roles that the Change.org Foundation plays in creating a better world – but one of them stood out as a unique driver of our work: “to build society’s belief that people power works.”
We do have a significant role in persuading people that campaigning works, but there are many reasons why this societal belief changes that are outside of our control—for example, new elections, coups or corruption scandals. So we also need to let go of metrics that are entirely within our control. We created a ‘People Power Index’ made up of a mixture of external and internal indicators, that we could study each year to learn about how our work interacts with the wider context.
Here is how the People Power Index works:
We take specific scores from:
- The Economist Intelligence Unit Democracy Index: there are specific sub-scores from this we picked as most relevant to our ‘role’ in society that measure freedoms, fair elections, good governance and political participation.
- The IDEA Global State of Democracy Index: has slightly different values and definitions than the Economist Intelligence Unit, so is a useful balance. Again, we picked the most relevant sub-scores from this index: media, citizens can participate in decisions, absence of corruption and civil society participation.
- The World Values Survey: the % of a population that have used a petition, and the % of the population who would consider protesting. For both of these we plan to carry out our own surveys.
- The Social Progress Index: which we use to give context for progress on a range of different issues in society.
- Change.org itself: which gives us some good data on the proportion of the population who sign online petitions or create their own petitions.
From this we are able to give country teams an ‘index’ that acts as a useful tool for their impact strategies:
- A political context score, made up of six sub-scores from the above external indices
- A belief in people power score, made up of six sub-scores—four external and two internal
- The relative ‘reach’ of change.org in their country
- A way to compare progress on a range of human development scores between countries
It’s motivating to see how this approach allows us to continuously learn about impact—attracting new insights on how our work interacts with various aspects of people power. Moreover, it satisfies our need to base our strategies on a common language of numbers, without oversimplifying or overstating our own role in changing society.
We plan to test this People Power Index and make it stronger over time, and the dream is to do that collaboratively with other organisations.
Interested? Get in touch and we’ll share what we have with you!
learn@changefoundation.org
