Why We Need Systems Change Report Cards (And How to Make One)
Written by: Monalisa Salib, The Convive Collective
In this post, we’re taking you into the land of rubrics (aka report cards) for systems change. That’s way too much jargon, so let’s start with what we mean.
What is a systems change report card?
Many philanthropies are trying to influence systems. Fellow Conviver Julia Coffman and her colleague Jewlya Lynn define systems change “as the practice of confronting the causes of social problems rather than treating their symptoms.”
So if a philanthropy is “doing” systems change, they are going under the hood, trying to change the conditions that influence how systems function. As a simple example, it’s not just supporting the food bank. It’s also trying to change the policies, resource flows, relationships, and other conditions that contribute to hunger and poverty in the first place.
Okay, but what are rubrics?
Rubrics are a way to assess whether systems change is happening. They help answer the questions: What is changing in the system? How are the underlying conditions changing? Is the system becoming more likely to produce positive outcomes over time?
Notice that the question isn’t whether we, as the philanthropy, are making this happen. The system a philanthropy is trying to influence is much bigger than the philanthropy itself. Many actors and forces are contributing to change, sometimes in the same direction and sometimes in opposing ones. A rubric helps us pay attention to what is changing in the system, without pretending that any one actor caused it.
Think of rubrics as a systems change report card. Here is an example of a generic systems change report card:
TABLE
Why should we use systems change report cards?
We think a report card can be useful for any philanthropy trying to influence systems change. Why? Because if you're trying to change a system, you need some way of describing what meaningful change in that system would actually look like. And then measuring your progress against that standard. A systems change report card simply makes that possible.
There is one prerequisite: you need at least a working hypothesis about how the system might change. It doesn't need to be perfect or fixed. In fact, you should expect to revise it as you learn. But without some conception of what change looks like, there's nothing for the report card to assess.
But when specifically would a rubrics report card make the most sense compared to other approaches? Perhaps:
Change is too complex for Key Performance Indicators (KPIs) to serve as proxies for systems change or the outcomes are emergent and impossible to predict from the outset;
The change(s) you’re seeking to influence takes too long to show up in your KPIs and you need to show incremental progress;
You’re drowning in data but struggling to see what matters;
Your collection of KPIs doesn’t tell you what it all adds up to. You need a way to make sense of multiple quantitative and qualitative data points and tell a bigger story about how the system is changing;
You want to make credible, context-sensitive comparisons across systems, industries, or geographies that use different underlying data and standardized KPIs may not cut it;
You want shared language across your organization for what systems change looks like and simply articulating what the report card is helps you achieve that shared vision; or
You want to be able to track when the system is regressing (because systems change is complex and you won’t always see a linear form of progress).
All of these challenges can be addressed by a thoughtful systems change report card. Importantly, it doesn’t replace other measurement and learning methods. Your KPIs, contribution analysis, outcome harvesting, Most Significant Change stories, network analysis, or other methods can all provide evidence for the report card. The rubric gives you a structure for making sense of what all that evidence says about how the system itself is changing.
What are some tips for creating systems change report cards?
Make it good enough, not perfect.
We all like to put our best foot forward and create A+ work, but perfection can be the enemy of the good and useful. Don’t focus on making your rubric (report card) perfect. Focus on making it actually useful to program teams and leadership. And importantly, we have to acknowledge that systems change is extremely complex, and no rubric (or anything really) will ever be fully able to capture reality. It’s an approximation–a good enough one to proceed.
Make its development a participatory process.
Creating a systems change report card can firstly get everyone aligned on the vision and hypothesis for what change would look like. That is mission critical; if we don’t all know where we’re headed and how to get there, we may end up rowing in different directions. And there’s an IKEA effect at play here: people tend to feel greater ownership over something they helped build (couch assembly optional!). Involving the people who will ultimately use the report card also increases the chances that it reflects what they need and becomes part of how they understand change, examine their own contributions, and adapt their strategies.
Have both the nerdy version and the layperson’s version.
You may create a backend that is far more technical and long-form, but it’s equally important to have a simple, layperson version available for your users. That simple version gives people the quick language to use in everyday conversation. We know one philanthropy that gives kudos to people for using their rubrics (report card language) in their slack channel. That simple version can also be a communications product that you share externally, so others (partners, funders, collaborators) understand how you see change happening and where you think the system is along that pathway.
At Convive, we have helped clients develop their systems change report cards, evaluate and assess progress using the report card, and facilitate reflective conversations with teams to make sense of that learning for their strategies. If you’re interested in doing this, please reach out using the form here. We’d love to hear from you.