In our latest viewpoint, Anne Whatley from Network Impact provides a summary of their latest report, which is about measuring and evaluating civic tech.
Digital technologies are connecting residents in neighborhoods, catalyzing community discussions, changing the way governments and citizens interact, and making government more transparent. This growing field of “civic tech” is aiming to harness technology to spur citizen engagement, improve cities and make government more effective. The Network Impact team spent 18 months consulting with civic tech thought leaders and offering technical assistance to Knight Foundation grantees focused on assessing the impact of their technologies. One thing that stood out in our conversations is that the civic tech field is ready for a deeper dive on outcomes. Designers of these civic tech platforms and apps are already measuring results, such as the number of users, and are very familiar with tools like Google Analytics. But these instruments don’t explore deeper changes like increases in civic engagement, more inclusive community decision-making, and open governance that may be occurring as a result of their initiatives. A big topic of discussion as the field is maturing is how to track on-the-ground outcomes for people and communities, in addition to counting clicks, views, downloads and tweets.
“Now we need to find out not only what works, but what works best over time.”
Code for America, one of the organizations that we worked with, is already implementing outcome assessments. Code for America tracks their impact on open governance ecosystems by monitoring a range of results, including the number of apps created by fellows that are sustained by municipal administrations. In addition to tracking app deployments, Code for America has developed metrics that help them understand their impact on the governments they work with, the communities they engage, and the larger civic technology ecosystem. When we asked Code for America team to summarize the need they see in the field, they responded, “The civic tech field is growing rapidly. Demonstrations of what is possible are up and running in communities of every size across the United States. Now we need to find out not only what works, but what works best over time.”
A few lessons for assessing civic tech from our work include:
A Team Approach
It’s important to engage your full team, especially tech development staff, early in evaluation planning to ensure that systems are in place to capture the data you’ll need to track progress and assess results. For example, if location information will prove useful in tracking place-based impacts, best to build this metric into registration from the start, rather than try to capture the information later.
“Strong evidence of success in reaching a targeted objective will create a firmer foundation for future efforts and send a clearer signal to supporters.”
Hone in on your true scope of influence
Start by focusing your assessment on the outcomes that your app or platform is most directly designed to produce (such as increased trust between neighbors or resident contributions to a specific local decision-making process) rather than on very broad overarching goals like “strengthening participatory democracy.” Strong evidence of success in reaching a targeted objective will create a firmer foundation for future efforts and send a clearer signal to supporters, users and the field than a broader claim for which you have limited evidence.
Learn and borrow from others
Seeing how others are doing this work can help you develop or improve your assessment plan and lead to improved results. The guides we created based on our work with civic tech teams include many examples of outcome assessments linked to overarching objectives that include increasing civic engagement, promoting deliberative democracy, and supporting open governance.
Guides to Civic Tech Assessment
We summarized what we learned in two guides that are designed to help civic tech innovators monitor progress towards their intended outcomes and evaluate the impact of their efforts.
- Assessing Civic Tech: Case Studies and Resources for Tracking Outcomes is a publication of the Knight Foundation with Network Impact that focuses on measuring the impact of civic tech platforms on people, places, and processes.
- How to Measure Success: A Practical Guide to Answering Common Civic Tech Assessment Questions is a Network Impact publication that offers examples and advice for monitoring a platform’s ongoing performance using tools and approaches that are effective and practical.
Many thanks to Anne Whatley from Network Impact and Beth Kanter, who first featured this article on her blog.
For more information about civic tech and measuring success visit the Network Impact website, which has useful resources and tools, or get in touch with them directly at @networkimpact. You can follow Beth on Twitter at @kanter.
To stay up to date with the latest Markets For Good articles and news, sign up to our newsletter here. Make sure that you are also following us on Twitter.