Amanda Babine, Director of Evaluate for Change, explains how social sector professionals can learn to embrace a culture of data use and evaluation to advance their missions.
Picture this: I’m standing in front of a room of dozens of social sector professionals, ranging from Executive Directors to program assistants, and I ask, as I often do, “Raise your hand if you have taken a statistics course before.” As usual, a majority of hands go up.
“Now keep your hand up if you enjoyed that class.” Hands shoot down almost as fast as they were lifted. Suddenly, we have only a couple people with raised hands and the room is filled with laughter.
As the conversation continues, we touch on their background in statistics or evaluation and quickly find that an overwhelming amount of participants found these courses “boring,” “dreadful,” or “difficult” — all their words, not mine.
The language they use describes a common experience with required statistics courses that focus on fundamentals. While important, this type of formal education can seem insignificant and unrelated to working in nonprofits across the country.
This is an all-too-common narrative that sets the stage for a day-long training on building a culture of evaluation among social sector professionals. After spending time assessing prior knowledge and the level of training that participants have, the question remains: are they truly prepared to effectively measure their impact? And if not, what needs to change in order to create a more data-driven and results-oriented sector that makes a measurable difference?
Rooting the type of formal education I speak of in real-world practice is just one way we can increase the motivation of tomorrow’s nonprofit workers to leverage data and evaluation. As grants are increasingly tied to outcomes, the social sector will be pushed to hire and maintain workers with a strong evaluation skill set. Not only creating but requiring curriculum that tackles how to measure impact among people, communities, and social issues is a step in the right direction. But this type of institutional change will take time, making it crucial to create alternatives.
Currently, one of the most common solutions to filling the shortage of staff with evaluation skills is to hire evaluation and data science consultants. There are many data scientists and program evaluators specializing in working with nonprofits; however, relying on their services is only a quick fix, not the long term solution that the social sector needs. While these specialists have strong technical skills, they may be disconnected from the experience of working with specific populations, and they may misjudge an organization’s ability for implementing an evaluation.
For years, I personally worked closely with both evaluators and nonprofits throughout New York City, serving as a resource to the mid-sized organizations that needed to collect and analyze data but didn’t have the expertise required to execute the project.
I’ve found that the relationships between evaluators and organizations can often become strained by not having clear expectations and an understanding for the nonprofit’s ability to leverage data. Most staff members I worked with showed an interest in learning the required skills, but they never had the opportunity to be properly trained to collect and report on data. Adopting the practice of tracking internal processes and overall outcomes has been shown to help organizations become more effective entities. So why are so many organizations still overlooking the techniques that us data nerds find so important?
In my experience, there are two mutually influencing factors behind this gap: commitment and capacity. While the technical skills are the fundamentals needed to measure true impact, there is a crucial step that most evaluators miss when they work or consult with organizations — culture creation. Organizations must strive to create a culture that understands the value and reward in implementing and tracking data within their organization. Like any new initiative, efforts to implement internal evaluation will almost always fall short without buy-in from the organization.
There are also other internal organizational barriers to culture change, such as concerns among staff who might be intimidated by math or might be worried about how the findings will be used against them, which happens when organizations have a punitive process. Once cultural issues like these are addressed, the learning can happen!
For the last three years, I have had the pleasure of directing an organization that focuses on building the evaluation capacity of social sector organizations. At Evaluate for Change, we do not consult, but rather teach. We use curriculum to give adult learners engaging content, real-world examples, and plenty of time to work on practicing the skills they need to measure the impact of their organizations.
The process we take isn’t solely about the technical skills — if that was the case, it may be a bit easier. Our pedagogy is rooted in the idea that organizations need a more holistic approach to implementing a culture of evaluation, and we tailor our classes to accommodate that. We take the fundamentals that most people have blocked out from their first introduction to statistics and reteach it in a way that is relevant and engaging. These are the skills that people in the social sector need to be data-savvy and understand the importance of using evaluation for advancing their organizational missions.
Let me be clear, Evaluate for Change is not the only alternative for helping the nonprofit workforce gain the skills they need to measure and grow their impact.
Currently, we rely on a fractured system that is not self-sustaining — the paradigm of working separately as data scientists and social sector professionals needs to shift to allow for a more integrated approach. It doesn’t matter if you’re an evaluation consultant, teaching statistics as a college professor, or are an enthused social sector professional — the key is to emphasize the need among social sector organizations for both commitment to using data, and capacity for implementing and sustaining sound data use practices. The resulting culture of evaluation can empower social sector professionals and their organizations to leverage their true impact.
Special thanks to Amanda Babine for sharing her expertise on how to empower nonprofits through a culture of data use and evaluation. Visit the Evaluate for Change website to learn more.
To stay up to date with the latest from Markets For Good, sign up for our newsletter and follow us on Twitter.