Conversations on nonprofit information usually branch out quickly to talk on impact. But what about process? Performance measurement along the route? Less common is the talk of what the steps are that lead up to impact on a real-time and day-to-day basis. Not paying attention to daily practice in this respect is to build a de facto infrastructure for wasted effort and/or poor performance. On the other hand, knowing what to keep track of, why, and what to do about those metrics – daily – is how we gain a clear line of sight into what delivering on a mission really looks like. Bridget Laird, CEO of WINGS, provides a close-up on the subject.
…
Although results-driven since our beginnings, WINGS gained new focus after developing a clear Theory and Change and clarifying precisely what factors and efforts were driving our educational outcomes. Detailed, easy-to-access records allow managers to monitor whether the program is being implemented with fidelity. Real time feedback provides powerful motivation for our staff in aligning efforts with outcomes.
But by far the most significant gain is that this continuous flow of data provides us with valuable lessons in how to improve our program model for maximum effectiveness.
WINGS is an education program that teaches kids how to behave well, make good decisions and build healthy relationships. We do this by weaving a comprehensive social and emotional learning curriculum into a fresh and fun after school program. Founded in 1996, we currently serve over 700 low-income students in South Carolina and Georgia and recently received a $2.5 million dollar grant from the Edna McConnell Clark Foundation through the Social Innovation Fund to replicate even further.
Because we are dedicated to analyzing the scientific effectiveness of our program WINGS, is currently undergoing a 4-year randomized control trial conducted by the University of Virginia.
However, as we await results from this trial, it is imperative we focus on monitoring daily internal data. Every WINGS program relies on a performance management system to track the progress of individual students toward the desired educational outcomes enumerated in our Theory of Change. That Theory states that kids who receive a comprehensive social and emotional education for 15 hours a week within an after school program for 2 years or more will develop high social and emotional skills, and utilize those skills in everyday life. They will demonstrate improved behavior and school attendance in elementary school, behave better and become more attached in middle school with the future hope and expectation of high school graduation.
Here are some of the short-term outcomes that are stepping-stones to the long-term outcomes we seek:
- WINGS students will develop strong social and emotional (SE) skills.
- WINGS students will utilize SE skills in everyday life.
- WINGS students will demonstrate high attachment to school.
We are able to track each student’s progress toward our outcomes, and pinpoint difficulties in mastering the curriculum’s 30 learning objectives.
WINGS is also able to use performance management system to analyze to see where we can make improvements in our model.
For example, one way we measure whether students are utilizing SE skills in everyday life is to review social development grades on students’ report cards. WINGS staff learned from data analysis that a number of students were not receiving positive social development grades from classroom teachers despite our target that 85% of students will receive positive grades. In response, each WINGSLeader (part-time staff) was required to develop a specific plan for the kids who were falling short, and to share the plan with their teachers to intensify and coordinate efforts.
The data showed us those deficits, and demonstrated whether the adjustments that WINGS made in response solved the problem. The results from the next round of report cards validated that the adjustments made a difference.
By building into the process ongoing review and assessment of the data, WINGS has become far more responsive and accountable. Monitoring the data at frequent intervals means that WINGS doesn’t wait until the end of the year, or even the end of a school term, to review how staff efforts are progressing.
But most importantly, this continuous flow and analysis of information allows us to refine and evolve our program model as we prepare to replicate as effectively as possible.