As we explore this theme of Evaluation, perhaps what you read next will be familiar to you no matter where your nonprofit organization resides in the world. This commentary, however, focuses on gathering the raw material in Mexico, where Filantrofilia, a rating organization with information on now over 250 organizations, conducts its work. On this backdrop, Anne Hand, International Resources Executive at Filantrofilia, argues that a culture change in the social sector is needed if we are to make the necessary steps toward a systematic and coordinated way to collect high-quality data.
Imagine you have a million dollars you would like to contribute to an organization that really impacts its beneficiary population. Imagine you would like to do this in the Mexican context. You do some research, and ask around for specific, detailed information on the performance of certain programs you discover, that catch your interest. Mexican nonprofit organizations tend not to have this sort of data, and if they do it’s generally incomplete. This is due to a number of factors:
- The Culture of Nonprofits. NGOs don’t necessarily see themselves as needing sustainable business models and don’t see the need to document everything.
- Relevant data rarely being used to make decisions. Sometimes data in the Mexican nonprofit context is only generated if a donor specifically requests it.
- Lack of Transparency. On this point, I note 3 types of NGOs in Mexico with respect to using data to self-monitor performance: 1) Those who know they should use data to measure their own performance and do it (a minority), 2) Those who know and don’t do it because the day-to-day is challenging enough (majority), 3) Those who choose not to do it, primarily for reasons of “safety,” which is reason but not justification.
- Market Forces. The Mexican social sector as a market is not yet mature enough to compensate for those who don’t like to collect data on impact and performance. This is a trend we’ve noted particularly with respect to welfare organizations.
- Different definitions of performance. There is a lack of sector-wide consensus on what “performance” might mean. When we at Filantrofilia talk about performance, what we are really discussing is the impact of an organization’s operations for its beneficiary population.
Filantrofilia’s rating system takes into account institutional development and social impact of an NGO and constructs an aggregate of this data to understand performance in terms of how successful is the NGO at its projects (Social Impact) and how organized the NGO is in terms of being a social business (Organizational Development). In addition, we always suggest that rated NGOs incorporate their own metrics and indicators in their work, if they do not already do so. These metrics will often include:
- Impact and results indicators for specific programs
- Systematized information
- Databases with relevant information for programs and services
To maintain our impartiality, however, we do not offer NGOs consulting services on implementing these recommendations.
With this backdrop, there is still much we do not know about the way that NGOs incorporate either their own data or data from our ratings to improve performance. This is because of Filantrofilia’s relatively short time operating, and the need for a critical mass of rated and re-rated organizations to effectively address this question. What we do know right now is that NGOs take and adapt many of our suggestions for improvement.
Most NGOs say they don’t have money to bring their data operations up to speed. ICT (Information and Communications Technologies) and M&E systems are expensive. Many also cite a lack of human resources, the talent needed for sustainable integration of data-driven management. Complicating the issue is that donors don’t give specifically for IT systems or databases: they give for operational support.
A business that only invests in the last step before the final output, say, a beverage manufacturer funding only the delivery of current product sets, misses out on dynamic market data pertaining to everything leading up to that step: R&D, management development, feedback, and information systems, among other critical business competencies. The social sector is no different.
What do we need, then? Most importantly, we need a culture change at the sector level that would welcome a natural alignment of professionalization and capacity-building with donors understanding of how their support might best impact organizational sustainability. If we take the for-profit example, funding allocation has been done in this way for a long time, simply to ensure competitiveness.
On behalf of the NGOs we work with, Filantrofilia would welcome this shift in culture toward a social sector that takes its data seriously. The current infrastructure, in these terms, is too highly fragmented by individual organizational capability. This will require a meeting of the minds and agreed-upon funding strategies quickly following, to the benefit of everyone currently involved, and all future participants.