The Democratization of Evaluation

A few weeks ago Andrew Means wrote a provocative piece about the ‘Death of Evaluation’ here on Markets for Good. Andrew and I had a bit of a back and forth in the comments section of his post, and others have written responses to Andrew’s arguments on this blog as well. Since Andrew’s original post, I have had the chance to engage in an offline discussion with Andrew and now better understand his more nuanced position on how he sees the ‘Role of Data’ (his follow up post) in the social sector.

Not surprisingly, those who stood up to defend against Andrew’s suggestion that evaluation is dying or dead were those who one way or another make their livings providing evaluation services, present company included. While I do not believe that evaluation is dying per se, I do think it’s evolving. The new area of evaluation we are entering will be great for the social sector, but could be trouble for us evaluators.

Let me explain.

Technology is a democratizing force. WordPress gave anyone with an opinion a publishing platform, and Google made hard to access information ubiquitous and instantly accessible. Technologies like these have been great for the majority of people, but threatening to a few. Newspapers have struggled to compete against blogs and information businesses like phone directories and encyclopedias have been all but decimated.

Just as so many other industries have been disrupted by technology, the proliferation of open data and continued evolution of social sector technologies has the opportunity to democratize evaluation.

New technologies are enabling social investors to investigate the social and environmental contexts in which they make their investments, and front line social service providers have access to sophisticated client management solutions that allow them to examine client outcomes as often as they choose, all without the intervention of so called “evaluation experts”.

This democratization of evaluation is great for the social sector. Evaluation should not solely be the domain of those who are members of American Evaluation Association. Instead, every person in the social sector should have an interest in evaluative techniques, and access to tools that lower the cost and increase the frequency with which we can assess whether our interventions are achieving their intended impacts.

Although evaluation is changing in so far as evaluation tools are becoming more accessible, I strongly disagree with any suggestion that evaluation is dying or that evaluative techniques are somehow passé. The evaluation principles of randomization, experimentation, and counterfactual estimation are tested, tried, and true. Indeed, evaluative techniques are not an invention of the social sector, but rather a toolset amassed from statistics, econometrics, and increasingly machine learning disciplines.

There is no pressing need to cast evaluation aside in the social sector. Rather, we face the exact opposite challenge of empowering everyone from philanthropists and foundations to nonprofits and individual donors with the tools and competences to assess social progress with the critical eye of an evaluator in near to real-time.

The reports of evaluation’s death have been greatly exaggerated. Evaluation, in both its summative and formative incarnations, is the life-blood of social progress. Evaluation provides a framework in which we can determine what is working, what is not, and how we can improve. Rather than snuff that type of thinking out, it is imperative we democratize evaluation so it is standard competence rather than the exclusive domain of specialists.

Many thanks to David Henderson for sharing his views on a topic where there has been so much quality debate. What do you think about the democratization of evaluation? Is this progress, or are we looking at this from the wrong perspective? As ever, let us know below, or on Twitter, where you can also follow David directly.