AfrEA Conference 2014 #3: The Three Stages of Evaluation

For a few years now, I have been traveling extensively in the US, the UK, Canada, Europe, and (now) Africa for client work, speaking engagements, workshops that I facilitate, and paper presentations at conferences. In that time, I have noticed something quite amazing.

Slowly and quietly, evaluation has been coalescing into a global movement. I believe it is poised, in the next two to three years, to become one of the dominant forces shaping policy and development worldwide.

But the movement is fragile, and window of opportunity short.

VOPEs

Currently, there are somewhere between 107 and 168 voluntary organizations of professional evaluators (VOPEs) around the world, depending on how you count. They may be local, regional, national, or international in scope; formal or informal in structure; and organized as associations, networks, or loose collaboratives. While they vary greatly, all are dedicated to advancing evaluation and the benefits it can have for society.

As this (poorly photographed) slide illustrates, the number of VOPEs has increased sharply over recent years. I believe this is due in no small part to the work of the International Organization for Cooperation in Evaluation (IOCE) and its initiatives, such as EvalPartners. Through these efforts, VOPEs are being supported as they start up and build their organizational capacity. Some of that support comes from more established VOPEs partnering with less established VOPEs in other countries.

Profession or Business?

In a very interesting session, Peter Walyaula from World Vision, Uganda, discussed the critical role VOPEs should play strengthening the capacity of evaluators at the local and national level. “Too many evaluators treat evaluation as a business, not as a profession,” he observed. Some have a weak base of skills, others strong skills but a limited understanding of what evaluation entails. Too few dedicate themselves to improving their craft.

He gave the example of a project that attracted a number of evaluation proposals. Most were from evaluators who were not up to the task. One written by a university professor stood out.  He was hired and at the conclusion of the project he delivered a well-crafted academic paper that was methodologically sophisticated—but addressed none of the evaluation questions.

It was good work by academic standards; useless by evaluation standards.

I suspect that story sounds familiar to evaluators around the world.

The Three Stages of Evaluation

As I listened to varied perspectives regarding evaluation capacity, VOPEs, and the future of evaluation, I noticed a progression of ideas. In the early days of a VOPE, the critical question for evaluators appears to be, “How can we find more work?” Without work, there is no practice. Without practice, there is no evaluation.

This was certainly the case in the US. The Council for Applied Social Research, one of three organizations that was later rolled up into the American Evaluation Association in 1986, had as its de facto mission helping members secure government contracts.

Soon, however, the question grows into, “How can we do good work?” There was a great deal of discussion on this subject at the conference. How should good work it be defined? How can it be supported? How can we ensure it is valued? It is a critical question, especially as the demand for evaluation continues to grow.

Then, there is a fundamental shift in perspectives and the question becomes, “How can we do good?” This is what binds the VOPEs together. Not economic self-interest, not technical proficiency, but impact. The impact that the profession as a whole can have on humans and the planet they inhabit. If we join together on a global scale for this purpose, we may—in this precious, fragile moment—make the world a better place.

Is this a romantic notion? Perhaps. But I believe we are on the brink of making the romantic reality.