Tag Archives: Program Design

Data-Free Evaluation

 curves

George Bernard Shaw quipped, “If all economists were laid end to end, they would not reach a conclusion.”  However, economists should not be singled out on this account — there is an equal share of controversy awaiting anyone who uses theories to solve social problems.  While there is a great deal of theory-based research in the social sciences, it tends to be more theory than research, and with the universe of ideas dwarfing the available body of empirical evidence, there tends to be little if any agreement on how to achieve practical results.  This was summed up well by another master of the quip, Mark Twain, who observed that the fascinating thing about science is how “one gets such wholesale returns of conjecture out of such a trifling investment of fact.”

Recently, economists have been in the hot seat because of the stimulus package.  However, it is the policymakers who depended on economic advice who are sweating because they were the ones who engaged in what I like to call data-free evaluation.  This is the awkward art of judging the merit of untried or untested programs. Whether it takes the form of a president staunching an unprecedented financial crisis, funding agencies reviewing proposals for new initiatives, or individuals deciding whether to avail themselves of unfamiliar services, data-free evaluation is more the rule than the exception in the world of policies and programs. Continue reading

3 Comments

Filed under Commentary, Design, Evaluation, Program Design, Program Evaluation, Research

Conflicts as Conflicting Theories of the World

nyt_2009_01_25

Theories are like bellybuttons-everybody has one and all are surprisingly different.  Last Sunday Scott Atran and Jeremy Ginges wrote an opinion piece for the New York Times in which they described their research on beliefs about conflict and peace in the Middle East.  In brief, they argued that what many outsiders consider rational and logical solutions to the Israeli-Palestinian conflict, insiders consider irrational and illogical.  The reason has largely to do with sacred beliefs.  In spite of the name, these are not religious beliefs, per se, but rather any deeply held beliefs that sit at the core of our world views and are highly resistant to change.

In an earlier post I described beliefs in general as a pile of pick-up sticks, with the most resistant to change-the sacred beliefs-at the bottom of the pile.  Accordingly, altering sacred beliefs in any significant way will disturb all the rest.  At best this is exhausting, at worst traumatic.

Given the variety of beliefs that abound regarding social problems and solutions, it seems that program designers and policymakers are always treading upon someone’s sacred beliefs.  One of the practical questions we have been wrestling with is how to help groups of people with disparate world views reach consensus about programs and policies.  With the approach that we have been developing, we engage a broad range of stakeholders in a simple, iterative process in which they reveal what they believe and why.

Leave a comment

Filed under Commentary, Design, Program Design