The Laws of Evaluation Quality

It has been a while since I blogged, but I was inspired to give it another go by Evaluation 2010, the upcoming annual conference of the American Evaluation Association (November 10-13 in San Antonio, Texas).  The conference theme is Evaluation Quality, something I think about constantly.  There is a great deal packed into those two words, and my blog will be dedicated to unpacking them as we lead up to the November AEA conference.  To kick off that effort, I present a few lighthearted “Laws of Evaluation Quality” that I have stumbled upon over the years.  They poke fun at many of the serious issues I will consider in the upcoming months and that make ensuring the quality of an evaluation a challenge.  Enjoy.

Stakeholder’s First Law of Evaluation Quality
The quality of an evaluation is directly proportional to the number of positive findings it contains.

Corollary to Stakeholder’s First Law
A program evaluation is an evaluation that supports my program

The Converse to Stakeholder’s First Law
The number of flaws in an evaluation’s research design increases without limit with the number of null or negative findings it contains.

Corollary to the Converse of Stakeholder’s First Law
Everyone is a methodologist when their dreams are crushed.

Academic’s First Law of Evaluation Quality
Evaluations are done well if and only if they cite my work.

Corollary to Academic’s First Law
My evaluations are always done well.

Academic’s Lemma
The ideal ratio of publications to evaluations is undefined.

Student’s First Law of Evaluation Quality
The quality of any given evaluation is wholly dependent on who is teaching the class.

Student’s Razor
Evaluation theories should not be multiplied beyond necessity.

Student’s Reality
Evaluation theories will be multiplied far beyond necessity in every written paper, graduate seminar, evaluation practicum, and evening of drinking.

Evaluator’s Conjecture
The quality of any evaluation is perfectly predicted by the brevity of the client’s initial description of the program.

Evaluator’s Paradox
The longer it takes a grant writer to contact an evaluator, the more closely the proposed evaluation approaches a work fiction and the more likely it will be funded.

Evaluator’s Order Statistic
Evaluation is always the last item on the meeting agenda unless you are being fired.

Funder’s Principle of Same Boated-ness
During the proposal process, the quality of a program is suspect.  Upon acceptance, it is evidence of the funder’s social impact.

Corollary to Funder’s Principle
Good evaluations don’t rock the boat.

Funder’s Paradox
When funders request an evaluation that is rigorous, sophisticated, or scientific, they are less likely to read it yet more likely to believe it—regardless of its actual quality.


Filed under Evaluation, Evaluation Quality

7 responses to “The Laws of Evaluation Quality

  1. Classic!! And so true in so many ways … thanks!


  2. Pingback: Usable Knowledge

  3. Pingback: Evaluation Quality

  4. Pingback: Quality is a Joke « Gargani + Company's Blog

  5. davidfetterman

    I think I said that: “Academic’s First Law of Evaluation Quality
    Evaluations are done well if and only if they cite my work.” (See Fetterman, in press; Fetterman, Deitz, and Gesundheit, 2010; Fetterman, 2010; Fetterman and Wandersman, 2007). Or was that the point of the First Law?!? Nice work John.

    • David — You make me laugh. Of course, the “Unwritten Law of Evaluation” is steal everything that is good (and give credit where it is due). Alas, I have stolen so much from you that I can no longer tell the difference between your ideas and mine. Perhaps we should establish a pact–I will cite you citing me citing you if you cite me citing you citing me. That should work, right?

      • davidfetterman

        Works for me. Just having fun with you (minor self-depricating humor by citing myself in the process).

        But do you remember Tom Lehrer’s song:

        “I am (sic) never forget the day I first meet the great Lobachevsky. In one word he told me secret of success in mathematics: plagiarize! Plagiarize, let no one else’s work evade your eyes. Remember why the good Lord made your eyes, so don’t shade your eyes, but plagiarize, plagiarize, plagiarize! Only be sure always to call it, please, “research”. And ever since I meet this man, my life is not the same. And Nicolai Ivanovich Lobachevsky is his name …”

        Keep up the good work!

        You take care.

        – David

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s