I have a great fondness for the American Evaluation Association and its Annual Conference. At this year’s conference—Evaluation 2012—roughly 3,000 evaluators from around the world came together to share their work, rekindle old friendships, and establish new ones. I was pleased and honored to be a part of it.
As I moved from session to session, I would ask those I met my favorite question—What have you learned that you will use in your practice?
Their answers—lists, connections, reflections—were filled with insights and surprises. They helped me understand the wide range of ideas being discussed at the conference and how those ideas are likely to emerge in practice.
In the spirit of that question, I would like to share some thoughts about a few ideas that were thick in the air, starting with this post on complexity.
Complexity: The Undefined Elephant in the Room
The theme of the conference was Evaluation in Complex Ecologies: Relationships, Responsibilities, Relevance. Not surprisingly, the concept of complexity received a great deal of attention.
Like many bits of evaluation jargon, it has a variety of legitimate formal and informal definitions. Consequently, evaluators use the term in different ways at different times, which led a number of presenters to make statements that I found difficult to parse.
Here are a few that I jotted down:
“That’s not complex, it’s complicated.”
“A few simple rules can give rise to tremendous complexity.”
“Complexity can lead to startling simplicity.”
“A system can be simple and complicated at the same time.”
“Complexity can lead to highly stable systems or highly unstable systems.”
“Much of time people use the term complexity wrong.”
We are, indeed, a profession divided by a common language.
Why can’t we agree on a definition for complexity?
First, no other discipline has. Perhaps that is too strong a statement—small sub-disciplines have developed common understandings of the term, but across those small groups there is little agreement.
Second, we cannot decide if complexity, simplicity, and complicatedness, however defined, are:
(A) Mutually exclusive
(B) Distinct but associated
(C) Inclusive and dependent
(D) All of the above
From what I can tell, the answer is (D). That doesn’t help much, does it?
Third, we conflate the entities that we label as complex, complicated, or simple. Over the past week, I heard the term complexity used to describe:
- real-world structures such as social, environmental, and physical systems;
- cognitive structures that we use to reason about real-world structures;
- representations that we use to describe and communicate our cognitive structures;
- computer models that we use to reveal the behavior of a system that is governed by a mathematically formal interpretation of our representations;
- behaviors exhibited by real-world structures, cognitive structures, and computer models;
- strategies that we develop to change the real world in a positive way;
- human actions undertaken to implement change strategies; and
- evaluations of our actions and strategies.
When we neglect to specify which entities we are discussing, or treat these entities as interchangeable, clarity is lost.
Where does this get us?
I hope it encourages us to do the following when we invoke the concept of complexity: define what we mean and identify what we are describing. If we do that, we don’t need to agree—and we will be better understood.
EvalBlog 