Update: the powerpoint file used during the presentation of this material at the AEA 2011 conference is here:
Evaluating the World: Perspective from a Chinese Proverb
A traditional Chinese proverb is – “盲人摸象” – it invokes the story of the blind men and the elephant. Let’s use this as a metaphor for our process of knowing and our approach to evaluation.
Four blind men heard there was something new and exciting in the forest – an elephant. None of them knew what an elephant was like, so they each took a chance to discover for themselves.
The first man to encounter the elephant felt its trunk. “Why,” he said, “an elephant is just like a snake!” He then felt he understood what an elephant was.
The other blind men came to the forest, wanting to know for themselves what the elephant was like. One man felt the elephant’s leg. “Oh! I see! The elephant is just like a tree!” he exclaimed. Another felt the elephant’s back and declared an elephant to be just like a wall. The last man felt the elephant’s tail and declared it to be just like a calligraphy brush.
This proverb, then, 盲人摸象, invokes our inability to see the big picture, our tendency to “miss the forest for the trees.” And it also serves as a poignant metaphor for the world we face as evaluators.
The natural world is our elephant, and we, in our limited capacities to see and understand everything that influences the way the world works, are the blind men. Adding yet another level of complexity to our task is the specific quality of what we in the social sciences set out to study – the human world.
The human world is even more daunting to our blind eyes. Because of phenomena such as intersubjectivity, complexity, unknown unknowns and gestalt, the human world is hard to describe – or even pin down – with our traditional, empirical approaches to evaluation based on scientific methods that came from the hard sciences. This is where new paradigms can help us.
Along with Patton’s Developmental Evaluation, I also studied the work of Elliot Eisner, specifically The Art Of Educational Evaluation: A Personal View. These two viewpoints serve as roadmaps that help the blind men expand their limited capacities and gain a better understanding of the realities of the human world.
Eisner details an approach to evaluation based on the same paradigm that underlies art criticism. Rather than reducing data to individual factors and statistics, he presents a detail-rich method to describe phenomena and report them in the scholarly literature. He has, in effect, invented a way for the blind men to converse with one another and describe what they are feeling.
Patton’s work on developmental evaluation also enhances our ability to more realistically assess what is happening in the world. By detailing an approach to evaluation that is longitudinal, that grows with the phenomenon being observed, and that is most importantly an integrated approach of evaluator and shareholder together, he has given us a 4-dimensional view of the world we are studying. His incorporation of the ideas of complexity as first suggested by chaos theory help us come to terms with a world that is constantly changing and unpredictable with our traditional models and static snapshots.
We are still blind men, of course. We are still limited by our own cognition and our own mortality. But by being able to view the human world and the systems around us with the added insight of educational criticism and developmental evaluation, we are able to really tackle the elephants in our lives, and improve our social efforts at the same time.
Review of Developmental Evaluation 
