Introduction

by Dr. David Williams

Title: Novices evolving evaluation values through Developmental Evaluation

Welcome to our session—

Graduate students in an introductory evaluation course compared Patton’s Developmental Evaluation and several other texts on evaluation.

They applied what they learned while evaluating an emerging technology-based learning suite being created simultaneously by their university’s center for teaching and learning.

The students created mind maps to summarize what they learned about values, valuing, realities of evaluation practice, their personal evaluation lives, contrasts among approaches, and essential principles of evaluation such as flexibility, authenticity, responsiveness and spontaneity.

In this session, three of the students will share their mind maps using stories, music, dance, mathematical graphs, and art to portray their evolving values and the values of stakeholders as these played out in this learning-by-doing evaluation apprenticeship.

As their professor, I was learning about Developmental Evaluation with the students. We met each week to discuss a new chapter, to see how we might apply the ideas in it to an evaluation project, and to compare what we were learning to other approaches to evaluation we were reading about in other texts. Since that class, I’ve been applying those ideas to an evaluation I’m helping conduct for several universities in Utah. I have been discovered that stakeholders who are are not forced to follow a particular evaluation model as part of their funding are very open to the freedom to adjust their learning processes as they go in pursuing their project goals. But if they are used to a certain approach to evaluation or are worried about what their sponsors might think if they “muddle around” trying to figure out what they really need, it takes a lot more work to persuade them to consider using Developmental Evaluation principles to energize and legitimize that muddling.

After hearing from the students, Dr. Michael Patton will explore implications and respond to these students’ efforts to dive into and begin interpreting and growing the field of evaluation.

Finally, we’ll invite you, the audience, to share your reactions and thoughts.

Additional thoughts I may draw upon in the introduction or in response to the students’, Dr. Patton’s, and the audience members’ comments:

As we develop the evaluation field, evaluators are responsible for involving new theorists and practitioners. An introductory evaluation class of graduate students joined a seasoned evaluation professor in discussions about values, valuing, audiences, stakeholders, evaluand definition processes, purposes, key evaluation questions as shaped by societal and political values, and quality criteria. They explored how these issues reflect participants’ values, and alternative ways for clarifying and comparing what should be to what is through evaluator-stakeholder communication, and associated design, collection, analysis, and reporting activities.

To promote dialogue with other evaluators, class members studied Patton’s Developmental Evaluation (2011) and helped evaluate an innovative suite of electronic learning/teaching tools. They also studied and shared ideas from studying other evaluation texts (Davidson, 2005; Eisner, 1985; Fenwick & Parsons, 2009; Fitzpatrick, Sanders, & Worthen, 2004; Kirkpatrick & Kirkpatrick, 2006; Morell, 2010; Preskill & Catsambas, 2006; Reeves & Hedberg, 2003), current evaluation articles, websites, their own metaevaluations of completed evaluations and by keeping journals in which they reflected on their personal evaluation lives and evolving views on evaluation theory and practice.

Each student will present their mind map in which they summarize what they learned about values, valuing, realities of evaluation practice, their personal evaluation lives, contrasts among approaches, and essential principles of evaluation, such as flexibility, authenticity, responsiveness and spontaneity. They will employ stories, video, music, dance, sports illustrations, and/or art to portray their values and those of stakeholders that played out in this learning-by-doing evaluation apprenticeship.

The students will address the relevance and importance of their experiences to their future activity in evaluation, to the field of evaluation, and to values and valuing, with implications for evaluation theory and practice that emerged from their studies and their efforts to apply what they learned to an evaluation study.

The chair and discussant will respond and invite audience members to exploring implications of the students’ experiences and insights for growing the field of evaluation.

References

Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks, CA: Sage.

Eisner, E. W. (1985). The art of educational evaluation: A personal view. Philadelphia, PA: Falmer Press.

Fenwick, T. J., & Parsons, J. (2009). The art of evaluation: A resource for educators and trainers, Second Edition. Toronto: Thompson Educational Publishing.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation:Alternative approaches and practical guidelines (3rd ed.). White Plains, NY: Longman.

Kirkpatrick, D. L. & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd Edition). San Francisco, CA: Berrett-Koehler Publishers.

Morell, J. A. (2010). Evaluation in the face of uncertainty: Anticipating surprise and responding to the inevitable. New York: Guilford Press.

Patton, M. Q. (2011). Developmental Evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford Press.

Preskill, H. & Catsambas, T. T. (2006). Reframing evaluation through appreciative inquiry. Thousand Oaks, CA: Sage.

Reeves, T. C., & Hedberg, J. G. (2003). Interactive learning systems evaluation. Englewood Cliffs, NJ: Educational Technology Publications.