Evaluation of educational change

These notes capture the state of evolution at the end of Spring ’01 of a graduate course on Evaluation of Educational Change that later became Action Research for Educational, Professional and Personal Change (using a framework described in the book Taking Yourself Seriously, http://bit.ly/TYS2012).


presented in a form that keeps evaluation in the center

A long version of the course title might be “Engagement in Educational Change with Evaluation as a Tool,” where
“Education” is construed broadly to include not only school curricula, but also educational policies and institutional arrangements, and training, coaching, and the conduct of workshops in any setting;
“Evaluation” stands for the systematic study of i) what has been happening before; and ii) of the effects of any changes you implement, presumably changes designed in response to evaluating what had been happening before; and
“Engagement” denotes that the course is not only about evaluating past situations or any future changes, but also about collaborating with other people in several ways:

  • to design and bring about constructive change, which includes-
  • to reflect on what change you really desire;
  • to undertake the evaluations;
  • to dialogue and reflect on the implications of any results; and
  • to insure that the results of the evaluation have an influence on the relevant people and groups, i.e., the potential “stakeholders.”

“Engagement” also reminds you that, in order to contribute effectively to change, you need to be engaged yourself—to have your head and heart together.  The course, therefore, provides tools for personal reflection on your practice.

Evaluation related to Engagement in Educational Change can be summarized in an “Action Research” spiral:

    Systematic study of what has been happening ->
    Reflection & dialogue ->Design an action/change/engagement ->
    Implement this action ->
    Systematic study of effects of the action ->
    Reflection & dialogue ->
    Revise the action to improve it and/or Promote its wider adoption ->

As the course unfolds you should come to appreciate the following flow of thought:
0.  Suppose you are concerned about some educational practice or policy or institutional arrangement, or the equivalent in some other setting [including your personal work and life].

1.  In order to influence/change what is going on, it is important to study systematically what:
a. has been happening; or
b. is about to happen (e.g., under a new mandate); or
c. could happen given a change you or someone else is designing.  (Reflective practice—the goal of the CCT Program—implies that we evaluate the effects of any changes we make and learn from that evaluation. )

2.  “Study systematically” means to evaluate the effects of changes in practices/ policies/ institutional arrangements, either by comparing before vs. after the change, or the changed situation vs. a unchanged control.

3.  Evaluation of changes help you to
a.  Promote their wider adoption, or
b.  Revise the changes, or advance new courses of action

4.  Whether anyone pays attention to the evaluation depends its political use/fulness for mobilizing support and addressing (potential) opposition.  The politics of evaluation and educational change more generally could be a course in itself, but for now note that:
a.  If you build evaluation into your proposals for change, it shows your preparedness to learn from the effects of the change, and this might increase support for making changes (3b) or promoting their wider adoption (3a); and
b.  If you identify the different stakeholders and look ahead to what the research results could allow them to do, this can enter into the process of designing the change and its evaluation.

5a.  Action Research typically starts with 1a, focuses first on studying what has been happening, but inevitably gets drawn into issues of constituency building or “stakeholder buy-in.”
b.  The Evaluation Clock (introduced in week 6) helps you keep an eye on the buy-in of sponsors or stakeholders in deciding what and how to evaluate, how to analyze results, etc.
c.  Participatory Action Research (PAR) achieves buy-in through participation of stakeholders in the designing changes, implementing them, and evaluating them.

6.  Participation is enhanced by facilitation, other group processes, and reflective practices that bring out and acknowledge the different participants’ voices.

7.  In this course you develop your ability to go from concern about some educational (or related) practice/ policy/ institutional arrangement to influencing what is going on.  To this end you:
a. experience, learn, and practice various ways to promote participation and reflective practice (including your own participation);
b. examine critically the evaluations of others (or the lack of the appropriate evaluations); and
c. undertake a project in an area of your particular concern in which you design and perhaps carry out a pilot version of either i) an evaluation of a change and/or ii) facilitating participation in change or facilitating reflective practice.


About Peter J. Taylor
Peter Taylor teaches and directs programs on critical thinking, reflective practice, and science-in-society at the University of Massachusetts Boston. He studies the complexity of environmental and health sciences in their social context as well as innovation in teaching, group process, and interdisciplinary collaboration (see bit.ly/pjtaylor). He is especially interested in conversations with others who are, in diverse ways, "troubled by heterogeneity" (bit.ly/tbhblog)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: