Moving beyond conventional rubrics

Using the following rubric, most conventional rubrics are inexpert.

The rubric:
Expert: Helps the evaluated person see how to improve and can be used formatively in self-assessment
Proficient: Helps the evaluated person see how to improve
Needs Improvement: Evaluates the person on multiple criteria
Does Not Meet Standards: Allows evaluations to be averaged out to an overall figure or category.

Moreover, they waste a lot of paper. In the following example (drawn from Marshall 2009), all the information we really need is that teachers should aim to anticipate misconceptions that students are likely to have and plan how to overcome them.

(At least this example does not commit the all-too-common sin of listing multiple unrelated criteria, which often leads to the evaluated person not fitting in any box because they meet, say, some of the expert criteria, some of the proficient, and some of the needs improvement.  The fudging that goes on to assign a box anyway undermines the credibility of rubric use.)

We could strip most conventional rubrics down to one column that captures the relevant criterion.  The question still remains: how does one bring any give criterion about if it is not happening?  In order to help the evaluated person see how to improve we need to move beyond the conventional rubric.

The first step I recommend to my students, some of whom are teachers, is to complete a regular plus-delta evaluation for a manageable set of criteria, say, 5-15 items that you are prepared to focus on at this point.  (Too many items and none get much attention or it is too time consuming to keep evaluating your performance regularly.)  The evaluation might be done by an observer or it might be done by you directly after the class or other performance being evaluated.  Paying attention to the things you did well (the plus) makes it more likely that you will work on the things that need improving or changing (the delta).  This approach assumes that you (or the person you ask to evaluate you) have ideas about ways to improve.  If some issue arose that you do not know how to address, the plus-delta evaluation puts you in a good position (in terms of rich detail about the class or other performance being evaluated and emotional state) to raise it for discussion with someone who might have more experience or ideas.

I do not know the history about the rise of conventional rubrics.  I do know that I have encountered many people who use or advocate rubrics but were not able to show me that rubrics are used to help the evaluated person see how to improve or how to do that.

Reference

Marshall, K. 2009. Teacher Evaluation Rubrics.  http://ecologyofeducation.net/wsite/wp-content/uploads/2009/09/teacher-eval-rubrics-may-16-09.pdf

Advertisements

About Peter J. Taylor
Peter Taylor is a Professor at the University of Massachusetts Boston where he teaches and directs undergraduate and graduate programs on critical thinking, reflective practice, and science-in-society. His research and writing focuses on the complexity of environmental and health sciences in their social context, incl. Unruly Complexity: Ecology, Interpretation, Engagement (U. Chicago Press, 2005) and Nature-nurture? No (2014, http://bit.ly/NNN2014). On reflective practice, see Taking Yourself Seriously: Processes of Research & Engagement (with J. Szteiter, 2012, http://bit.ly/TYS2012).

One Response to Moving beyond conventional rubrics

  1. Mare Ambrose says:

    Thanks, Peter, for your suggestion. I have been using rubrics for a long time and I have found them helpful. But I have also struggled with creating ones that are not cumbersome and, frankly, in the end, a useless waste of time. The four part rubric is definitely too much – and not effective. I agree that one column is all you need – the person being evaluated must know what is expected as the standard, and strive to get there. I think the criteria needs to be specific and skill oriented so that you know what you need to work on. If some kind of measurement is required, the rubric could offer some kind of continuum (on a scale of 1-4, for example) so that you know where you stand. 4 means you’re good to go; anything else indicates room for improvement. But I’m not sure this is necessary.

    Using the rubric for teacher evaluation (taken from the example in the Marshall reference), one item might refer to assessment: Prepares diagnostic, on-the- spot, interim, and summative assessments to monitor student learning. It’s not enough to say, “You’re not measuring students appropriately.” The rubric spells out what it is that is expected of a teacher who should be assessing her students in a variety of ways. If you’re not doing it, you need to get there. The missing pieces can be circled or underlined to indicate what needs to be changed. That would be a delta.

    By the way, the best approach is for each party to prepare the rubric together – beforehand – so that there is agreement on what is expected as the standard. It is worth a conversation in the beginning. In this way the distinction between instruction and assessment is blurred. When I took the Research for Better Teaching training (Jon Saphier and Robert Gower), I remembering reading an article about instructional rubrics: “Using Rubrics to Promote Thinking and Learning” by Heidi Goodrich Andrade, an assistant professor at Ohio University in 2000. She did the research at Project Zero at Harvard.

    http://www.ascd.org/publications/educational-leadership/feb00/vol57/num05/Using-Rubrics-to-Promote-Thinking-and-Learning.aspx

    In the article is a testimony that “test scores show that students who use the rubrics to assess themselves learn more.” I’m not sure what tests she is referring to, but there are some articles referring to her work with student writing.

    I think it takes practice to prepare and use rubrics effectively. But I think it’s worth the effort.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: