Evaluating Teaching Evaluations

Re-evaluating students' evaluations requires making assumptions explicit. "The reliability of student evaluations could be calibrated if we had a standard and direct way to measure teaching effectiveness. We don't have that, so faith in student evaluations is just that, a matter of faith." [Leamnson, 142] To make his case, Leamnson cites four studies, each of which clearly suggests that students are overly influenced by the presentation of the mode of instruction [Ambady & Rosenthal, Naftulin, et al, Rodin & Rodin, and Williams & Ceci, cited op cit]. Several writers call for distinguishing between summative evaluations (for personnel decisions) and formative evaluations (for improving teaching). Seldin [p. 39], for example, suggests that assessment data gathered for the purpose of improving teaching performance should not be used for personnel decisions, such as merit increases, contract renewal, tenure, and promotion.

Peer evaluation of teaching through classroom visitation is another widely used method of evaluating teaching. However, "peer ratings based on classroom visitation do not appear to be very reliable or to correlate substantially with SETs or with any other indicator of effective teaching...[T]hese findings neither support nor refute the validity of SETs [and] clearly indicate that the use of peer evaluations of university teaching for personnel decisions is unwarranted". [Marsh & Dunkin, 277]

Classroom visitation is likely to have a greater success when used for purposes of performance improvement rather than for personnel decisions. Faculty resistance is less an issue, and anxiety is deflated. [Seldin, 43] According to Marsh and Dunkin [op cit] "Cohen and McKeachie (1980) and Braskamp, Brandenburg, and Ory (1985) suggested that peer ratings may be suitable for formative evaluation, but suggested that they may not be sufficiently reliable and valid to serve as a summative measure". Murray (1980) found peer ratings to be (1) less sensitive, reliable, and valid; (2) more threatening and disruptive of faculty morale, and (3) more affected by non-instructional factors such as research productivity, than SETs. [Marsh & Dunkin, 277]

On the other hand, peers might contribute something of value to the evaluations of teaching, if teaching became more open to view. "Faculty are understandably concerned about bias when it comes to classroom visits and even to reviews of materials. But their fears can be allayed if colleagues who are conducting the evaluation also are meaningfully engaged in other ways. Colleagues are, after all, best situated to evaluate the course objectives, format, content, and grading policies of their peers, and they can win trust and be more helpful when expectations and understandings of the criteria for good teaching are public and shared. [...] The hope that some form of peer review might be brought to bear on teaching is sparking experimentation throughout higher education". [Glassick, et al, 47]

Rodrigo Cambray, a doctoral student in Adult Education, is the Doctoral Assistant in the Office of Teaching and Learning Advancement.

Ambady, N. and R. Rosenthal (1993) Half a minute: predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology 64: 431-41.

Braskamp, L.A., D.C. Brandenburg, J.C. Ory (1985) Evaluating Teaching Effectiveness: A Practical Guide. Beverly Hills CA: Sage

Cohen, P.A., W.J. McKeachie (1980) The role of colleagues in the evaluation of college teaching. Improving College and University Teaching 28: 147-54.

Committee on Undergraduate Science Education. (1997). Science teach-ing reconsidered: A handbook. [http://www.nap.edu/readingroom/books/str/5.html]

Glassick, C. E., Taylor H. M., & Maeroff, G. I. (1996). Scholarship Assessed: Evaluation of the Professoriate. San Francisco: Jossey-Bass.

Leamnson, R. (1999). Thinking about Teaching and Learning: Developing Habits of Learning with First Year College and University Students. Sterling, VA: Stylus.

Marsh, H. W. & Dunkin, M. J. (1997). Students' evaluation of university teaching: A multidimensional perspective. In: R. P. Perry & J. C. Smart (eds.). Effective teaching in higher education: Research and practice. New York: Agathon Press, 241-320.

Murray, H.G. (1980) Evaluating University Teaching: A Review of Research. Toronto: Confederation of University Faculty Associations.

Naftulin, C.H., J.E. Ware, F.A. Connelly (1993) The Doctor Fox Lecture: a paradigm of education seduction. Journal of Medical Education 48: 630-35.

Rodin, Miriam, Burton Rodin (1972) Student evaluation of teachers: students rate most highly instructors from whom they learn least. Science 177: 1164-66.

Seldin, P. (1988). Teaching and learning: Lessons learned in evaluating teaching. Fourteenth International Conference on Improving University Teaching. Umea, Sweden: The University of Umea and The University of Maryland / University College, 38-48.

Williams, Wendy M. and Stephen J. Ceci (1997) How'm I doing? Problems with student ratings of instructors and courses. Change (Sep/Oct): 13-23.