Guidelines for interpreting student comments on end-of-term course evaluations

January 12, 2021 | WKU, Claudia Stanny (ed.)

Guidelines for interpreting student comments on end-of-term course evaluations

The term is finished. You submitted your grades. Now you can take a look at the ratings and comments on your course evaluations.

Perhaps this is your first term teaching. Or you might be a new chair, reviewing student comments on courses taught by faculty in your department. The task of making sense out of the ratings and the comments can be daunting.

Linse (2017) provides a thoughtful analysis of the kinds of information student ratings can and cannot communicate. She reviews the extensive research literature on the strengths and weaknesses of student ratings, ranging from questions about what ratings actually measure through questions about potential biases and distortions in aggregated data. Key findings from her review appear below.

Student ratings are student perceptions about a course and the instructor
Student ratings and comments communicate student perceptions about a course and how an instructor teaches. They can accurately reflect whether students perceive the instructor to be organized and responsive to students and whether course assignments and assessments create experiences that promote student learning. 

Student ratings are not evaluations of teaching or assessments of learning

  • In spite of the labels used for many course rating forms (course evaluations, teaching evaluations, student assessment of instruction), student ratings are not evaluations of teaching, which require specific expertise with disciplinary content and teaching skill that few students possess. 
  • Although students who earn higher grades in a class are more likely to rate the class favorably, the correlation is not perfect. Moreover, positive correlations between grades and course ratings does not provide sound evidence that awarding high grades will cause higher ratings (a common reasoning error). Students who learn more may simply value their class experience more, earn better grades, and rate courses more favorably.

Advice to faculty when reviewing student ratings

  • Avoid becoming defensive in the face of negative student comments. New faculty should avoid reflecting on their first course evaluations alone, when they are most sensitive and may over-react to individual comments. Seek the advice of a trusted colleague or consult with staff at the teaching center to interpret course ratings in context.
  • We remember the first and last comments we read. Comments with an emotional charge (positive and negative) create more lasting impressions. Negative comments are especially prone to linger in memory and can have an oversized influence on our impressions. Thus, when we read a summary of student comments in the order they appear, we might develop a distorted impression.
  • Look for patterns in the comments, which help defuse the hurt associated with a lone comment from one unhappy student. To avoid the distortions created by these cognitive biases, organize the comments and look for patterns. Keep counts of both positive and negative comments. Typically, a group of 15-20 comments produce 3 or 4 common themes. Count how often students comment on each theme. 

Seek feedback about your teaching from additional sources

  • Ask colleagues who are skilled teachers or who share expertise in your discipline to give feedback about the quality of course design, assignments, and how you interact with students.
  • Join Teaching Partners and get feedback from a colleague on your class. If you teach online, ask for a Quality Matters review. Teaching Partners can also give feedback after visiting an online class as a guest instructor or guest student.
  • Student comments often describe behaviors and personality characteristics of teachers that researchers identify as characteristics of master teachers (Keeley, Smith, & Buskist, 2006). Descriptions of faculty nominated for teaching awards include comments on the following themes:
    • Knowledge of the subject matter, 
    • Communication skills (clarity of explanations, use of examples, clarity of speech), 
    • Ability to explain challenging content, 
    • Organization / preparation for class,
    • Approachable / available to help students, 
    • Respectful behavior/attitude toward students, 
    • Enthusiastic about teaching and their discipline, and
    • Timeliness and usefulness of feedback.

 

Resources

Keeley, J., Smith, D., & Buskist, W. (2006). The Teacher Behaviors Checklist: Factor analysis of its utility for evaluating teaching. Teaching of Psychology, 33 (2), 84-91. https://doi.org/10.1207/s15328023top3302_1

Lewis, K. G. (2001). Making sense of student written comments. New Directions for Teaching and Learning, No. 87, 25-32. https://doi.org/10.1002/tl.25 

Linse, A. R. (2017). Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees. Studies in Educational Evaluation, 54, 94-106. http://dx.doi.org/10.1016/j.stueduc.2016.12.004 

Teacher Behaviors Checklist. PDF available on the CUTLA website. https://uwf.edu/media/university-of-west-florida/academic-affairs/departments/cutla/documents/Teacher-Behaviors-Checklist.pdf  

01/12/2020 ajc