Here’s an interesting article on this subject by Phil Ray Jack, originally posted on January 4, 2008. An excerpt follows:
……At most colleges, the work we [contingent faculty] do is judged primarily, if not exclusively, on our student evaluations. As public perceptions concerning higher education have changed, so have student reactions to what we do. Several years ago, we were seen as the experts who had valuable knowledge to impart to our students; now we are seen as clerks who dispense grades, certificates and diplomas to customers who are constantly encouraged to express their displeasure……[emphasis added]
The result of the corporatization of the academy has been that contingent faculty are increasingly evaluated on the basis of customer (student) satisfaction. Given the fact that administrators and supervisors rely heavily on ratings by students of contingent faculty in personnel decisions, here’s an article that concludes that such data are not methodologically sound, and that such data ought not be treated as admissible evidence in any legal or quasi-legal hearing related to decisions on the reappointment, pay, merit pay, tenure, or promotion of an individual instructor. This article quotes Mary Beth Ruskai (1996), an associate editor of Notices of The American Mathematical Society:
…Many experienced faculty question the reliability of student evaluations as a measure of teaching effectiveness and worry that they may have counter-productive effects, such as contributing to grade inflation, discouraging innovation, and deterring instructors from challenging students. [emphasis added]
The same article also quotes J.V. Adams (1997):
Teaching, as with art, remains largely a matter of individual judgment. Concerning teaching quality, whose judgment counts? In the case of student judgments, the critical question, of course, is whether students are equipped to judge teaching quality. Are students in their first or second semester of college competent to grade their instructors, especially when college teaching is so different from high school? Are students who are doing poorly in their courses able to objectively judge their instructors? And are students, who are almost universally considered as lacking in critical thinking skills, often by the administrators who rely on student evaluations of faculty, able to critically evaluate their instructors? There is substantial evidence that they are not. [emphasis added]
And here’s another article that not only echos the above, but also cites research that has debunked the reliability and usefulness of student evaluations of teachers. For example, in a major study by Ohio State University in 2007, student reviews were linked to actual learning by examining grades in subsequent classes that would have relied on the learning in the class in which the students’ evaluations were studied. It found absolutely no correlation between student evaluations and actual learning. What the Ohio State researchers did find, as many other studies have found, was clear correlation between the grades the students receive and those they give their professors, providing evidence for the more cynical/realistic interpretation – namely, that professors who are easy graders (and aren’t necessarily the best teachers) earn good ratings. In another finding of concern, the study found evidence that students, controlling for other factors, tend to give lesser evaluations to instructors who are women or who were born outside the United States. And they found this despite not finding any correlation between instructor identity and the level of learning that took place.