Will the comments ever come down?

Sheesh, I taught for a few years and none of my students bothered to rate me on the site, though my student evals were quite good. Then suddenly, a few students (or maybe it was just one?) decided to gang up on me and said mean things that weren’t even true in some cases and didn’t reflect my ratings in my evals.

So okay. The thing is, I haven’t taught now for three years, but if one googles my name, there are those mean comments, right there for anyone to see. If I apply for a job now and my prospective employer googles me, guess what? How long will these mean-spirited comments remain there for anyone to google? I don’t know what to do. . . Should I change my name?

usedtoadjunct, at 5:20 am EST on January 23, 2008

Course Evaluations

Some universities already post student course evaluations on public websites, so that students and others can review them. I know that my own institution, Florida State University, has done this for years.

Bruce Thyer, at 7:25 am EST on January 31, 2006

RMP is hideous. My scores on RMP were FAR lower than I receieved on my official evaluations. Since I taught large lectures, and the major was popular frat and sorority students, I suspect I was probably the victim of a student or group of students who could easily round up friends to log on and bombard me. But the problem could be attributed to any number of things, because RMP is incapable of determining if a rating comes from someone who was even at the rated profs. university (I’ve been asked to help boost the ratings of a colleague of a colleague). All it requires is that a rating come from a computer with an IP address different from any previously used to rate a particular professor—something that anyone with access to a whole lab of computers can easily exploit single-handedly. Anyone who says differently is just displaying a vested interest or incompetence regarding computer/internet issues and/or data collection and analysis issues.

So, I applaud any suggestion for driving out the bad data provided by RMP. However, the author wrongly assumes that most universities have good evaluation data. Even if one grants that all universities have good evaluation questionnaires (I’ve seen my fair share of bad ones produced by folks whose supposed expertise was quantitative analysis). However, raw data produced by even the best questionnaire is NOT necessarily “good” if not appropriately normed in terms of such factors as size of class, grade expectations of students, motivation of students, student reactance to class (as affected by whether a course is required, a weed-out course, etc.) I have NEVER seen or heard of a university ever norming its data in such a way. That’s because administrators at every level have a vested interest in possessing numbers that they can make mean anything they want them to mean at tenure or promotion time, and that interest is not likely to stop any time soon.

ex-prof, at 10:05 am EST on January 31, 2006

Three cheers to Daryl evaluations for the most part are a total waste and a further promote the idea that an education is same things as buying cereal in supermarket.

Also, more important: I have been to two academic conferences within the year (academic year 2005-06) where colleagues were running tenure-track job searches (political science) and when I made recommendations regarding two individuals who I thought might be a good fit for both jobs, I received subsequent emails that,"after having checked RMP” (talk about unprofessional behavior!!!) there were “concerns” whether either of the recommended colleagues could teach in liberal arts enviornment. Clearly RMP is being looked at by folks on search committees. Don’t believe for a minute that after having looked at RMP folks are not influenced by what they read. And don’t believe that search committee members are not going directly to RMP to, as I was told, “a snapshot” of job candidates. AAUP and the national associations for the various disciplines ought to step in on this debate and come down clearly on RMP and its use in job searches etc.

Best, Andrew

andrew, at 11:31 am EST on January 31, 2006

A Little Grain of Truth in Every Evaluation

Though the whole truth about some individual professor may be something beyond the reach of RMP (or of any system of evaluation), I have a hard time believing that there is not even a grain of it in each entry. And I think that applies even in (rare) cases of frat/sorority types ganging up on a professor. Something triggers these responses. We ought to ask ourselves about those triggers. We don’t need to think they are all equally serious or important, but to understand ourselves as instructors we need to understand the full range of influence that we exert. Indeed, I think that goes as far as understanding how “hot” we are in the eyes of our students: Plato, long ago, understood that knowledge and eros are connected (read the Symposium). We should not think that teaching occurs in a vacuum. There are all sorts of friction, and we ought to thank students for helping us see it. Once we see it, we can either try to reduce it, or use it as a teaching tool itself.

Yavo, Adventurer, at 11:45 am EST on January 31, 2006

Wrongo Chuck

Chuck claims that RMP is irrelevant and even that NO evidence can be found to prove the contrary. Like many categorical claims, the latter reaches too far. For but one example, I was asked to boost the ratings of a colleague-of-a-colleague because that poor guy’s department head publicly announced he’d be using RMP ratings as input on tenure and promotion cases. Further, that head not only teaches variable analysis, but publishes articles that use that method as well, so he should be far more aware of potential threats to data validity than scholars who do not conduct quantitative analyses. Sure this is but one example, but I have also listened in on conversations where a group of well-known, quantitatively-oriented Internet researchers discussed RMP without ever once questioning the validity of its ratings, despite how obvious it is that these can be easily manipulated. How could this be? As I said before, incompetence is not the only reason why some folks accept RMP ratings; they may also do so due to vested interests like inflating their ego or defalting the reputation of a competitor. Whatever the case, given the examples above, it seems VERY likely that there are a LOT of people who assume RMPs ratings are valid, and thus legitimate data for judging their colleagues and/or themselves.

ex-prof, at 1:01 pm EST on January 31, 2006

Truth in EVERY evaluation? NOT

Truth in EVERY evaluation? I’ve heard more than one person claim that when confronted with the reality of RMP’s invalidity. I don’t know if those claims were sincere or just the result of embarassment caused when confronted with the undeniable fact that RMP is grossly and obviously invalid. Assuming Yavo is sincere in claiming that EVERY evaluation possesses some truth, what truth was there in the positive rating I provided to the person whose class I had never taken, in order to save him from his department head? Even if we accept that most RMP ratings are the reflective of an actual student’s feelings (even if expressed through an organized recruitment of frat brothers, etc. who may or may not have been in the class) how are we to determine whether the feelings expressed are a truth about one student or many? Of course, we can’t, and if we can’t determine that, we cannot claim that we can derive a “truth” from many, if any, RMP ratings. Indeed, even if we were able to determine that, e.g., 30 negative RMP ratings were part of an organized campaign and thus expressive of the depth of one person, what “truth” does that give us. Perhaps nothing more than that the poor professor in question had the bad luck to have had an incredibly vindictive person in their class who bullied a bunch of other people with easy internet access into doing their bidding. Don’t think these types of people exist? AFTER the end of one semester I lamented to a student in one of my last classes that her classmates hadn’t responded at all to any of my attempts throughout the semester to initiate discussion. At that point she provided the reason: most class members wer in a fraternity or sorority, and one of them was the party coordinator for one of the most popular frats on campus. Since he started the first class by shooting down other class members’ contributions (despite my efforts to stop this), the other members had decided to shut up instead of looking “uncool” and thus potentially being dropped of the perpetrator’s invitation list.

ex-prof, at 1:40 pm EST on January 31, 2006

Faculty Evalustions Are Judgments

Rene Descartes said, “Everyone complains about their memory, but no one about their judgment,” adding, “Good sense is of all things in the world the most equally distributed, for everyone thinks himself so abundantly provided with it that even those most difficult to please in all other matters do not commonly desire more of it than they already possess.” _Discourse on Method_ 1637.

Descartes was not known for his use of irony, but it sure is operative in this passage.

First let me associate myself with the comments of Daryl Close. Student evaluations of faculty are neither useful nor necessary for evaluatiing faculty performance and their use sometimes can be not only misguided but also unjust. Evaluation implies a judgment of value and as a judgment it rests on criteria but more importantly on the knowledge of the person making the judgment. The mission of the college teacher is to educate, not primarily to entertain or to merely train students to be economically successful. I am not demeaning the need for professional training or the value of being economically successful. Nor am I saying there should be no relationship between being educated and being economically successful. Nor am I saying that teachers should never be entertaining. I am saying that being educated is primarily something other than those two laudatory goals and something more than either or both of them. The task of education is to enlighten the student about what is valuable in the past, to help the student develop good judgment about the things they experience and confront in the present, and to open them up new horizons and reveal new possibilities for their future. In a word, a college education is about learning how to live your life intelligently, reasonably, and responsibly to the fullest as an excellent human being: An old fashioned notion forgotten, but not thereby irrelevant. What students say about the college professor who tries to do those things can range anywhere from angry denunciation to ecstatic praise depending on the quality of the judgmment of which the student is capable.

The practical problems arise when student evaluations are used indiscriminately to evaluate faculty performance and that happens more often than administrators like to admit. The RMP is a bad sample of student evaluation of faculty to begin with because it tends to be dominated by the extremes: the disgruntled students who want to punish professors who they did not like for whatever reason; and the students who liked or were inspired by their professors. Add to that the problem a professor might have who teaches in an open admissions college where he has to deal from time to time with students ill prepared for college or for the courses he is supposed to teach. Even standard student evalutions that include all the students who take a professor’s courses suffer from the limitations of the quality of the judgment of the students doing the evaluation. I have been teaching college for more than 35 years and have had students come back to see me after 5 or 10 years and tell me that at the time they did not much like the courses they took with me, but now they realize how valuable they were and how what they learned in them was worth while. Some have even apologized for the bad evaluations they gave me at the time. Life history puts perspective in our judgments.

What other profession judges the quality of their member’s performance by the popularity of the client’s reviews? Is that not the whole idea of a profession: that the professionals are judged, evaluated, by other professionals who are knowledgeable in the field specialty? “Professional standards.” This is the real problem of faculty evalution: When student evaluations are used indiscriminately to determine the quality of the professor’s teaching, the college adminstrators and the senior members of the faculty have abdicated their responsibility for evaluting their faculty fairly either because they take the easy way out (too lazy) since it is a lot of work, or they are just not themselves competent to judge them (not being knowledgeable of the field of study). A fair system of faculty evaluation should be based primarily on:

1. Peer evaluation by the senior members of a department who examine the written and the classroom work of a fellow faculty member,

2. The administrator’s visits to the classroom as well as an examination of the substance and methods used by the faculty member in teaching his or her courses. If this procedure were more than ritually followed, I doubt we would have a large number of incompetent or even unpopular teachers because ironically in the long run most serious students really want to learn in the sense we described above.

When students ask me about how other students have evaluated my courses, I advise them to take any student evaluation of a faculty member with a grain of salt; and if they can, ask the student why they praised or complained about a teacher; but then make up their own minds on the basis of investigating the courses themselves and ultimately on the basis of their own experience.

Frankly, I would prefer not to have students in my classes who are primarily concerned with getting good grades without hard work or without having to learn anything that challenges them. Do such students belong in college? I suppose that the notion that some students just should not be in college is an old fashioned idea whose time may be coming back given the fact that the majority of our graduate students in the challenging disciplines like mathematics, the sciences and engineering are not American students but foreign nationals. People who think about this trend are worried about what this might mean for our future.

Emile Piscitelli, Professor at NVCC, at 3:00 pm EST on January 31, 2006

Rate my professors

Although I have made only one comment on RMP, I looked up all my teachers before choosing classes. Almost every student I have talked to uses either RMP or a similar student input, no university oversight website.

No one likes being told they are not good at what they do. That is the reality of having a service job, however. Start learning to satisfy your consumers or perish like the other industries that thought they were above petty competition and customer service.

Don’t like the service reviews? Improve your service. Simple as that.

Kevin, Undergraduate, at 5:16 pm EST on January 31, 2006

Let’s have objective info online, not evaluations

So, far from publishing course evaluations, we in the academy should begin quietly dismantling this wasteful and pointless exercise

Bravo! I’m not a service worker or a consumer product. If students want to rate me, fine. If colleagues and administrators use this stuff to make decisions about reappointment and tenure that’s objectionable.

What’s remarkable is that with all the subjective student chatter about us in the ether, students don’t have easy access to objective, factual info about our classes, e.g. syllabi and reading lists online, handouts, or other info about the content of courses, requirements, grading policies, etc. Qua academic advisor, I’d like to see complete information for all courses—syllabi, reading lists, handouts, etc. accessible so that students know exactly what they’re getting into. I put all my stuff up, accessible to the whole world—so should every instructor. Then students could make informed decisions.

LogicGuru, at 4:35 am EST on February 1, 2006

The Bankrupt State of the Professoriate

Among all the pitiful weepings and moaning from the big, bad, all-so-powerful RateMyProfessor.com bashers, I have yet to read one single, solitary statement here that actually identifies an actual professor at a real college whose tenure or promotion decision was influenced in any way whatsoever by that hilarious and impertinent web site.

All I see are vague claims, dark innuendoes, sinister allegations, and the same sort of all-consuming obsessions that one associates with Joe McCarthy and the communists or the parish priest and evil sex.

Either provide some clear, verifiable, documented cases or recognize that your lamentations and crying are signs of the thin-skinned, terrified, and irrelevant nature of the American professoriate.

Truly pathetic, especially Mr. Ex-Prof. Good grief what a hot air balloon!

Shannon, at 10:05 pm EST on February 1, 2006

Two reviews of Johnson’s book

http://mtprof.msun.edu/Spr2004/estyrev.html

http://www.ent.ohiou.edu/~manhire/grade/gireview.pdf

Brian Manhire, at 7:20 pm EST on February 2, 2006

Those of you who like Johnson’s book, can you explain this for me? He apparently links lower grades to lower evals, and says that student access to evals creates pressure not to grade hard. (I’m going strictly off of the book reviews.)

But doesn’t that pressure disappear once the department imposes a standard mean and deviation for the grades in the courses? In other words, why deprive professors of vital feedback when there is an obvious cure for the problem of tough graders getting low evals?

Bernardo O’Boyle, at 10:35 pm EST on February 4, 2006

standardizing evals

Bernardo suggests that evaluation ratings could be made more informative if they were normed in relation to grades given. That may be true, but grades aren’t the only signficant factor affecting evaluations ratings—research has shown that method of delivery has a substantial effect on evaluation ratings (hence the Dr. Fox effect and the research showing that students substantially made up their minds about professors within the first 30 seconds of class, even when that span consisted of a sound-less video of the instructor.) Further, as reported in the Chronicle of Higher Ed. a couple years ago, an econ. prof. at UT-Austin found that a very large portion of the variance of student eval.s was explained by the prof.s perceived attractiveness (the chili pepper effect, as it were). (Of course, the latter fact should come as no surprise to anyone who has read any social psychology research.) Indeed, these factors explain so much of the variance that there is little left to make any other kinds of conclusions about professors, making the evaluations essentially useless. And even if evaluations would be useful if standardized, don’t expect administrators to standardize them. First, can you imagine a school going out and collecting data on the perceived attractivenss of its professors? That raises all kinds of problems for institutions trying to be careful about sexual harassment, much less their own perceived laughability. Second, why would administrators deprive themselves of what they now find to be such a useful tool? If they like someone, they just pass over their poor evaluations by saying something like “well, you know there are problems with teaching evaluation validity” but if they don’t like someone they cite the unfortunate soul’s un-standardized scores as though they were truth itself in order to roast him/her. Don’t think this happens? I’ve sat in on meetings where a merit pay committee chair has uttered both sentiments within a span of minutes. I suspect that MANY other folks have had similar experiences. Other reasons for the use of such flawed materials are provided by Michael Huemer at http://home.sprynet.com/~owl1/sef.htm. There, Huemer usefully summarizes much of the research showing the flaws of teaching evaluations.

ex-prof, at 10:36 am EST on February 6, 2006

I loved it when the so-called ex-prof huffed and puffed that “I’ve sat in on meetings where a merit pay committee chair has uttered both sentiments within a span of minutes. I suspect that MANY other folks have had similar experiences.”

I happen to enjoy and appreciate the use of sharp language, but whenever I read the vague blandishments and cryptic assurances of someone like Mr. Ex-Prof who is afraid to actually back up his sinister claims with anything substantive, then I thank God not to work in higher education with people like that.

The RMP website is great fun, light hearted, funny, amusing, insightful and irreverent. I have read absolutely nothing from Mr. Ex-Prof and his clones that confirms with examples that ANY prof was ever terminated or denied a promotion BECAUSE of RMP. There were undoubtedly other more salient reasons behind the actions.

I mean, among the professoriat, just like among skilled professionals everywhere, someone had to finish in the bottom 25% of the Ph.D. programs!!

Chuck, at 5:45 pm EST on February 6, 2006

As a student, I appreciate a ‘heads up’ on the professors that do not care a lick about their students. If you want to fight RMP, step it up a notch and make a difference in the students lives. I have to say, this article, as well as some of these comments, are shocking—I had no idea professors whine about this stuff.

Adam the Student, at 8:55 pm EST on February 7, 2006

In my humble opinion, the initial proposal, namely, that schools release the results of their own internal evaluations, is necessary but not by itself sufficient. It is necessary not just because the data generated by the internal evaluations is vastly superior (many more evaluations, less chance of manipulation, and so on) to the data on RMP. It is correct because students and their families are paying a lot of money for their educations, and we should be providing them with the tools to make intelligent choices about their classes. We are not service providers in the sense of owing students an “A” just because they take our classes, but we are service providers in the sense that we owe our students the best education possible. They can only get the best education possible if we help students to make well-informed choices about their courses.

The reason that colleges have to this point refused to release this information, frankly, is to protect their most poorly regarded teachers, whom they cannot fire and replace. What student is going to want to enroll in the courses of the most poorly regarded professors? How will colleges deal with parents demanding that their children get into the courses of the most highly regarded professors? That will indeed be a problem, but refusing to release information is hardly the right way to deal with it—and, anyway, the cat is already escaping from the bag. Now that RMP is up and running, it is simply a matter of time before colleges have to confront this issue.

If colleges had been releasing this information all along, RMP would never have come into existence.

However, in addition to releasing the results of course evaluations, colleges should also release 1) each professor’s course grades (how many As, Bs, Cs, etc.), and 2) each professor’s syllabi. There is no question that some professors simply give everyone an “A” to curry favor with their students, and that some professors assign little or no work to their students for the same reason. The only way to deal with these unethical practices, really, is through the public shaming that would come with the release of course grades and course syllabi. And, frankly, we’d all be better off for it. The students who want easy As given for no work are welcome to enroll in the courses of our obliging colleagues.

State U. History Prof., at 2:45 pm EST on February 9, 2006

Educating your consumers

1. OK, I’m not in higher education, I’m an “older returning student” working in industry. However, I have been teaching in my field for 30 years, and will soon leave the working laboratory and enter your profession. I have much to learn, and I also have much to bring to the formal classroom. But, since I’m not yet in your profession, you can hereby safely dismiss my comments as those of an “outsider".

2. Clearly, many people are worked up about RMP — whether that is reasonable or not, the very existence of this string of comments shows the stridency of offended humans.

3. Don’t think you’re providing a service to consumers? Prepare to become irrelevant. ALL transactions for a fee MUST contain an element of “value for money". Many of you and your institutions have avoided this for so long you think it can never happen. That attitude has CREATED some of the problems showing up now. Unions, revolutions, and sniping websites gain little ground where The People In Power come to terms with the important issues. You have waved them off for decades, but now these particular chickens have come home as rocs.

4. It simply is not reasonable to dismiss all negative student comments as coming from lazy fools with time on their hands. Some people in your field perform abominably, and should be dismissed. Some are merely uninspiring, and need training or a new field. You should be helping this to occur, or you become part of the problem. YOU should be the ones striving to give validity and value to your profession. And don’t blame your admittedly poor administrators — unless you simply MUST have that job, and then who is clamoring about value?

5. That being said, there is no reason to think that you are powerless to influence the value equation. It is part of your job to educate your students on the value of what you are delivering. You can’t just phone in your standard lecture and call it education. Help them see your passion (if you have any), make them see the value of what you offer for their money (if there is value), and most of them will respond appropriately. But if you claim your status, refuse to listen, deny change, and remain distant, dismissive, and unhelpful, you will join the Romanovs in the forest.

Professor-in-wating, at 5:35 am EST on March 1, 2006

Amazed at the Low Quality of this Discussion

But not surprised, given that the author of the initial article specializes in Fox-worthy oversimplification and obfuscation. (If Dr. Miller is represntative of the educators of the economic elite in this country, is it any wonder we’re getting our asses handed to us in the global market?)

Here’s an attempt to uncloud what “Space Elevator” Miller has beclouded. There are 2 issues involved:

1)RMP provided feedback based on an inadequate sample. I’ve got about 15 ratings out of several thousand students taught (around 3.8, btw, and a significant bit lower than my official evaluations). Compounding that is the phenomenon, well known to behavioral psychologists, of outlier amplification. The better the class for the majority of students, the more incentive there is for the disgruntled student to post a nasty review at RMP (and correspondingly low incentive for the happy students to post). This is all so obvious, it hurts to mention it.

2) Even if one could somehow solve the sample problem by requiring random and heavy participation in RMP, you’d still confront the problem of the consumer model of education. That model may work for standard consumer goods where “the customer knows best” (and actually it doesn’t work too well there, either) but it is completely inadequate in the context of a teacher-student relationship premised on the exact opposite assumption.

Excuse me, I’m running off to soil Dr. Miller’s rep further.

UC Prof, at 5:40 pm EST on March 14, 2006

If you can’t beat ‘em...

I visited RMP for the first time last week, when five of my rhetoric/compositon students told me they took my course because my ratings termed me a “lenient grader.”

I pointed out that my departmental evaluations rate me much differently—I use portfolio grading and demand revision of six writing pieces over the course of the semester. Students label this grading “unfair.” Certainly a different picture than the RMP site gave them.My TWO RMP ratings do indeed note that I am a lenient grader. Dandy. Bring on the students who want an easy ride.

On the other hand, these two ratings (written in 2002) say nothing of substance. I was surprised that the students were so easily convinced to take my course based on such poor rhetoric. All they saw was the work “lenient,” overlooking the grade distribution chart that shows that 70% passed the course with a C or better. Eureka—next semester, I plan to use RateMyProf as an example of public rhetoric. They can learn to critically analyze the data, and draw better conclusions. Perhaps, they will learn to write better ratings!

Marcus, prof, at 11:30 am EDT on April 21, 2006

A Highly Venerated Faculty at our small school

According to our RMP listing, the faculty of our small community college includes Sigmund Freud, Carl Jung, Louis Pasteur, Herbert Hoover, and many other. So far, none has received a bad rating (probably because the really mean students haven’t taken their classes yet).

soc. prof., at 5:40 am EDT on May 13, 2006

Maybe students should be spending more time actually studying and doing coursework, rather than picking on their professors online. Also, if students acted respectful in class, maybe their professors would see that too. Professors have feelings and some of the ratings are unfair. I’m a beautiful, successful, well-spoken woman and I know I’m a good teacher. My RMP ratings are always put on when students get an exam back and are upset that they didn’t do well. I do everything I can to facilitate the education of the students in my class.

I make everything that will be on the exam explicitly clear. I post all the lecture notes online, give practice exams with problems that appear later on the actual exams, and I give extra credit. I hold an optional review session every week, and one right before the exam, and have office hours and appointments with students all the time. I sit at home each evening emailing students back for hours, and checking our classes’ discussion boards. I teach five courses and I don’t do research. I’m not tenure-track. For the amount of time I put in I probably make less than minimum wage. Yet students do not respect me and do not respect themselves by studying. They use RMP and other ways to get back at me. One day in their future they will realize that they only sold themselves short!

G, Dr., at 4:25 am EST on December 4, 2006