Every year, we (members of my department) turn in four different forms reporting on everything we have done for the past year/two years/six years/career (depending on form). There's the what-have-I-done-for-you-lately form that goes to the next level up, and the teaching report, the scholarship report, and the CV, which remain in the department. Different kinds of work "count" for different amounts of time.
If you're just a member of the faculty, you complain about having to fill out all the different information in different places, and struggle to figure out what goes where, and whether 2003 is six years ago or seven, and then you hand everything in and you're done with it.
I'm on the committee that reads through all these forms and assigns a number to everyone's performance. I also get to (read: have to) read everyone else's student evaluations, which is no more fun than reading my own, and look at syllabi and sometimes other teaching-related materials. If there's time, I might even read some of my colleagues' publications.
Why do I hate reading them?
Well, let's see. Some of my colleagues publish much more than I do, and I am jealous. I wonder where they find the time, and whether they have partners who do a great deal on the home front, or if they live on frozen pizza, and if they can only do this because if you're writing about an author who is still living you don't have to wade through 150 years (or more) of scholarship.
Some of my colleagues publish much less than I do, and I am jealous. I am convinced that they have rich and fulfilling personal lives, involving either warm, close families who eat dinner together, share games nights, and offer great emotional support, or else involving travel to New York every weekend, with walks in the Village, fantastic food, and season tickets to both ABT and NYCB. (Or plug in whatever you'd get season tickets to: those are my picks.)
Some of my colleagues get terrific teaching evaluations from their students, and I am jealous. I wonder if they are simply naturally talented, in which case is it really fair to reward people for personality traits they were born with, when others give exactly the same kinds of assignments and don't get the same results? Some of them, of course, may get good evaluations because they give really easy assignments and good grades, and I wonder if I should do the same (not to game the evaluations, but to buy time for research). Some of my colleagues don't get good evals at all, and I think "There but for the grace of something or other go I." Because these people seem perfectly reasonable, decent, and conscientious to me; why do they make such a different impression on their students?
Some of my colleagues do much more service than I do (hard to believe, but true), and then I feel bad for complaining about the amount that I do. Most of them do about the same amount, but I am convinced that in many cases, they are working on committees or other service work that are much more fun than the ones I'm on. In fact, in at least one case, having been on the other committee before, I know for a fact it's more satisfying (at least for me), and I'd like to get back on it, but there are term limits, so I have to wait for awhile.
Clearly I have a bad case of grass-is-greener syndrome.
Seriously, it's pretty easy to evaluate service, and not too bad to go through scholarship: number of pages is an objective calculation, and while we can argue about the value of various journals, at least circulation figures and acceptance rate are usually available and incontestable. But teaching is a bear. What is good teaching, and how do you tell? We don't test students as freshmen and again as seniors; we have no mechanism to gather their opinions of their education five years after graduation, when they've had time to figure out what stuck with them and what turned out to be useful no matter how irrelevant or hateful it seemed at the time. Is it good teaching to make students happy so they are receptive to new knowledge and willing to attempt new skills? Is it making them revise papers multiple times? Is it rigorous reading and writing assignments, perhaps with workshopping and performance requirements? Can students really evaluate good teaching, or should the committee just look for possible problems like complaints about the time it takes to return papers, a small number of assignments, or consistent professorial lateness? (And even these need to be double-checked: is "a long time" ten days or a month? "He's always late and cancels office hours" sometimes translates to "one week when his whole family was sick he was late twice and cancelled physical office hours but was available via e-mail.")
My questions are rhetorical. Dr Crazy, among others, has discussed teaching the kinds of students we get at regional universities far more thoughtfully and incisively than I think I could. I'm not trying to get into a debate about good teaching practice, but rather to show why it is difficult to demystify the annual evaluation of teaching. Every year, some members of the evaluation committee rotate off and new ones come on. So, though the name is the same, the committee is different; members interpret evidence differently, put more or less stress on student happiness, spend more or less time looking at supporting documents; and the numbers change.
We have to do these evaluations. But I sometimes think the most important element is the self-reflection they induce: what have I done lately? Where is my time going? Are my students well-served by the number of assignments I give? Trying to compare my colleagues' apples, kumquats, quinces and raspberries, and assign a score to such very different fruit, is difficult and frustrating for all concerned. I wish we could just assume everybody is doing the best s/he can, and leave it at that.
UTA: Ink has a good post up about student evaluations.