New FCEs yield mixed student results

Recent changes to the Faculty Course Evaluation (FCE) process have dramatically changed its purpose and design. The new FCEs are designed to provide better input than the previous edition. As in prior versions, these evaluations are voluntary. A major change, however, is students? new opportunity to fill out the FCEs online.
Response rates, initially a concern, did not appear to differ from the pre-change rates at first. Data is available from two pilot studies and two summer sessions. The online version records a 46.5 percent response rate, while the old pencil-and-paper method had a 52 percent response rate.
As pointed out by the FCE website, the pencil-and-paper method had huge swings in response rates from class to class. The response rates in different courses ranged from 16 percent to 100 percent. Defenders of the online deployment cite studies from other universities that show a rise in response rates as students become more accustomed to answering the FCE online.
These rates are also dependent on completion times. The old FCEs took 2 or 3 minutes to complete. The new FCEs are lengthier, and, as the FCE website states, may take as long as 10 minutes to complete. Similar to the older version, online FCEs will not affect students? grades in any manner. These evaluations are submitted after final grades are entered. While users are required to enter their Andrew IDs and passwords, there are assurances that no personal association is stored with the comments. The submissions of the evaluations are anonymous.
The issue of privacy online versus the more private feel of the pencil-and-paper version is addressed in the FCE policy on privacy. Students are required to log in with their Andrew information to enter the FCE form, but this is simply to make sure students are commenting on courses in which they have participated. As per the FCE privacy policy: ?Once you begin filling out an evaluation, your Andrew ID and any other personally identifying information are discarded. Data collected from your responses does not contain personally identifying information, though some information you provide in your responses could provide potential clues to your instructors.?
The format of the new FCEs hews to the main principles of the former. However there are plans to make sections such as Student Effort & Participation, The Course, and Learning Outcomes customizable. In this case, students would be able to edit items by their elimination or addition. This option is intended to address those questions in the FCE that are not appropriate for particular courses.
The new program is a topic of debate with professors and students alike. Susan Ambrose, CMU?s Associate Provost for Education, spoke of the changes, saying, ?We were quite pleased with the first full implementation ... [There were] not many problems and a pretty decent response rate given that it?s both a new instrument and a new system.? The new system, she says, features easy acccess and makes the results easy to view. Interesting statistics are available immediately: those such as the overall instructor ratings by college are readily available.
Results are available shortly after the final grades are submitted. At that point all results for classes at Carnegie Mellon participating in the program are accessible. These new features are designed to help in reaching the program?s overall goal of enabling teachers to refine their courses in an attempt to better their teaching.
However, user reaction to the new FCEs is split. Paul Day, a junior in mechanical engineering, said: ?I prefer the old FCEs, where we filled out a sheet of paper in class. What?s my motivation to go online to fill out the new ones, especially when they take about 10 minutes apiece? Multiply that by five or six classes for the average student, and you?re asking students for a good chunk of their time.?