Students can view ratings for almost 17,000 courses taught since 2004 through the Student Accessible Course Evaluation System, but they might be missing some of the most valuable information.
Although the average course rating out of five on SACES—which corresponds to the qualitative ratings students indicate on ACES—is 4.34 and the average instructor rating is 4.46, professors ultimately decide whether or not their evaluations are available to students to SACES. As a result, the information on SACES is not necessarily representative of all student evaluation submissions and 75 percent of all classes for which ratings are available have instructor ratings of at least 4.22.
“The instructors can decide which course to have put into SACES but cannot select or deselect individual items,” Matt Serra, director of the Arts and Sciences Office of Assessment, wrote in an email April 13.
The "opt-in" system in which professors determine whether or not information becomes available after seeing student-submitted data was last voted on at a meeting of the Arts and Sciences Council in late 2004. A close vote at the meeting rejected a Duke Student Government proposal to link to the independent website www.ratemyprofessors.com.
Although professors can still determine whether or not students see the results, Serra is always provided a report each term with the evaluation results and can request reports on certain faculty when necessary, said Lee Baker, dean of academic affairs for Trinity College of Arts and Sciences. The data is also reviewed as part of an annual faculty review process.
“Each term [Baker] is made aware of those instructors in the top and bottom five percent in both quality of instruction and intellectual stimulation,” he wrote.
Part of the dilemma faculty face when deciding whether or not to make results visible to students is a declining response rate. Since the evaluations have moved online in Fall 2013, one of the most powerful tools for students to impact the administration’s decisions is seemingly not being used as often, said Kelly Alexander, a professor at the Center of Documentary Studies who teaches a highly-rated class about food and culture.
“When [the system] switched to online, less students seem to do it, which is a problem because the more students do it, the more feedback the University gets,” she said.
Although Serra noted that ratings have been consistently high on SACES, another byproduct of the online evaluations is that students appear to feel more secure in presenting their true opinions, which has still caused ratings to dip slightly.
"This is evidenced in the aforementioned decrease in ratings across the board as well as the amount of written feedback that the students are providing," he wrote.
From a student perspective, the obstacles facing the current evaluation system go well beyond rates and transparency, as many find the actual information provided in the evaluations too generic because of the imposed five-point scale.
"I think the [numeric] scales are good, but more comment space would be good as well," said Abby Muehlstein, a freshman. "There's space for commenting on the professor and commenting on the course in general, but there's not space for commenting on how the material flowed or [other specific course details]."
The length of the evaluations also gives some cause for concern. Even beyond taking up a student's time, the sheer number of questions provided could play a role in increasing the bias of the procedure.
"[Course evaluations] are too long, I think everything should be quantifiable in a few questions," said freshman David Yan. "It's like seven pages and I feel like they ask the same questions over and over again."
Regardless of the deficiencies of the current rating system, two notable trends emerge. The first is that intellectual stimulation rating and course difficulty are the two best predictors of both course rating and instructor rating, Serra wrote.
The second is that the departments with the highest rated instructors and courses tend to focus on liberal arts subjects. The Theater Studies, Russian, Dance and English departments have the highest average class ratings of departments with more than 100 classes rated on SACES. Subjects with lower average ratings include the natural sciences, math and engineering. The three departments with the lowest average class ratings out of departments with more than 100 rated classes are Physics, Mechanical Engineering and Engineering.
Directors of Undergraduate Study for the Economics, Mathematics, English and Biology departments declined to provide more complete data or summary statistics on a department-by-department basis.
With these trends and challenges in mind, Serra wrote that several changes, particularly changing from an opt-in to an opt-out process—in which professors would only specifically decide not to display data—could increase the system's transparency and effectiveness.
“A group of student representatives along with a group of faculty and administrators to determine exactly what students would like to see and benefit most from seeing in terms of data [would help the system],” he wrote.
Editor’s Note: The Chronicle has chosen to make only aggregate statistics and highlights from the data analysis publicly available at the request of the Office of Assessment. The data for this piece was collected by Gautam Hathi and students in the "Everything Data" class in the Computer Science department.
Get The Chronicle straight to your inbox
Signup for our weekly newsletter. Cancel at any time.