The independent news organization of Duke University

Ranking methods under fire

In a move intended to protest college rankings that are often seen as misleading and unfair, Harvard Business School and the University of Pennsylvania's Wharton School announced they would no longer fully cooperate with news media creating such rankings. Duke officials, though similarly scornful of media-generated college rankings, said they had no plans to follow in Harvard and Wharton's footsteps.


 Harvard and Wharton's decision was announced Tuesday after the two schools--both of which have ranked among the top five MBA programs in numerous publications for years--refused to release current and former graduate students' contact information to Business Week for the magazine's biennial survey of MBA programs. The magazine typically uses the contact information to help measure students' levels of satisfaction with their education.


 "The Harvard and Wharton action, and news coverage of it, has certainly sparked a lot of discussion over here [at the Fuqua School of Business], but at this time we are not considering doing the same," said Jim Gray, associate dean of marketing and communications at Fuqua.

Gray said the prevailing view at Fuqua is that the rankings will continue to exist, whether or not the school's administrators are content with the rankings methodology used by the news media.


 "They're still there, and they're still credible with some of the people who are thinking of going to business school," he said. "You have to remember that the real purpose of these rankings is to give prospective students good information on which they can act."

David Lampe, a Harvard spokesperson, told The Chronicle of Higher Education that the business school's decision was not meant to limit the information available to prospective students. "Our interest is not in restricting information, but in improving the usefulness and transparency of that information," he said. "The media haven't paid particular attention to the rigor of their method or the real needs of the students."


 Both Harvard and Wharton will continue to provide basic data, like class size, to news media.


 Discussions about college rankings are by no means limited to the nation's business schools. Christoph Guttentag, Duke's director of undergraduate admissions, said he has long been concerned about the rankings systems used by popular news media.


 "Rankings are inappropriately reductive in the sense that they take these large, complex, multifaceted, outstanding institutions and reduce all of those qualities to a single number that eliminates any sense of richness of an institution," Guttentag said. "They appear to be objective when in fact they're not."


 One of the biggest methodological flaws in current college rankings systems, he added, was that the organizations that publish the rankings determine for themselves which factors matter and how much weight each factor is given.


 "I have thought for a long time that companies like U.S. News and World Report would do everyone a service by allowing students to create their own rankings, by allowing each student to weight each factor whichever way they wanted to," Guttentag said. "I'm not saying the information is inaccurate, but what the organization thinks counts does not necessarily matter to individual students at all."


 Karen Kemp, senior public relations specialist for the Sanford Institute of Public Policy, said administrators at Sanford have thought long and hard about effective ways to encourage organizations that rank schools to rethink their methods. Sanford administrators have enlisted the support of professional groups like the Association for Public Policy Analysis and Management and the National Association of Schools of Public Affairs and Administration in expressing their concerns to the news media.


 Kemp noted that no one at Sanford has suggested the school withhold information like Harvard and Wharton.


 "We've used what you might call other channels to communicate about our desire to see these numbers looked at closely," she said. "Frankly, it hasn't had much effect. U.S. News and World Report this year did their rankings the same way they did them in 2001."


 Guttentag noted that there is some merit in the rankings produced by news media, despite the methodological flaws in the ranking systems.

"In compiling in one place the public information that allows people to compare one institution against another on the basis of objective information, I don't think we can fault a magazine for that," he said, noting that students and their families are often overwhelmed by the surfeit of data available on each school.


 Kemp added that rankings keep schools on their toes.


 "They remind all of us that we're always being evaluated and that we always need to strive for excellence," she said. "We would do that anyway, but it doesn't hurt to have an external reminder that it is important for us to be constantly evaluating the effectiveness of our programs and the quality of teaching and research that happens in those programs."


 In a statement released online in response to Harvard and Wharton's decision, Business Week said its biennial rankings help students make informed decisions by providing objective, unfiltered information about each school.


 "[W]e believe prospective MBAs, current students and alumni have a strong need for the independent information gleaned from our surveys--including crucial details about student experience from those who have just completed the degree," the statement read. "Just as investors today are clamoring for more transparency on the part of companies, so should students expect a similar degree of openness and cooperation from the very schools that nurture new business leaders."


 The magazine noted that it will continue with its survey for the 2004 MBA rankings, which will be published next fall.


Share and discuss “Ranking methods under fire” on social media.