Outlook India defended itself against the criticisms in last year’s complaint by NUJS Kolkata professor Shamnad Basheer and two students against itself and India Today for publishing allegedly error-riddled and misleading law school rankings. Outlook revealed the details of its complex weighting system of ranking but declined to publish further information, in what the NUJS complainants called a ‘lackadaisical manner’.
In August 2010 Basheer and NUJS third-year students Shambo Nandy and Debanshu Khettry had written a complaint to Outlook India and India Today criticising their law school rankings, which they alleged suffered from “gross inaccuracies and methodological flaws” that violated “canons of journalistic ethics” and did a great disservice to students.
Outlook replied on 1 September 2010 [download letter here] and revealed that it had outsourced the survey to consultancy MDRA, which had attached replies on the magazine’s behalf.
The NUJS professor and students replied in a letter [download letter here] dated 14 January 2011: “We note in particular that MDRA has responded in a lackadaisical manner to some of our queries. We once again reiterate that the survey was commissioned by you and published by you in your own magazine. And we expect that being a responsible publisher, you will take responsibility for this, whether done through one agency or the other. Therefore, we will treat MDRA’s response as your response.”
Outlook response
MDRA denied that advertisement made any difference in the rankings. “Re your observation about the ‘influence’ of advertisements, rest assured that has NO influence on the rankings or our stories,” it wrote.
MDRA declined to disclose the original data provided by colleges for the survey citing the colleges’ “privacy of data” and also declined to publish the details of the experts in the magazine due to “available publication space”.
Basheer and the students contested the privacy defence in their response: “We wish to bring to your notice the fact that there was no privacy clause in the questionnaire that you sent to colleges, including WB NUJS, the college to which we (the undersigned) belong. Therefore, there was no expectation of privacy. In fact, the introduction to the questionnaire clearly stated that: ‘The findings of the survey will be published in weekly newsmagazine Outlook.’”
MDRA did, however, set out the weighting of each category used in the magazine’s rankings (see tables, l. & r.). Basheer and the NUJS students criticised several elements of these, particularly why “age of the institute” was “so important that it carries almost four times the weightage given to ‘Quality of Permanent faculty”, more than five times the weightage given to ‘Low attrition rate of Faculty’ and almost seven times the weightage given to ‘Publication of research papers and books by Faculty’”.
The NUJS response continued: “In fact, the combined weightage given to [those three criteria] is less than 60% of the weightage given to ‘How old is the institute’! This effectively means that merely by virtue of being an older institute (though low in academic quality and research potential), one can still be marked to be a much better college than the ones having better quality of permanent faculty and a better intake of students.”
The letter also criticised that three-and-a-half times as many points were allotted to ‘Number of applications received to selection ratio’ (59.2 points) than ‘Student-faculty ratio’ (17.2 points). The former was doubly unrealistic since it disadvantaged larger colleges under the CLAT. “(E.g. GNLU picks 160 students and NUJS picks around 125, whereas NLS Bangalore picks only 80 and NALSAR, Hyderabad picks 70 through CLAT) then under your point system, the marks allotted to GNLU and NUJS will reduce in comparison to NLS or even NALSAR merely because it takes more students and therefore its application to intake ratio is lower than the other college.”
The NUJS letter argued that in general the ‘selection process’ of an institute, which had only four sub-categories, weighed in at 287.5 marks, as compared to ‘academic excellence’, which had 12 sub-categories but only carried 200 marks.
NLIU v NUJS: fractional difference
In 2010’s Outlook rankings, NUJS had dropped from second place in 2009 to fifth place in 2010, just behind NLIU Bhopal, ILS Pune and Nalsar Hyderabad.
In its response MDRA revealed that NLIU Bhopal and NUJS Kolkata were actually tied in their scores in last year’s rankings, which was due to the rounding off of scores. “As we can see that the sub-parameters scores when rounded off to one decimal place, yield NLIU's total of 779.4 as well as 779.4 for NUJS, however, the actual total when sub-parameters scores are rounded off to three decimal places give a sum of 779.514 for NLIU and 779.374 for NUJS,” wrote MDRA (see table left).
Basheer and the NUJS students concluded: “May we please reiterate that rankings are serious issues and students often rely on them? Anyone undertaking this exercise must be sensitive to the impact it has on students, who may end up making a wrong choice, if the ranking methodology is shoddy and yields obviously false results. And the nature of this moral responsibility only increases when the rankings are done by a magazine such as yours that stands for openness and the truth. As we have demonstrated, your ranking this year suffers from serious lapses and methodological flaws. This has no doubt impacted the future of many who may have relied on your rankings. We therefore call upon you to immediately take steps to nuance your methodology and make a better effort next year. Our interest is that you become more evolved and better in your approach, as we think that although rankings have inherent flaws, they still serve a useful purpose on the balance. We hope that you are able to incorporate some of these suggestions as you work towards a more robust methodology in future.”
Rival rankings
In December 2010 Lawyers Update magazine compiled its own ranking, also using data from Legally India’s Mooting Premier League to rank colleges. NLSIU Bangalore topped the ranking for the five-year LLB, followed in the top 10 by NUJS, Nalsar Hyderabad, NLU Jodhpur, NLIU Bhopal, Symbiosis Pune, GNLU Gandhinagar, HNLU Raipur, NLU Delhi and GLC Mumbai.
That ranking put NUJS at first place in law school academics followed by NLSIU and Jindal Global Law School (JGLS Sonepat) in third place, just ahead of Nalsar.
NLSIU led in the three categories of student quality & achievements, infrastructure & library and placements & alumni network. Nalsar was ranked second in all those three categories and NUJS came third in all three.
threads most popular
thread most upvoted
comment newest
first oldest
first
Please go to court guys!
Mr. Basheer's point that the ranking weighage system is foolish is well made and the wags at outlook should spend less time jockeying for soundbytes and focus on preparation of a ranking system that at least looks reasonable.
These magazines (predictably) have a journalistic tendency of shuffling the top 10 ranks each year so as to print large headlines and increase circulation (after all who would be interested in reading a survey once it were established that a few schools were consistently in the top 5 or top 10?), yet do not stop to wonder how silly their own ranking must be if reputed law schools shift 4-8 places within one academic year.
I know from experience that students scoring well in the CLAT prefer NLSIU, NUJS, NALSAR and NLIU well ahead of the rest (though not necessaily in that order) and nearly all those seeking admission at ILS (ranked 3rd??) have been unsuccessful at CLAT. Outlook should wonder what drives such decisions instead of printing a foolish set of rankings.
I used to read a bit about rankings before I entered law school in the US (JD) and it's common knowledge that american law school rankings have been nearly the same for years (if not decades). There would be a scandal (and perhaps a official inquiry!) if Virginia Law were to be ranked over Yale, Harvard and Columbia - as India Today Ranked ILS over NLSIU a few years ago.
A comparison of the ranking system will easily demonstrate that Indian newsmagazines are even more behind their western counterparts than indian law schools are behind theirs. Opaque factors such as 'perception', etc. (used by both Outlook and India Today) would get an editor fired in New York as would providing the ridiculous weightage of longetivity, number of applications, etc. over quality of faculty and number of research papers published by them, as well as placement success. Another case for illustration is Stanford Law that is very close to Harvard (and the other ivy leagues) in the rankings based entirely on its phenomenonal faculty and quality of research despite being established more than 80 years after Harvard. By Outlook's system, SLS would not even feature in the top 50 law schools!
Though a lawyer, I have been in banking advisory for nearly ten years now and having seen, hired and worked with law graduates from all these institutions I can safely say that by putting ILS before NALSAR or NUJS only serves to discredit Outlook.
I wish Mr. Basheer and his students the best.
[We will be moderating comments more tightly on this thread, which has turned into a bit of a bout of personal attacks. If you wish to make points of legal arguments, fair enough, but please keep this from deteriorating into a mud slinging match. -Ed]
Moderating comments and threads is exceedingly rare and in the case of this story and the other Spicy IP story became necessary because things had become too personal.
If your comment was attacking someone personally, then maybe it was moderated and for good reason. Opinion is fine, being mean to named individuals anonymously and without a purpose is another matter.
I have only once moderated a personal attack/criticism against me when it really went too far.
Apart from that, you are free to say what you like, I am sure opinions far more offensive than yours have been posted on the site in the past too.
I hope this makes sense.
Best regards
Kian
Such hypocrisy and double standards.
I know from a modestly reliable source that the 'top 10' college rankings is purely focussed on milking as much revenue from ads and 'gratuity' from featured colleges. Objectiveness is the last thought in their minds.
After all when these private law colleges such as amity, symbiosis, etc. spend so much money on advertising, is it not reasonable that they should get a good rank?? What has NUJS done to deserve objectiveness? Has it placed even one ad in the last decade ??
threads most popular
thread most upvoted
comment newest
first oldest
first