•  •  Dark Mode

Your Interests & Preferences

I am a...

law firm lawyer
in-house company lawyer
litigation lawyer
law student
aspiring student
other

Website Look & Feel

 •  •  Dark Mode
Blog Layout

Save preferences

NUJS v law school rankings: Outlook India stonewalls, reveals some inner workings

Outlook India defended itself against the criticisms in last year’s complaint by NUJS Kolkata professor Shamnad Basheer and two students against itself and India Today for publishing allegedly error-riddled and misleading law school rankings. Outlook revealed the details of its complex weighting system of ranking but declined to publish further information, in what the NUJS complainants called a ‘lackadaisical manner’.

In August 2010 Basheer and NUJS third-year students Shambo Nandy and Debanshu Khettry had written a complaint to Outlook India and India Today criticising their law school rankings, which they alleged suffered from “gross inaccuracies and methodological flaws” that violated “canons of journalistic ethics” and did a great disservice to students.

Outlook replied on 1 September 2010 [download letter here] and revealed that it had outsourced the survey to consultancy MDRA, which had attached replies on the magazine’s behalf.

The NUJS professor and students replied in a letter [download letter here] dated 14 January 2011: “We note in particular that MDRA has responded in a lackadaisical manner to some of our queries. We once again reiterate that the survey was commissioned by you and published by you in your own magazine. And we expect that being a responsible publisher, you will take responsibility for this, whether done through one agency or the other. Therefore, we will treat MDRA’s response as your response.”

Outlook response

MDRA denied that advertisement made any difference in the rankings. “Re your observation about the ‘influence’ of advertisements, rest assured that has NO influence on the rankings or our stories,” it wrote.

MDRA declined to disclose the original data provided by colleges for the survey citing the colleges’ “privacy of data” and also declined to publish the details of the experts in the magazine due to “available publication space”.

Basheer and the students contested the privacy defence in their response: “We wish to bring to your notice the fact that there was no privacy clause in the questionnaire that you sent to colleges, including WB NUJS, the college to which we (the undersigned) belong. Therefore, there was no expectation of privacy. In fact, the introduction to the questionnaire clearly stated that: ‘The findings of the survey will be published in weekly newsmagazine Outlook.’”

image
image
image
image
MDRA did, however, set out the weighting of each category used in the magazine’s rankings (see tables, l. & r.). Basheer and the NUJS students criticised several elements of these, particularly why “age of the institute” was “so important that it carries almost four times the weightage given to ‘Quality of Permanent faculty”, more than five times the weightage given to ‘Low attrition rate of Faculty’ and almost seven times the weightage given to ‘Publication of research papers and books by Faculty’”.

The NUJS response continued: “In fact, the combined weightage given to [those three criteria] is less than 60% of the weightage given to ‘How old is the institute’! This effectively means that merely by virtue of being an older institute (though low in academic quality and research potential), one can still be marked to be a much better college than the ones having better quality of permanent faculty and a better intake of students.”

The letter also criticised that three-and-a-half times as many points were allotted to ‘Number of applications received to selection ratio’ (59.2 points) than ‘Student-faculty ratio’ (17.2 points). The former was doubly unrealistic since it disadvantaged larger colleges under the CLAT. “(E.g. GNLU picks 160 students and NUJS picks around 125, whereas NLS Bangalore picks only 80 and NALSAR, Hyderabad picks 70 through CLAT) then under your point system, the marks allotted to GNLU and NUJS will reduce in comparison to NLS or even NALSAR merely because it takes more students and therefore its application to intake ratio is lower than the other college.”

The NUJS letter argued that in general the ‘selection process’ of an institute, which had only four sub-categories, weighed in at 287.5 marks, as compared to ‘academic excellence’, which had 12 sub-categories but only carried 200 marks.

NLIU v NUJS: fractional difference

In 2010’s Outlook rankings, NUJS had dropped from second place in 2009 to fifth place in 2010, just behind NLIU Bhopal, ILS Pune and Nalsar Hyderabad.

Difference between NLIU and Bhopal
Difference between NLIU and Bhopal
In its response MDRA revealed that NLIU Bhopal and NUJS Kolkata were actually tied in their scores in last year’s rankings, which was due to the rounding off of scores. “As we can see that the sub-parameters scores when rounded off to one decimal place, yield NLIU's total of 779.4 as well as 779.4 for NUJS, however, the actual total when sub-parameters scores are rounded off to three decimal places give a sum of 779.514 for NLIU and 779.374 for NUJS,” wrote MDRA (see table left).

Basheer and the NUJS students concluded: “May we please reiterate that rankings are serious issues and students often rely on them? Anyone undertaking this exercise must be sensitive to the impact it has on students, who may end up making a wrong choice, if the ranking methodology is shoddy and yields obviously false results. And the nature of this moral responsibility only increases when the rankings are done by a magazine such as yours that stands for openness and the truth. As we have demonstrated, your ranking this year suffers from serious lapses and methodological flaws. This has no doubt impacted the future of many who may have relied on your rankings. We therefore call upon you to immediately take steps to nuance your methodology and make a better effort next year. Our interest is that you become more evolved and better in your approach, as we think that although rankings have inherent flaws, they still serve a useful purpose on the balance. We hope that you are able to incorporate some of these suggestions as you work towards a more robust methodology in future.”

Rival rankings

In December 2010 Lawyers Update magazine compiled its own ranking, also using data from Legally India’s Mooting Premier League to rank colleges. NLSIU Bangalore topped the ranking for the five-year LLB, followed in the top 10 by NUJS, Nalsar Hyderabad, NLU Jodhpur, NLIU Bhopal, Symbiosis Pune, GNLU Gandhinagar, HNLU Raipur, NLU Delhi and GLC Mumbai.

That ranking put NUJS at first place in law school academics followed by NLSIU and Jindal Global Law School (JGLS Sonepat) in third place, just ahead of Nalsar.

NLSIU led in the three categories of student quality & achievements, infrastructure & library and placements & alumni network. Nalsar was ranked second in all those three categories and NUJS came third in all three.

Photo by hikingartist.com

Click to show 10 comments
at your own risk
(alt+c)
By reading the comments you agree that they are the (often anonymous) personal views and opinions of readers, which may be biased and unreliable, and for which Legally India therefore has no liability. If you believe a comment is inappropriate, please click 'Report to LI' below the comment and we will review it as soon as practicable.