We have just witnessed a live demonstration that the National Law Aptitude Test (NLAT), the online home-proctored admissions test announced eight days ago by NLSIU Bangalore that has taken place today, is impossible to conduct without significant risk of cheating.

We can confirm that it will be all but impossible for NLSIU to eliminate attempts to gain unfair advantages on the exam, even if only needing to weed out the dishonest from the top 80 candidates out of the up to 27,500 candidates (or at least those who managed to log in).

NLSIU, for one, when asked whether it could identify the candidate, has commented that they would be disqualified (see below). However, this seems unlikely based on the technology that was in play today, which did not include the full software suite normally offered by NLS’ vendor.

The final slot of the NLAT that started at 4pm has just concluded, as at the time of publication.

Step 1: How to Zoom conference, NLAT style

Shortly after 2pm, several minutes before the second slot of the NLAT was due to start, we were sent an invite to a Zoom video chat room.

When we followed the link and joined the room, we began witnessing one candidate’s entire 45-minute NLAT exam session live via Zoom’s screen sharing feature, including with repeated attempts at the initial verification of their face and photo ID, and the minutes spent waiting for each verification / rejection to take place.

At no point did the proctoring system flag anything (except for some of the sorts of messages that a lot of candidates received, such as ‘excessive background noise’, please make sure your face is fully visible on screen, etc; it is conceivable the sensitivity of the AI proctoring system was ramped up today, resulting in a lot of false positives).

We watched the entire 45 minute exam, until its submission.

Step 2: ???

Once someone else has access to your screen (and there are many different ways of doing that, besides just Zoom, as outlined in our earlier article), it is pretty much game over, as far as the exam’s sanctity is concerned.

If you are determined enough, you can enlist a committee of friends or even experts, with ready access to Google, calculators and years of experience of cracking entrance tests and reading comprehension questions.

Then, there are thousands of different and obvious ways of communicating the correct answers to whatever question is currently on screen. These include the very easy (but a tiny bit risky) method (which has been demonstrated on Twitter with chopsticks) of just sticking a mobile phone in front of or behind your laptop screen, and then receiving (silent) text chat message updates with the correct answers from your expert counterparties. Conceivably, the NLAT can try to argue that it might be possible to detect hints of this happening with perfect eye-tracking technology from low-resolution webcam footage, unlikely as that seems.

Even safer (but slightly more complicated) would be electronic buzzers or mobile phones in your pocket that can signal you via vibrations, morse code or other agreed-upon signals, which one is the right answer. Even a human super proctor physically standing in your room might not necessarily be able to detect something like that.

Alternatively, in perhaps the most low-tech approach of all for this kind of online exam: a very smart person with a laptop watching your livestream sits under your desk and taps you on the leg in code to indicate the right answers.

Step 3: Profit / loss

While we only saw the first step demonstrated, the second step is trivial to implement for any candidate seriously looking to extract a massive advantage over other candidates.

This creates a huge problem for the sanctity of the NLAT and it does not appear there are any realistic solutions to stop this.

An issue of software

We asked NLS official for comment and to identify the candidate who had shared their screen with us via Zoom.

A member of NLS’ administration commented: “All candidates actions are logged. This candidate will be disqualified.”

The problem is, it may literally be impossible for the NLAT to do so with the technology it employed.

According to IT experts, we understand that modern web browsers intend to make it impossible for software or web pages running in a tab (such as the NLAT online exam), to get information about any other software or processes that are running on a computer, including Zoom or Skype or any other of myriad other screen sharing apps.

The only way for a website to find out which software you are running on your computer would be if it exploited a so-called zero-day security bug in the browser.

Modern browsers, such as Google Chrome or Mozilla Firefox or Apple’s Safari, use sophisticated and advanced techniques known by terms such as sandboxing, to ensure no nefarious website can find out what websites are open in another tab on your browser, let alone what else is running on your computer right now. or which documents you are editing in your Word Processor or what emails you are reading.

If it was possible for the NLAT to find out that Zoom was running, the vendors would be sitting on a bug that, if disclosed to browser vendors, could net you lakhs to potentially millions of dollars, or even more on cybercrime black markets.

It therefore seems highly likely that the audit logs will be limited in usefulness at present.

We have also reached out to Aon Cocubes for comment on the above.

Why is Zoom even possible?

However, most proctored exams would not have allowed you to run software such as Zoom in the background.

As we had already reported in our post-mock exam analysis of the risk of cheating, the NLAT problem boils down to software.

From our LSAT-India interview from July, about its home-proctored test, we asked about the need to download a separate piece of software on candidates’ computers, other than just a web browser window:

the software developed by LSAC partner and international testing multinational Pearson Vue, has a few tricks up its sleeve.

First of all, it was properly compatible only with Windows operating systems (OS), which means no dice for those on Macs (and probably none for Linux or other niche OS users either, though Abdul-Kareem was not able to confirm that for sure).

The programme running on candidates computers, will then take complete control of the system, much like an anti-virus programme (or indeed, a virus) might, and shut down everything else running on the computer, so you are left with only the test window.

Initially, the NLAT too had required candidates to download a separate Google Chrome browser extension, called Safe Assessment Browser (SAB) Tool.

This SAB tool would probably have given the NLAT much wider access to what’s going on in your computer, including presumably detecting that Zoom is running in the background and streaming your entire session.

However, as suspected, yesterday’s FAQ update confirmed that the SAB tool, which some candidates were looking for in vain, was “no longer required for the exam”.

This was most likely a consequence to NLS’ admirable aim of making the NLAT accessible to a wider variety of operating systems (OS) than just Windows, which happened only four days before the exam when the exam was ported to Android, MacOS and Linux, which appears to have been done by simply running the exam in the web-browser, without the SAB tool software.

But compared to other admissions tests such as the LSAT-India or the SLAT (which also forced you to install a software tool on your computer), the lack of such a software will give NLS and its tech vendor, Aon-owned company Cocubes, a very limited data trail to do a forensic analysis on about what was happening on a candidate’s laptop, whether using machine learning or not.

Is there any way NLAT could still catch a 'good' cheater?

As far as we can see, the only possible way that remains of identifying candidates who managed to cheat successfully, would be to disqualify candidates who performed too well.

If someone gets 100% on the NLAT, it’s a good sign that they cheated, particularly if they get 100% in sections such as general knowledge or maths.

On the other hand, seeing as the exam is only 40 questions over 45 minutes, how can you say for sure, on the basis of the scores alone, that you are not dealing with a genius who got lucky?

If the NLAT will have to begin taking calls on a case-by-case basis on whether the 2020 batch of NLSIU will consist of cheaters or geniuses, the entire exercise will be in trouble.