As the number of students enrolled in online courses continues to increase—28% of all students take at least one online class—so, too, does the need to protect the integrity of those courses. Either recognizing that cheating is a real issue on their campuses or feeling pressure from outside accreditors to demonstrate that it is a high priority, a growing number of institutions are seeking out technology to help. Given that a majority of cheating in online courses typically occurs at the bookends of a student’s academic career, either during large general education courses or during high stakes exams required for graduation, online proctoring is seen as the obvious technology solution to deter it.
In a previous Wake-Up Call, Eduventures initially predicted that the number of institutions using online proctoring would grow from 1,000 to 2,000 by the end of 2016. Based on our conversations with institutions evaluating these products, we believe the market is well on its way to exceeding our initial prediction. The most common questions we get from our clients as they evaluate online proctoring solutions are “Does it really work?” and “How much can we expect our cheating problem to be reduced if we use [a particular vendor’s] products and services?”
Having dug deeply into this topic over the last year, we’ve found limited data to definitively answer these questions. Most institutions do not collect data on cheating, and if they do, it is based on self-reported student surveys that are highly unreliable. Likewise, while online proctoring vendors themselves may have data to demonstrate the incidents of cheating from their current clients, they do not have access to pre-implementation data to make a comparison.
We do know that regardless of the ability of any tool to catch an instance of cheating, actual cheating will decline in the short term due to students’ belief that they are being watched or that the technology will catch them if they try to cheat. This initial decline in cheating is known as the placebo effect. Similar to the placebo effect seen in medical science, if students taking online exams are informed that a sophisticated technology is watching their every action during an exam, the likelihood of those students to cheat drops precipitously.
Many proctoring vendors market the overall efficacy of their products when in reality they are including (and greatly benefiting from) this placebo effect. Other vendors leverage the placebo effect to reduce the costs or complexity of their platform. Some vendors only analyze a portion of recorded exam sessions leaving students to wonder if they are being monitored at all. Other vendors, such as Software Secure, ensure that 100% of their recorded sessions are reviewed by a human proctor.
Students might be tempted to roll the dice with fully automated proctoring software, thinking that they can fool the algorithm and get away with certain types of cheating. This isn’t wise, as vendors such as ProctorFree, are still using human proctors to audit the results of their automated solutions. In fact, ProctorFree is so committed to improving the quality of their algorithm they are now auditing 100% of their automated exam sessions using real people. This has the effect of proctoring every session twice – not an environment where students should press their luck by testing the system.
In some cases, online programs have reported a drop in cheating from nearly 20% of all students in a general education math class down to less than 2% after the first academic term when using a new proctoring solution. They find, however, that this placebo effect doesn’t last forever. After the initial term, students might begin to “test” the system by searching for ways to cheat the new technology.
If the institution is using a solution that only samples a portion of its exams, this cheating may go undetected. Word then travels fast within students’ social networks and cheating begins to rise again. When cheating is finally detected by the system, the actual rate of cheating could be considerably higher. Institutions using a 100%-reviewed solution should conceivably be catching every instance of cheating if the service performs as advertised.
First, Establish a Baseline
In order to measure the effectiveness of any given online proctoring solution, institutions must first establish a baseline of cheating before implementation. This baseline provides institutions with something to measure the implemented proctoring solution against and can be a critical tool for holding vendors accountable for the efficacy of their products. Yet, rarely do institutions collect longitudinal data on the rate at which cheating occurs in their classes or programs. Often any record keeping of cheating incidents only reflects those who have been caught, and the actual numbers are usually quite higher.
Prior to implementing a proctoring solution, there are several steps institutions can take to collect baseline information:
- Within the context of an honor code, create a policy for the reporting of incidents of cheating to a database managed by a centralized office such as institutional research.
- Ensure that additional information is known about each incident of cheating, such as the course, instructor, student, modality of instruction, and type of incident that occurred.
- Share information on which courses and programs suffer from cheating the most with deans, faculty, and staff.
In order to maximize and capitalize on this placebo effect and to ensure the long-term viability of its online proctoring implementation, institutions should be vocal with their student body and faculty to show that the online proctoring solution is doing its job. Follow these guidelines when sharing information about integrity solutions with your students:
- Communicate the technical capabilities of the solution to students during its launch. Emphasize that all students are subject to monitoring when using the service.
- Post notices that include institutional policy and quotes from the honor code when logging into the learning management system and when launching proctored exams.
- As the new system detects incidents of cheating, share this information with faculty, staff, and students in accordance with the honor code and related policies.
- Make integrity protection part of the culture of online learning at your institution.
[Editor’s note: This post’s content was updated on 10/5/16 to correct the reporting on ProctorFree’s auditing of exam sessions.]