The Proctor is In: Assessment Integrity in a Pandemic
Though nearly impossible to quantify, cheating remains a persistent and resilient issue for many institutions. More so than ever before, schools are calling on technology to assist with the new modality reality wrought by COVID-19. Mid-pandemic, we noted a marked increase in the adoption of products designed to uphold academic honesty online.
Eduventures has identified 22 different products we call “Assessment Integrity Solutions” within our 2020 Higher Education Technology Landscape (Landscape). Our analysis has found that about 1,300 institutions across all sectors of U.S. higher education are currently using 2,400 of these implementations. Here are some highlights from our forthcoming report.
Assessment Integrity Solutions
In order to conceptualize the full scope of what we mean by “assessment integrity,” Eduventures has created the Assessment Integrity Lifecycle (Lifecycle) that defines the end-to-end functions required to deter and detect cheating during assessment (Figure 1). Each solution within the Assessment Integrity segment has core functionality that addresses one or more stages of this lifecycle.
In practice, the majority of schools are using proctoring solutions such as those from Respondus, Proctorio, ProctorU, Examity, and Honorlock. This segment also includes plagiarism solutions like Turnitin and SafeAssign, and specialized identity management solutions such as Acxiom.
Figure 1.
As our framework demonstrates, the process of maintaining academic integrity during assessments is not something that starts and stops with the delivery of a test. Rather, it is continuous. It begins before a test with the authentication of the test taker, and continues after the test with aggregation and analysis of the results. This last step, analysis of trends, helps institutions understand the whole picture of their assessment integrity efforts and pinpoint where and how cheating may be occurring.
In order to realize all of the functions within the lifecycle, institutions will often deploy more than one solution. As shown in Figure 2, 42% of institutions use multiple Assessment Integrity Solutions. This is most common among public four-year institutions, which also enroll 52% of all for-credit students.
Figure 2.
Common combinations of solutions used together are, unsurprisingly, proctoring solutions and plagiarism solutions. Many institutions were also simultaneously using multiple solutions that have duplicative functions—a trend that highlights decentralized decision making processes. The majority of schools using a single solution (58%) are likely not using the complete set of functions identified within the Lifecycle. In fact, we found only one vendor, Proctorio, which appears to offer all Lifecycle functions: proctoring, plagiarism detection, and data analysis.
Within the forthcoming report on this segment, we expand on the numerous combinations of technology that institutions are using to prevent cheating, and which specific solutions are being deployed at which types of schools. We also explore some of the special considerations institutions must address.
What are the Considerations for Institutions?
As with any technology, institutional leaders should review Assessment Integrity Solutions not only through the tactical lens of features and functionality, but also through a strategic lens: How does the solution fit within my ecosystem?
When it comes to Assessment Integrity Solutions—and specifically those that record user behavior—there is an additional, and often controversial concern: Will my students view it as a type of surveillance technology? While some may argue that the purpose of using these solutions is far more benign than forms of surveillance, students may nonetheless have a similar response to their use.
Institutions considering deploying these solutions, therefore, should adopt a similar approach as they would with surveillance technologies. For example, they should ask vendors about what data they collect, how they use it, and how they ensure data privacy. Likewise, schools should keep in mind the possibility that students may become anxious about being recorded or watched during test-taking, which could impact student performance on assessments. Students may also object to taking tests in a remotely proctored environment altogether. In some cases, it may be necessary to develop and communicate transparent policies to protect privacy rights or allow students to opt out of proctored examinations in certain scenarios.
The Bottom Line
The growing need to ensure academic honesty during assessments has not been limited to technology considerations. Many schools are also indicating a willingness to change grading practices—in many cases allowing for more pass/fail courses. Others are challenging the value proposition of point-in-time knowledge demonstration altogether, rethinking traditional means of assessment within courses.
For those (re)considering the role of technology in their assessment integrity efforts, we recommend learning about the full suite of technologies offered, and about what combination of functionalities is working well for these efforts at peer institutions.
Never Miss Your Wake-Up Call
Michael Miller
Eduventures Senior Analyst at Encoura
Contact
Thursday, August 13, 2020 at 3PM ET/2PM CT
The annual CHLOE (Changing Landscape of Online Education) survey chronicles the evolution of online policies and practices in U.S. higher education, from the perspective of Chief Online Officers (COOs). The fifth CHLOE survey captures leadership reflections on the pandemic-induced emergency remote instruction in the spring: what happened, what worked, and implications for fall. The CHLOE 5 report is published amid a coronavirus spike that is persuading many colleges and universities to re-think their fall re-opening plans.
This webinar will distill lessons from the spring and, in conversation with a panel of online leaders, gauge early fall realities. Online infrastructure, experience and leadership played a pivotal role in enabling schools to see through the last academic year, but many leaders have been quick to draw a distinction between hastily assembled “remote instruction” and true online learning. As fall 2020 gets underway, and schools face a yet stiffer test, online leaders are stepping up once again. Robust, engaging, high quality remote learning, by whatever name and whether or not combined with a socially distanced campus, will define this one-of-a-kind semester.
Also in Adult Learner Demand
Understanding the Enterprise and Point Solutions Balance
A commonly held belief is that the enterprise solution is more valuable than the point solution. Indeed, multi-purpose solutions, such as the Student Information System (SIS), can allow schools to simplify their ecosystems by addressing many needs with one system—avoiding complex integrations across solutions that address only one requirement.
Technologies that Support the Student Journey: Five Vendors to Watch
As we enter 2023, our conversations with higher education leaders have become increasingly focused on how to best support the student journey: improving support for admissions, bettering onboarding, and streamlining transfer student pathways, for example.
How We Can Improve Technology Selection Through Exploratory Data Analysis
A recent survey by Educause noted that median technology spend among colleges and universities in fiscal year 2021 was $7.7 million, with nearly one-third of respondents saying that they plan to increase this amount in the following year.
Finding Patterns in Teaching and Learning Technology Implementation Trends during COVID-19
Technology implementations related to teaching and learning increased during the pandemic, with a combined annual growth rate of 14% between 2019 and 2021.
