EdTech and the Changing Role of Course Evaluations

Years ago, Mayor Edward Koch of New York City famously asked New Yorkers outside of the subway stops, “How am I doing?” When asked about this in a 1981 interview, he said, “Some people have said that’s a mark of insecurity. Gee, I have to be patted on the back … I want you to think about this: do you know people in public life who are sufficiently secure to ask people to rate them?”

Higher education has long asked students to provide this type of insight—via course evaluations. More recently, however, the ideal role for these evaluations has come into question, sending course evaluation vendors in several different directions with their products. How is this tension shaping course evaluation technology, and how will you know which solution is the best fit for you?

The Tension Around Course Evaluations

As shown in Figure 1, course evaluations, which have long-fulfilled compliance or human resource requirements, have evolved in recent years to satisfy three other needs: supporting faculty management, informing teaching, and assisting with institutional effectiveness efforts.

Figure 1.

These needs are playing out among colleges and universities as follows:

1. Supporting faculty management. Many institutions, such as New York University and Kean University, conduct course evaluations as part of their accreditation activities. Other universities, such as the University of Oregon and the Community College of Rhode Island, deploy course evaluations to hire or keep faculty or manage faculty promotions.
2. Informing teaching. Although some scholars have recognized the complexity of using course evaluations to improve student outcomes, there is a shift to using these evaluations to do just that. Vanderbilt University encourages its faculty to leverage course evaluations as part of an overall approach to improve teaching. Likewise, the University of Notre Dame deploys evaluations as part of a general move toward teaching self-reflection. Faculty use the evaluation results to explore ways to improve their teaching.
3. Assisting institutional effectiveness efforts. Finally, institutions, such as Appalachian State University and the University of Richmond, have also deployed course evaluations to inform teaching and their overall institutional effectiveness initiatives. For leaders at these institutions, course evaluations are a vital input, along with periodic comprehensive reviews, continuous improvement reports, and institutional strategic plans, to measure their success toward providing value to all stakeholders.

This tension between these different needs has fractured the market for these solutions. For example, the Mentor (Axiom) product focuses on leveraging course evaluation data for faculty management. Other products, such as Anthology’s CourseEval with IDEA or Blue (Explorance), fold in questions and frameworks that allow institutions to collect and analyze data to improve instruction. Finally, solutions like Watermarks’ EvaluationKit or Class Climate (Scantron) support evaluating data to support institutional effectiveness efforts.

With the changing role of course evaluation and its supporting software, it may be daunting to think about the best way to navigate this fluidity and select the best solution. We recommend three considerations to keep in mind when approaching this decision.

1. Decide the purpose of your course evaluations. As mentioned, institutions deploy course evaluations either for compliance, human resource management, or instructional improvement. Before reviewing products in the course evaluation segment, decide which purpose fits your needs.
2. Determine the audience for your course evaluations. The purpose of your course evaluations will determine the audience for the data. For example, should you choose to use data for compliance or human resource management, the data’s audience might be department chairs or hiring committees. Leveraging this data for instructional improvement, however, might require that you bring students, faculty, instructional designers, and others into the mix.
3. Identify the data needed for your course evaluations. At its core, a course evaluation comprises ratings students submit about a course. Depending on the purpose and audience for the data, you may need additional data, such as student performance or attendance. Here, you might need to consider integrating course evaluation data with data from other systems, such as the student information system or learning management system.

The Bottom Line

The role of course evaluations is undergoing a pivot, albeit slowly. What makes this pivot especially interesting is that it signals the increasing role of the student voice in helping institutions understand the value of their teaching.

Those institutions that embrace the pivot toward a better understanding of whether students receive value in the classroom tackle it with humility and clear-eyed steps toward achieving that understanding. Likewise, some vendors have recognized how their tools might provide the most significant support for these institutions. Taken together, institutional and vendor efforts to support course evaluations may go a long way toward helping all of us know the answer to the question: “How am I doing?”