All higher education leaders face a critical challenge: determining the right technology tools to support the student journey. They strive to create a harmonious edtech ecosystem, selecting and arranging these tools to address various needs.
Solutions like Learning Management Systems (LMS) and Student Information Systems (SIS) commonly form the backbone of these ecosystems. But do these solutions truly capture and track all the data elements related to student learning? Is there a better approach to support teaching and learning?
In this post, we explore an alternative to the "solution-first" approach, offering insights on how to better understand student achievement and improve educational support.
Context
When building an effective teaching and learning ecosystem (see those of Educause and McGill University, for example), various views converge on three key components:
- Technology elements: such as LMS and Digital Courseware Solutions
- Stakeholders: like students and faculty
- Interoperability: for seamless data sharing between systems
Relying solely on these elements, however, doesn't necessarily ensure a comprehensive understanding of student progress or the optimization of teaching and learning. Even thoughtfully designed ecosystems may still lack insights into crucial factors, like the correlation between course delivery and student grades.
To address this example, we took two initial steps:
- Identifying essential data categories using established approaches in course delivery
- Mapping these data categories to potential technology solutions for data collection
Table 1 summarizes the results of this exercise.
Essential Data Categories Used for Optimizing Teaching and Learning Ecosystems
| Essential Data Category* | Definition | Examples of Related Solutions |
|---|---|---|
| Assessment Resource | Type of assessment, such as quizzes or projects, that instructors use to evaluate student progress |
|
| Assessment Design | An assessment plan developed by the instructor, either for a program or course, that comprises the strategies, instruments, and assessment criteria |
|
| Evaluation Resource | Feedback provided to institutional leaders about the instruction, such as student evaluations of courses or programs |
|
| Learner Information | Information about students, such as names, educational status (full-time, part-time, etc.) |
|
| Learning Content | Any digital and non-digital material involved in teaching and learning, such as simple web pages, lecture slides, a textbook, and online course content |
|
| Learning Opportunity | Educational activities of students, such as online learning modules, face-to-face courses, group projects, etc. |
|
| Learning Outcome | Refers to statements of what a learner knows, understands, and can do, such as credentials |
|
| Personal Achievement Profile | Collection of student achievements, such as degrees, awards, etc. |
|
| Teaching Method | Principles and methods instructors use to deliver teaching, including flipped learning, hybrid learning, etc. |
|
Table 1.
We then took the nine essential data categories we identified in the left-hand column of Figure 1 (assessment resource, assessment design, evaluation resource, learner information, learning content, learning opportunity, learning outcome, personal achievement profile, and teaching method) and looked at the percentage of schools in various sectors that use them (Figure 2 below).
Here are three findings of this analysis:
- High overall capacity to track required data: As the first column in Figure 2 indicates, many institutions follow various data categories in their ecosystems. For example, all institutions can track learning information and teaching methods, and most can collect data related to learning content, personal achievement profiles, and learning opportunities.
- Overlooking course evaluation and feedback: Regarding data related to evaluations, institutions in most sectors overlook the tracking of evaluation resource data in their ecosystems, with only public four-year, public two-year, and private not-for-profit four-year institutions using this capacity.
- Mixed capacity to track assessment design: While 72% of institutions collect and track data related to assessment design in their ecosystems, these institutions are only in four sectors. For example, 96% of public four-year institutions and 91% percent of public two-year institutions have this capacity. Notably, of two-year institutions, only public institutions choose to track and collect this data in their ecosystems.
Implementations by Data Element Category and Institutional Sector
| Category | Overall | Private for profit, 2-year | Private for profit, 4-year | Public, 2-year | Private not-for-profit, 4-year | Private not-for-profit, 2-year | Private not-for-profit, 2-year |
|---|---|---|---|---|---|---|---|
| Assessment Resource | 93% | 74% | 90% | 100% | 99% | 96% | 78% |
| Assessment Design | 72% | 0% | 50% | 96% | 91% | 75% | 0% |
| Evaluation Resource | 63% | 0% | 0% | 92% | 83% | 66% | 0% |
| Learner Information | 100% | 100% | 100% | 100% | 100% | 100% | 100% |
| Learning Content | 97% | 93% | 99% | 100% | 100% | 99% | 97% |
| Learning Opportunity | 97% | 93% | 99% | 100% | 100% | 99% | 97% |
| Learning Outcome | 95% | 71% | 69% | 98% | 94% | 85% | 70% |
| Personal Achievement Profile | 95% | 91% | 93% | 99% | 98% | 96% | 92% |
| Teaching Method | 100% | 100% | 100% | 100% | 100% | 100% | 100% |
Table 2.
Source: ListEdTech
What can we learn from this analysis? First, it raises the question of why institutions fail to track all data types within their ecosystems. Perhaps some data is tracked locally instead of within the ecosystem, possibly due to instructors wanting to handle certain assessment data, like quizzes, themselves. Additionally, some institutions may deem specific categories, such as course evaluations, as irrelevant for improving teaching and learning.
Our take: Gathering data within ecosystems is preferable as the "solution-first" approach leads to gaps. For instance, if leaders desire detailed insights into student performance inputs and outputs, they need comprehensive data integration from various systems. Therefore, these leaders should organize their ecosystems based on the data and information provided, rather than the functionality of solutions.
The Bottom Line
The "solution-first" approach to building an ecosystem may overlook the optimization of teaching and learning. While solutions like LMS and SIS capture valuable data, they may not cover important data elements like assessment design or course evaluations.
- Advice for Institutions: Carefully consider your desired insights into the teaching and learning experience and avoid choosing solutions solely for functionality or meeting student and faculty needs.
- Advice for Vendors: Recognize that existing ecosystems may lack comprehensive data tracking, highlighting opportunities to fill these gaps. Understanding accurate data coverage in teaching and learning ecosystems informs acquisitions, product positioning, and decision-making.
Never Miss Your Wake-Up Call
Learn more about our team of expert research analysts here.
Eduventures Senior Fellow at Encoura
Contact
July 17-19, Nashville, TN
Join us in Nashville this July for a professional development conference focused directly on your most pressing enrollment challenges. Educational tracks include:
- Admission and Marketing Best Practices
- Excellence in Test Optional Practice
- Diversity, Equity, and Access
- SEM Planning and Leadership
Never Miss Your Wake-Up Call
Learn more about our team of expert research analysts here.
