Last week, Eduventures attended the 21st Annual Online Learning Consortium (OLC) International Conference in Orlando, FL. The event brings together a combination of faculty members, instructional designers, administrators, and technology and service providers focused on online and distance education.
One big announcement leading up to the event was that OLC received a $2.5 million grant from the Bill & Melinda Gates Foundation. The grant will help expedite the adoption of next-generation digital courseware solutions at higher education institutions in order to improve outcomes in general education courses, particularly for disadvantaged and underserved student groups. It will accelerate the expansion of the OLC Quality Scorecard and fund a competition recognizing and rewarding exemplary institutions and faculty for the effective use of digital courseware.
On the one hand, the grant presents a compelling opportunity; the budget to purchase digital courseware is one of the biggest challenges facing institutions that are seeking to lower costs, increase access, and improve student outcomes. On the other hand, it raises questions about the definition of efficacy and effectiveness for the competition. It also raises the question of whether this approach can be the catalyst for impacting success among disadvantaged and underserved student groups.
How do we define efficacy?
While everyone talks about the need to measure the outcomes of edtech solutions, there is very little being done to truly address efficacy. While there are a few initiativesin place, mostly in K-12, they are still in the very early stages. For the OLC competition, there will likely have to be a wide range of definitions of efficacy. These may vary based on an institution’s level of maturity, the scale of the problem being addressed, and the size of the impact made. These definitions might include:
- Successful implementation and integration with legacy systems and a new digital courseware solution.
- Providing a more cost-effective general education course for students with financial challenges.
- Generating insights from data created within digital courseware using predictive models to better identify at-risk students.
- Proactively delivering adaptive content to students in general education classes to enable them to address their own success.
- Proving that a digital courseware solution works by moving the needle on retention rates, persistence rates, and graduation rates among underserved populations.
- Sustaining the year-over-year impact of student success by continuously improving retention rates, persistence rates, and graduation rates.
While these are all valid definitions of what efficacy could mean, only the last two definitions get to the root of the problem: outcomes. Even then, I would assert that the timeline to demonstrate these outcomes is too long and the level of measurement is still far from the validity of a randomized control trial, the gold standard for demonstrating effective practice. It begs the question: What do you think OLC’s measures of outcomes and effectiveness should be in its competition?
What will it take to really address student success?
A modest investment is only the beginning. Achieving outcomes that will impact a challenge of this scale will require vision, strategy, and a concerted effort across institutions. Can this competition be the catalyst that prompts institutional leaders to begin thinking more strategically and holistically about student success for underserved populations? The award might provide funding for a year or two, but what happens beyond that initial period? Where will the ongoing investment of time, energy, and resources come from to ensure that scale can be reached to achieve impactful outcomes?
While the digital courseware solutions that support student success are increasingly important for higher education leaders, the market is highly fragmented. There is a multitude of solution providers at different stages of maturity and offering a range of products. Each of these vendors purports to address the retention problem and improve student success. They each operate with dramatically different definitions, as do the higher education institutions that are looking for assistance in improving student success and student outcomes.
This definitional problem is confounding for both the solution providers and higher education leaders. There is an inherent misalignment between the problems being solved and the solutions being offered. It is exacerbated by the fact that no single provider offers a solution to all of the problems across the institution through an enterprise-wide approach. Instead, the market is comprised of many point solutions (frequently under one vendor’s product suite), each addressing a few, very specific problems. This only creates further integration challenges and generates additional costs.
Eduventures has been actively researching this area in order to make sense of the retention solution market to enable higher education leaders to not only understand different solution providers, but also make sense of how to think holistically about student success.
For this competition, we want to encourage OLC, the Gates Foundation, and higher education leaders to ask the questions: In what ways can you think more strategically and holistically to develop a shared solution across institutions with OLC at the hub of the wheel? If this is the catalyst for beginning to solve the problem, what will it take to achieve sustained outcomes?
We want to know what you think about how efficacy should be measured and what it will take to ensure this grant has the intended impact. Email us with your answers, or come debate with us in Boston at the Eduventures Summit next month.