Data Analytics Solutions: Cutting Through the Hype

The world of data analytics in higher education is simultaneously becoming crowded and pressurized. Myriad products are available, with each promising to unlock critical patterns and insights from large data sets for institutions of all shapes and sizes. Given this noise, many institutions find it difficult to identify the solution that best meets their needs.

In our upcoming report, we provide institutions with a clear-eyed approach to selecting and deploying an analytics solution. We derived a set of criteria that applies broadly to key use cases based on vendor interviews, product demonstrations, and follow-up conversations. Using these criteria, we evaluated the strengths and weaknesses of the analytics products of six vendors: HelioCampus, Jenzabar, Blackboard, Civitas, Intellify, and Acrobatiq.

Not all criteria, however, are created equal. Some institutions have a greater need for scalability over integration with data sources, for example. Their search for a vendor should focus on which products excel at their priorities. This Tech Alert focuses on helping institutions inform this evaluation by highlighting the companies we believe are the strongest performers in each criterion we use.

Key Criteria for A Successful Analytics Solution

There has been much discussion in the analytics space around the types of analytic models (i.e., predictive, prescriptive, diagnostic, and descriptive), visualizations, and integrations different solutions offer. While initially helpful, we believe that evaluating an analytics solution should also consider the ongoing value it provides institutions after implementation. In our view, there are five key criteria that solutions should satisfy to provide institutions with this value:

1. Maintainability: Must be responsive to any changes, such as any new questions users may discover because of the iterative analytics process.
2. Privacy and Security: Adheres to best practices and prevents unintended disclosure of personally identifiable information (PII) through sound statistical methods (e.g., suppression, etc.)
3. Suitability: Fully meets certain critical use cases, including:
   1. Acquiring data from source systems.
   2. Ensuring data quality through the rectification of anomalies and inconsistencies.
   3. Integrating different data sets into a single data set.
   4. Unearthing patterns or insights from the data, using sound analytics models.
   5. Developing clear reports that illustrate findings.
   6. Providing alerts or indicators to highlight areas where stakeholders should act.
4. Usability: Provides users with an easy way to manage, review, and understand the output of the analytics.
5. Compatibility: Integrates multiple, external, heterogeneous data sources.

Our Findings: The Best of the Best

If you are interested in more detailed information about how each vendor performs on each of these categories, you’ll have to read our Analytics Deep Dive Report. Although each criterion had more than one solution that stood out, here are our picks on which analytics vendor excelled in each:

• Maintainability (HelioCampus): Primarily due to its origin as a spinoff from University of Maryland University College (UMUC), HelioCampus excels in how it designs its solution to meet institutional needs that may change over time. Although other products we reviewed also performed strongly in this criterion, HelioCampus squeaked ahead because of the extent to which it adapted to shifts in the requirements for analytics, such as revisions of models or introduction of new source systems.
• Privacy and Security (Jenzabar Retention): In addition to adhering to best practices, Jenzabar implements its Retention solution through a careful, collaborative, and institution-specific approach. Some of the other reviewed products also followed a similar implementation approach, but Jenzabar excelled because it included the determination of the right business rules to prevent the unintended disclosure of personally identifying information (PII).
• Suitability (Civitas): Civitas’ Illume solution meets all critical use cases. While we noticed that other products also met all use cases, Civitas inched ahead because of its greater care of ensuring data quality through data completeness checks for each source system and of alerting stakeholders on risk areas, such as student performance on “gateway courses” (i.e., foundational or required courses).
• Usability (Acrobatiq): Acrobatiq’s Learning Dashboard provides users with engaging and intuitive visuals, allowing them to filter and focus on the most important aspects of the data quickly. Acrobatiq excelled over others because of its collaboratively developed models that highlight student progress—tracked via learning and student outcomes—toward the goal of mastery of course material.
• Compatibility (Intellify): Intellify Learning positions its cloud-based solution as a platform as a service (PaaS) model, based on the interoperability standards of IMS Global. Intellify stood head and shoulders above other reviewed products because of its range of data integration options, bringing together many diverse sources to source an analytics engine, which, in turn, can share data with a broad spectrum of systems.

Ensuring Success

A clear understanding of the analytics domain and the relative strengths and weaknesses of the different solutions is necessary. We believe that the first step toward ensuring that a solution provides the greatest value is to first establish the audience and rationale for the analysis, as well as the required data systems and preferred analytics methods. Taking this first step will help with clarifying your unique needs and identifying which aspects of a given solution is the most important to you.