When marketing online degrees, many schools make big claims. A degree is the ticket to a promotion, salary increase, or a new career, and online learning is a convenient way to get there. Usually, however, an obvious data point is missing. What proportion of online students actually finish? There is no requirement for schools to disclose an online-specific graduation rate, but the ratio would surely inform the conversation about the value of online learning. Does delivery mode help or hinder graduation?

While most schools do not trumpet the numbers, the U.S. News Online College Rankings do include online graduation data for both bachelor’s and master’s programs. Notably, U.S. News uses this data to inform only 10% of the ranking, offers no commentary on trends, and does not allow prospective students to directly compare graduation rates. Regardless, this is important data that deserves wider attention.

U.S. News scores graduation rates on a two-year average. At the bachelor’s level, the typical amount of incoming credit determines whether a three, four, five, or six-year graduation rate is used. For master’s programs, the standard is three years. The data is reported to U.S. News by schools that choose to take part in the rankings. Here are the results for the top 10 ranked bachelor’s programs overall and top five master’s programs by field:

Top Online Programs - Top 10 Bachelor’s and Top 5 Master’s by Field U.S. News Online Rankings 2015
*Six-year graduation rates at bachelor’s level (entering class of 2008-09) and three-years for master’s level. Some highly ranked schools/programs did not report graduation data, perhaps due to recent market entry.

Read the small print

Top-ranked online schools and programs—including both public and private schools with regional and national brands—exhibit a wide range of graduation rates. Many report rates in the 80s and 90s, but others are in the 40s and 50s. The top ranked online bachelor’s program, from Embry-Riddle Aeronautical University Worldwide, reports a graduation rate of only 24%. A glance at a random selection of lower ranked programs at the bachelor’s and master’s levels, reveals a similar spread.

How can we explain such variation? Relative selectivity appears to be a factor at the graduate level. More prestigious institutions tend to report the highest graduation rates, and U.S. News rewards schools for selectivity. There are exceptions. Johns Hopkins University’s (JHU) online graduate IT program is ranked first for selectivity and fifth overall. While it reports only 53 students, its graduation rate is 41%. Does this suggest the program is more rigorous than average or that it is poorly designed and students lack support?

Because many of the highly ranked graduate programs are quite large, reporting hundreds of online students, it is hard to conclude that stellar graduation rates are dependent upon small enrollment. At the bachelor’s level, scale may make higher graduation rates more challenging to achieve. Among the top 10 schools, University of Illinois Chicago has the best graduation rate performance with 90% success, but it enrolls a mere 182 students. Embry-Riddle enrolls over 14,000 online bachelor’s students, and Penn State World Campus enrolls over 5,000, but their respective graduation rates are 24% and 40%.

Some lower ranked schools combine both scale and above-average graduation rates. One university cites over 7,000 online bachelor’s students and a six-year graduation rate of more than 70%. The school ranks in the top 20 for “student engagement” and “student services and technology,” but lower for “faculty credentials and training” and peer assessment. Does this indicate a winning formula or low standards?

While U.S. News provides considerable detail about the online student body by gender, ethnicity, age, military/veteran status, and incoming credit, it is difficult to see any firm patterns by graduation rate. At the bachelor’s level, “maximum” class size is reported, but this reveals little about the experience of the typical student or pedagogic strategies regardless of class size.

Delivery mode doesn’t matter, but should it?

The obvious take away from the U.S. News data is that delivery mode is a poor guide to the quality of the student experience or outcomes. Among the top 10, the mean online bachelor’s ratio of 53% (with a mean of 55% among a random selection of lower ranked schools) is in line with the national bachelor’s graduation rate of 57%. This reinforces the notion that from the perspective of students’ likelihood to graduate, delivery mode does not matter. After all the hype, is the best we can expect from online learning that it matches conventional delivery?

The U.S. News data is not organized in a way that easily permits multivariate analysis of graduation rate performance. Undoubtedly, student demographics and prior academic record are relevant, but it is tough to explain schools that buck the trend. The rankings feature a wealth of detail about structures, services, and tools for online programs, but not their nature, quality, or value. The specifics of any particularly effective approach to teaching and learning online are not captured. This speaks to the weakness of rankings derived from quantitative inputs only.

For its rankings, U.S. News considers higher graduation rates as “better.” It assumes that a 100% graduation rate is best, which implies that it is the institution’s job to make sure it admits the right students, crafts an engaging program, and offers adequate support. From this view, non-completion is not the student’s fault. This seems simplistic, and creates a perverse incentive for schools to employ lax grading. Equally, U.S. News does not regard a high graduation rate as essential for a high ranking. Some top-ranked schools fail to graduate even a quarter of their online students. Does it make sense to have a top-ranked program with a low graduation rate? Does graduation rate merit a higher weighting?

U.S. News has done a great service in persuading a large number of schools to disclose online graduation rates, and further analysis will be illuminating. An assessment of the quality of graduates and the impact of pedagogic specifics, online or otherwise, remains elusive. As for those claims schools make about promotion potential and salary, prospective students must fend for themselves in the marketing Wild West.