Last week, both the U.S. Department of Education (ED) and U.S. News launched new efforts to help consumers make smart college choices. These efforts come in response to stubbornly low graduation rates, tuition outpacing inflation, and uneven quality. While these problems are real, how helpful are these latest developments?

Over the weekend, ED unveiled its revamped College Scorecard, which allows prospective students to compare colleges on price, financial aid, debt, graduation rate, and programs offered. The big addition is former students’ average earnings six and ten years after enrollment, which is meant to help consumers judge the value of tuition.

Just a few days earlier, U.S. News introduced its list of “Most Innovative Schools,” which is based on a survey of college presidents, provosts, and admissions deans. Respondents were asked to name the schools “that are making the most innovative improvements in terms of curriculum, faculty, students, campus life, technology, or facilities.” U.S. News wants to highlight schools that the “public should be watching,” even if those institutions don’t do well in the company’s conventional rankings.

Both the ED Scorecard and the U.S. News rankings are valuable, but they are very limited. The key outcomes metrics on the Scorecard either exclude large numbers of non-traditional students (in the case of the graduation rate metric) or do not distinguish between graduates and dropouts (the earnings metric). The U.S. News “Most Innovative Schools” ranking offers no details on why a particular institution is deemed innovative, uses a blunt innovative/not innovative threshold, and obscures the vital difference between innovative plans and results. The link for each ranked school takes the reader to a general profile, which offers no insight into the supposed innovation.

The glaring hole in both the Scorecard and U.S. News rankings generally is the student experience. Both “rankings” rely heavily on inputs (e.g., admissions selectivity, faculty pay) and outputs (e.g., graduation rates, earnings) but struggle to say much of substance about what happens in between. U.S. News points to “intangibles” and makes do with opinion surveys. Yet surely college is a transformation of inputs into outputs. Neither the Scorecard nor the U.S. News rankings discern value added; that is, they don’t judge outputs based on inputs to gauge whether the student experience had any impact.

Eduventures clients are schools, not consumers. What is striking about efforts like the College Scorecard and U.S. News rankings is scant evidence that such tools help schools improve. Methodological limitations persuade universities to spend time explaining why a particular ranking is a poor reflection of reality or bemoan that rankings simply state the obvious: that wealthy, selective institutions admit lots of rich, well-prepared students who graduate and get good jobs. Other schools try to game the system.

Many colleges and universities feel that they are held hostage to standard data collection and vague, self-perpetuating prestige surveys, but better alternatives seem illusive or risky. What might an alternative look like? Key features should be:

• A balance between inputs, student experience, and outcomes.
• Metrics that get to the fundamental value of the college experience without biasing against non-traditional schools.
• A digest of how individual schools actually articulate the student experience and define outcomes, rather than settling for the weak standardized metrics available today. We need to deal with the true complexity of the situation before trying to rank anything.
• The use of inputs to evaluate outputs.
• An emphasis on comparisons that help schools improve, not simplistic rankings.
• School feedback to enhance the process.

In fact, Eduventures is working on such a tool—The Eduventures Dashboard—to help our clients better understand themselves, gain external perspective, and learn from each other. We will unveil Version 1.0 at the Eduventures Summit in Cambridge, MA on November 18-20.

The Summit will also give delegates an opportunity to evaluate the claims and counterclaims of innovation in a range of areas, from the role of social media in fundraising and the use of data analytics to improve student success to LMS platforms and competency-based education. We even have a session on the question of defining, tracking, and reporting student outcomes. Keynote speakers include:

• Arthur Levine, President of the Woodrow Wilson Foundation and former President of Teachers College, Columbia University, on the six forces changing the higher education landscape.
• Catharine Bond Hill, President of Vassar College and Professor of Economics, on the economics of higher education.
• Kathleen Styles, Chief Security Officer at the US Department of Education, on data privacy, security and student data.
• Linda Hill, Professor at Harvard Business School, on leadership development as a strategic imperative in higher education.
• Roger Novak, a founder of Novak Biddle Venture partners, which has invested in 2U, Blackboard, Fidelis Education, and Parchment, on technology innovation in higher education and the focus of the next generation of investments.

In addition, we are launching an awards program to recognize and showcase innovative programs that are making a difference for institutions of all sizes and demographics. The 2015 Eduventures Innovation Awards will highlight the achievements of individuals and organizations that share Eduventures’ vision for improving outcomes by supporting critical institutional areas. We’ve asked our client institutions to tell us about their work and team efforts in four critical areas:

• Successful enrollment management strategies in an evolving environment amid rapidly changing demographics.
• Examples of applying technology to boost student success.
• Compelling stories about successfully defining and reporting outcomes.
• Successful fundraising initiatives that are built on a technology-based approach, like crowdsourcing or social media.

We are accepting nominations for the 2015 Eduventures Innovation Awards through October 1, 2015. Learn more and submit your nomination here. Registration for the Eduventures Summit is currently open to clients. If you are interested in attending, contact us at summit@eduventures.com.