...
Program Innovation

Don’t Believe the Hype: The Danger of One-Dimensional Program Assessment

Blog post cover image

A recent news article named Econometrics as one of the fastest growing bachelor’s markets. By many measures, it does appear that way. Indeed, data from our own research suggests not just fast growth but significant scale and high productivity.  

An Eduventures investigation, however, reveals that not all is what it seems when it comes to Econometrics and illustrates the broader danger of one-dimensional program assessment.  

Read on to follow the clues for this example.  

Signs of Strength 

Econometrics is among the strongest bachelor’s fields of study today based on many key metrics used to assess program market performance. According to Eduventures’ 2023 Bachelor’s Market Update, of the more than 1,200 programs reported to IPEDs, Econometrics ranks third among those with the fastest growth (45% CAGR over 2017-2022), third among those with the most growth (+13,752 conferrals over 2017-2022), sixth in program market productivity (91 average conferrals per program in 2022), and 25th in market size (16,340 conferrals in 2022).  

In fact, this report reveals that among all programs reported to IPEDS, Econometrics was the only one that ranks among the top 30 bachelor’s fields of study in each of these metrics, signaling a position of strength. These signals alone should make any school investigate what seems like a surefire opportunity.  

New preliminary 2023 conferral data, however, shows that Econometrics bachelor’s conferrals were down 8% year-over-year from 2022 – the first such decline in this market. But this recent development does not provide much pause as it follows the trend of overall bachelor’s conferral decline (-7% year-over-year) and is likely a delayed reflection of the pandemic disruption.  

Even with the 2023 dip, Figure 1 confirms this market has seen significant growth in a short period of time.  


Figure 1.


When we apply additional context around this data, however, several questions begin to form.  

Illusory Growth? 

Comparing this program market to other economics-aligned fields - and to the Economics market overall - reveals Econometrics to be somewhat of an outlier. Figure 2 helps illustrate this point by layering these additional markets on top of those featured in Figure 1.  


Figure 2.


Note that while Econometrics is up 34% CAGR over the examined period. Its rise is almost a mirror image of the decline experienced in Economics, General (-13% CAGR). Numerically, Econometrics saw an increase of about 13,300 conferrals while Economics, General saw a decline of around 19,400 conferrals across these seven years. In other words, the growth in Econometrics is almost completely nullified by the Economics, General decline.  

The overall economics market, therefore, inclusive of all economics-aligned CIP codes (the yellow trend line in Figure 2), reveals a more stable market with some decline in recent years (-4% CAGR from 2017-2023), which is aligned with the overall bachelor’s market.  

Figure 2 immediately presents two questions: What is happening between the two trends of Econometrics and Economics, General? And why? 

The What 

Institution-specific data provides great insight. The main force behind the rise of Econometrics and fall of Economics, General appears to be a simple CIP code reporting switch. Let’s look at some specific schools to illustrate this point.  

  • University of California-Berkeley is the largest provider in the Econometrics market with 753 conferrals in 2023, but it didn’t start reporting to this field until 2019. In that year, 517 conferrals were reported – a total too high for a recently-launched program reporting its first conferrals. In 2019, U.C. Berkeley reported 202 conferrals to Economics, General but stopped reporting to this CIP code altogether in 2020 – a far cry from its 2018 conferral count of 655 in 2018.  
  • University of Chicago reported 522 conferrals in the Econometrics market in 2023, but it didn’t begin reporting to this market until 2017 with 366 conferrals at that time. 2017 also marks the first year that the University did not report any conferrals to Economics, General – a market it reported 370 conferrals to in the year prior (2016).  
  • University of Virginia reported 299 conferrals to the Econometrics market in 2023, which is just the second year UVA reported to this market. In the same year (2023), UVA also reported zero conferrals to Economics, General CIP code for the first time.  

This list goes on, but this sample provides strong evidence of schools changing CIP code conferral reporting altogether from Economics, General to Econometrics.  

The Why 

Why the reporting shift? There are two likely explanations.  

At first glance, it may seem that this conferral data could signal a shift in learning outcomes and/or program content; however, it appears that this is not likely the case. To expand, Economics, General focuses on “the systematic study of the production, conservation, and allocation of resources in conditions of scarcity, together with the organizational frameworks related to these processes.” Meanwhile, Econometrics focuses on “the systematic study of mathematical and statistical analysis of economic phenomena and problems.”  

With the rise of data analytics and data science across academic portfolios, and aligned skill demand proliferating in the labor market, we could be seeing a concerted effort to ensure that programming incorporates these components more to complement economics-aligned learning goals. But this does seem to be the most likely explanation, as a review of the schools’ programs mentioned above does not reveal any major positioning around Econometrics concepts.  

Therefore, we suspect this shift is mostly explained by Econometrics’ status as a STEM CIP code under the Department of Homeland Security’s (DHS) Student and Exchange Visitor Program. This classification is critical as it provides a 24-month optional practical training (OPT) extension for international student program completers – a major benefit for students looking to extend their time in the U.S. with work experience. Of note, Econometrics is just one of five social science-aligned CIP codes to have this designation and the only economics-aligned CIP code on this list.  

Why do we suspect this is the main explanation for these trends? Just look at the conferral data in Figure 3 for additional clues.  


Figure 3.


As Figure 3 reveals, just 5% of all bachelor’s degrees are conferred to non-U.S. residents. The Economics, General market reports three times the international average with 15% – over-indexing all social science programs at the same time. Econometrics beats all benchmarks with one-quarter of bachelor’s degrees conferred to international students.  

The DHS STEM designation for Econometrics came in 2012 – a year that saw just 399 bachelor’s conferrals reported to that CIP code. In 2013, conferrals jumped by over 50%, the largest year-over-year (YOY) growth spike seen in the market up to that point, to 609. Since then, the market has jumped by at least 20% YOY growth six times with two years seeing over 100% year-over-year jumps. This shows just how significant this designation is and helps explain the overall economics shift to the Econometrics CIP code.  

The Bottom Line 

Data in isolation can tell one story – like a high-flying program market – while additional context can quickly ground the analysis in market realities.  

The Econometrics story is one of largely illusory growth – in that the growth tracked is not what it seems. Scale is evident. Productivity is strong. But what looks like a greenfield growth opportunity is more the result of shifting institutional reporting than organic growth (a much less exciting story…for some).  

This is not to say that growth in this field is not possible. Just look at the University of Chicago, which has almost doubled its conferral count in this market since it began reporting to the Econometrics CIP code. But these examples can be matched by schools seeing a decline in the market as well.  

Rather than viewing Econometrics as a new growth opportunity for all, it is best to view the field through the broader lens of all economics programming – a large, productive, and mostly stable field that has moved as the overall bachelor’s market has moved.  

This Wake-Up Call demonstrates the many nuances at play when assessing academic program opportunities and how a one-size-fits-all method for assessing program markets has its pitfalls. Program assessment work is best done when leveraging a range of key market metrics combined with in-depth knowledge of the underlying data and market conditions along with a qualitative perspective.

Like, Follow, Share.

Subscribe card logo

Never Miss Your

Wake-Up Call