Measuring Credential Value & Why It Matters

A decade ago, the “college for all” movement was in full gear. Since then, a sizable shift has taken place with a move towards other forms of credentials. Is that a good thing? How should we think about this shift? In this episode, we are joined by Matt Sigelman, President of The Burning Glass Institute, to discuss credential value, how to measure it and why it matters for learners, institutions and the workforce.

If you’d like to read more from The Burning Glass Institute, check out their recent report, “No Country for Young Grads”, by clicking here – https://www.burningglassinstitute.org/research/no-country-for-young-grads

Transcript

Matt Sterenberg (00:00.982)
All right, Matt, welcome to the podcast.

Matt Sigelman (00:04.142)
Great to be together.

Matt Sterenberg (00:05.91)
So tell me a little bit about the Burning Glass Institute and the work you’re doing.

Matt Sigelman (00:11.374)
So we’re a fully independent nonprofit data laboratory. Data laboratory sounds kind of funky, but what it essentially means is this. There’s all sorts of great data that are out there today. Some of it, proprietary data sources, big data sources and the likes, some of it public data, at least for now. And what we try to do is to bring it all together to answer questions.

It often happens that we start out from the data and we try to say, okay, what problems can we answer with data? We start the other direction. We say, what are the big transformations that are needed at the intersection of work and learning? What are the big transformations that would drive greater economic mobility that would build, that would boost outcomes for learners and workers? And then we say, what are the data that would help?

drive those transformations? What are the models you could build on top of the data that would deliver the kind of insights that could point the way?

Matt Sterenberg (01:15.692)
that’s why I’m excited for this conversation because there’s a huge need for data in this field. And one of the things I want to get to is the credential value index. How do we start to think about the value of these credentials? You what’s the trade-off? How do we begin to these transparent? Transparently, we think of credentials as currency. Everyone kind of believes that it’s a door opener, but we haven’t had a way to measure value, but I want to take a step back a little bit and we’ll get there. But how

Matt Sigelman (01:35.853)
Yes.

Matt Sterenberg (01:45.272)
How different is our perspective on equipping the American worker? Like how has it shifted over the last many years? Because for a while it was college, college, college. What shifts are you seeing and how should we think differently about it?

Matt Sigelman (02:04.622)
So I want to be a little cautious here because there’s a whole course of people who say that we don’t need college degrees. And I think that’s more easily said than, or put it differently, that’s a glib thing to say. It’s not necessarily the best advice for everyone.

It is absolutely true that we need to make sure that there are good options for the two thirds of Americans who don’t have a college degree. But it is also true that a college degree remains on the whole a singular force for economic mobility, full stop. Now, here’s the thing. So.

Many of us are familiar with some of the overall stats that say, you know, just how many more millions of dollars you make by getting a college degree over the course of your career versus, you know, what you’d earn if you didn’t. And on average, those are true, but averages hold, you know, hide a lot of dispersions. There’s a lot of people who are doing astronomically well for having pursued a degree, but there’s a lot of people who never get any return at all.

And so when we’re talking about, you know, whether, you know, the shift away from the College for All movement, what we have to bear in mind is what we have to substitute. We need to make sure on the one hand that we are addressing needs and providing meaningful alternatives, and at the same time that we’re not throwing the baby out with the bath water. So it’s not that college isn’t worth it.

it’s that college isn’t worth it to enough people. Put differently, we have a lot of big discussions about the cost of college. We actually have less of a college cost problem. We have a college outcomes problem. Think about this, right? If the degree is supposed to give you about a million, a quarter, million, a half dollar incremental earnings over the course of your career, even if the average

Matt Sigelman (04:23.022)
student had to pay full price at a private institution. You you’re paying $300,000, $400,000 to get a million and a half dollars back. Who wouldn’t take that bet? The problem is that, you know, we are, our outcomes are inconsistent. So I want to, I want to call that out because

A lot of it comes down to how we connect between college and careers on the one side. And on the other side, how we make sure that people aren’t going to college can launch. We did some work recently where we looked at the careers of 9 million people who went straight from high school to work, bypass college. We found that the top 20 % of them

out-earn the median college graduate. So it is absolutely possible to have a good career and a middle-class lifestyle without going to college. And it’s not only possible, but it’s commonly done. What we found specifically was that there were about, there were a bunch of jobs that you could get coming out of high school.

about 17 % of people who come out of high school wind up in one of these jobs, where you’re increasing the likelihood that you’ll be a top earner by fourfold. At the same time, there is a set of jobs, and it’s an equivalent percentage of the directions that people go in that make you four times more likely by the time you’re 40 to wind up in or near poverty. And the bigger problem is that

twofold where the bigger problems are. One, that it’s actually sometimes really hard to tell one from the other. Sometimes, you know, coming out of school they pace very similarly. And second, to land into one of the good jobs, you have to, in many cases, there’s a set of skills you’re to need before you graduate. So, you know, I think before we start really kind of trying to reset the landscape, we need to make sure on the one hand how are we

Matt Sigelman (06:45.44)
making sure there’s better options in high school to prepare students for those who aren’t going to go to college? And how do we make sure that colleges are doing more to line themselves up with opportunity?

Matt Sterenberg (07:00.642)
I’m glad you said it could be glib because it’s usually a bunch of PhDs, college educated people saying college isn’t necessary. And it’s like a room full of people that are. Yeah, exactly. Exactly. But really what we’re talking about is college for the right people, right? It’s not college for all. think no one would agree with that today, but it’s okay. If not college, then what other credentials lead to.

Matt Sigelman (07:09.23)
And they’re never talking about their own kid.

Matt Sterenberg (07:29.624)
economic mobility, right? So how do we, and that’s where the credential value index comes in. And how do we measure these things? The outcomes piece is interesting because, you know, the last mile problem is something that colleges are talking about. Okay, how do we get our graduates employed or how do we gear you up for a career? And they’re feeling that pressure because it’s so expensive, right? And so,

Matt Sigelman (07:32.034)
Exactly.

Matt Sterenberg (07:56.554)
it’s an outcomes problem. think it also is a cost problem too, where if it was free and it was just for years of your life where you get to explore and learn, you might not feel the pressure to be like, you know, that this outcome is necessary. But given the investment, I think there is some added pressure there too. But the credential value index, we’re talking about other forms of credential, short form, short term credentials, workforce credentials.

and measuring the value of these, what do we mean by value? What is actual value?

Matt Sigelman (08:30.606)
So value, before we talk about the credential value index itself, which has some specific and disparate measures of value, I think this is a moment to talk more broadly about how we evidence outcomes. I think a lot of the conversation about outcomes has been about dollars and cents. How much are students earning when they graduate? How much are they earning a few years out, five years out, 10 years out?

number of states for public institutions have implemented state longitudinal data systems that use UI wage records in order to be able to say how much students are earning. And we use that as sort of this definitive measure. And I think that’s part of why this feels really uncomfortable to people in academia.

It feels really reductionist to say that ultimately everything that we do in helping people explore new horizons and in helping them define who they are as people comes down to dollars and cents. And the reality is that we have the ability today to have a more comprehensive set of measures. It would be just as it’s reductivist to say that

you can measure the value of a degree in dollars and cents. It also would be deceiving ourselves to say that student earnings don’t matter. that is certainly one of the things you would want to look at. But you’d also want to look at, for example, what percentage of students in any given program of study are working in high social impact fields or working in public service.

Matt Sterenberg (10:15.992)
That’s a great point because thinking about like, we only like, okay, no more social work majors and no more teachers. Yeah. And if we are saying this is we’re only valuing wages and this is the, what we’re valuing and we’re making it more public, more transparent. And at the same point on the other side of it, we’re like, we need more teachers. We need more social workers. What are we going to get? Yeah.

Matt Sigelman (10:23.298)
No more teachers, no more journalists.

Matt Sigelman (10:41.012)
Exactly, right? you know, and by the way, it also helps to offset, right, when you start looking at a program and say, gee, you know, people aren’t earning as much. Well, let’s be able to actually get to the bottom of why. We know that

some work that we’ve been doing, understanding the outcomes for graduates of historically back colleges and universities. And we know that HBCU graduates are substantially more likely than graduates of other institutions to wind up working in jobs with a strong social impact element. Okay, well.

Does that go part of the way toward helping us understand differences in earnings after graduation? But there’s other dimensions that you’d want to look at as well. You’d want to look at what percentage of students advance to leadership roles within their field over time. You’d want to look at what percentage of students start businesses. All those are things that we’re measuring.

And so you can actually take an empirical lens to being able to track those kinds of things, whether for a degree or as in the case of the credential value index for non-degree programs. So the credential value index was our, is our attempt to take a similar empirical lens to understand what happens to when people earn a given credential. So we looked at.

the career histories of millions of people. And we identified the 23,000 most common credentials in America, including every certification of any job market currency and then another about 20,000 credentials. And then we said, okay, how can we measure how that credential bends the shape of your career?

Matt Sigelman (12:43.63)
Think back to Algebra 2. You remember that y equals mx plus b thing? Your career has a slope. Yeah, your career has a slope. And when you get a credential, it change the slope of the line? underneath the Credential Value Index is a massive data set that for each of these 23,000 credentials and all of the people who earn them is measuring

Matt Sterenberg (12:47.894)
Yeah, slope, right? That’s slope. Yeah, there you go.

Matt Sigelman (13:13.26)
huge array of variables, but we distill it down to a few that we think really form almost kind of a nutrition label, if you will, for a credential. One, does it unlock the new jobs that the credential is targeting? So two, does it increase your likelihood of getting a promotion within your field? So the first is, does it help you enter a new field? The second is, does it move up in the existing field? Three,

Does it boost my earnings? And it’s a simple thing. A credential should do one of those three things. And our measures are all relative to a counterfactual set. So, you know, we’re rigorously defining a group of peers who are coming from who are, you know, at similar levels of educational attainment, who are demographically similar, who are

are working in similar occupations or entering from similar programs. And we say, OK, relative to that counterfactual, does earning this credential mean that you wind up earning more? Does it mean that you’re more likely to be able to enter that career, that new career that you were targeting, that you were trying to switch lanes into? Does it help you move up in your existing field?

times 23,000 credentials from over 2,000 different providers. And so those kinds of insights we think are really important in informing what has really been a Wild West. There are over 1 million credentials out there today, 1.1 million, according to the Credential Engine Project. Just to put that in perspective, there’s 170,000 head words in the Oxford English Dictionary. So it’s literally,

six times as many credentials out there as there are words in English language. So for whether it’s students who are trying to figure out what credential to earn, whether it’s schools that are trying to figure out what credentials to offer or what credentials could boost the outcomes of students in their programs, there’s really been no source of insight. The Credential Value Index is an attempt to provide

Matt Sigelman (15:41.616)
empirical evidence as to which credentials are working and which are not. The unfortunate thing is, and I’ll tell you why I think of the glasses is a third full, but only a third of credentials meet a minimum threshold for doing one of those three things I talked about a minute ago, for either getting you a wage boost, moving you into an enter, helping you enter a new job.

or moving up in your existing field, only a third. Now, that’s the two thirds empty. But the one third full is this. We did this times 23,000 different credentials. So do the math. That means there’s almost 8,000 credentials out there in America that really move the needle for people. So.

Matt Sterenberg (16:28.992)
And we didn’t have this data before, right? So, and we don’t even know what the purpose was in someone pursuing that credential, right? Were you getting it as part of a program that you were part of? Like, I was awarded this on my way. I didn’t necessarily pursue it for this purpose. And we didn’t have the data, right? So the idea here, I think, and I’ll let you continue as well, but it’s if…

Matt Sigelman (16:32.942)
Yes.

Matt Sterenberg (16:56.828)
I don’t have this information. don’t know which ones to pursue. And as we start to unlock that, it lets me know where to go.

Matt Sigelman (17:05.549)
I think it not only lets you know where to go, but it also helps us diagnose where we can do better. know, one of the things that we found was this isn’t one of these sorts of things where there’s good providers and bad providers. There are bad platter providers, but even the best providers have some credentials that don’t seem to obviously deliver value to the people who acquire them. As you said, there may be other reasons to pursue a credential.

We can’t know but but what that tells me is this that we all need a feedback loop that tells us whether what we’re doing is working and for most of us whether in higher education whether we’re talking about non-degree credential providers some of these online providers if you don’t have that evidence base and it’s been historically very hard to track these kinds of things without that evidence base

you don’t have a way of knowing whether what you’re doing isn’t working.

Matt Sterenberg (18:09.686)
Yeah. And for the providers, they want to have this information if the results are there. And in terms of the enrollment though, if you could say, hey, come get your full bottomy certification, there’s $10,000 in wage gains. That’s going to get people interested in your programs or your certification program. So you think they want that information.

Matt Sigelman (18:16.048)
I hope they do.

Matt Sterenberg (18:35.704)
And more people are going to want this information and need to capture this data because of workforce Pell short-term Pell that was recently passed. And we did a whole episode with Van Davis from WCET and Wichita explaining how it got passed and kind of the rigor behind it what people have to prove. Tell me a little bit more about where you think this is going and how institutions and states are going to have to start thinking about this.

Matt Sigelman (19:05.466)
So I think this forms the basis for crafting a workforce Pell strategy. Ultimately, you want to lay down tracks for programs and at an intersection of, the one hand, which programs and which credentials are, have a track record of working, of delivering for students. And on the other hand, what are the opportunities in your local?

What are the jobs that are at this kind three-way intersection of there’s huge and foreseeably durable demand not going to be automated away by AI. So employers can’t get enough of them on the one hand, they’re key to sectoral and regional growth on the other. And they offer good wages and upward mobility for workers. And so that

says, hey, look, these are the kinds of jobs that we really want to make sure we unlock. And then what are the credentials that align to them? It’s important to have a strategy here for a number of them on one hand. It’s always helpful to have a strategy, right? But while Workforce Pell is a watershed moment, let’s not overstate the

you know, kind of the level of funding that this is going to make available. The amount of funding that is likely to come through Workforce Pell is still going to be dwarfed by the degree funding in Pell. So, you know, this is a lot more than what’s been available historically. It’s a lot less than I would argue we need.

It’s a good start, but that means that there’s going be a lot of folks competing for those same funds. And so as a school or as a system, you want to be able to know which bets to make. Not, okay, here’s dozens and dozens of options, but instead, hey, look, there’s a limited set of programs that we really want to lean into. Ones that are aligned to opportunity, that are aligned to growth sectors.

Matt Sigelman (21:29.688)
and that we know work.

Matt Sterenberg (21:33.143)
One of the challenges that I think about, and this is probably far, far in the future, but we kind of see it on the college side where it was college for all. We have a lot of college graduates because we push that. And now it becomes, are there enough jobs that college graduates are gonna be satisfied with?

I just am thinking far, far in the future where we say, these are the credentials of value, these are the wage gains and then all these people rushed to it. And then we have tons of supply, right? And how do we create like a nimble system that works where it’s like, hey, Boise, Idaho is a great place to live. And then during COVID everyone moved there. And then people are like, it’s not the same Boise. Like, how do we create a nimble system?

Matt Sigelman (22:18.99)
Hahaha!

Matt Sterenberg (22:24.438)
that we don’t have lagging data that people can trust.

Matt Sigelman (22:28.474)
Yes. I think you’re answering your own question. I think part of this is about the agility of the system and the timeliness of the system. If we have that, and also about the granularity of the system. Agility and timeliness, think, probably pretty obvious. You need to be able to be responsive to changes in conditions. And it needs to be timely so that you can see those changes. But the granularity is also really important.

I mentioned that we’ve identified 8,000, roughly 8,000 credentials that work. The good news there is that there are high value credentials in every field. And that’s important because it says not, okay, gee, you have to be a cybersecurity expert. That’s the only credential out there. that’s how you get the kind of whipsawing that you’re describing.

If instead we can be sensitive in any given field, there are skills that you can learn that will help you move up. And there’s credentials that will add currency to your career. That gives us a lot more flexibility. Ultimately, what I want to see is something that works a lot more like Google Maps or Waze, where, look, the traffic starts building over here. It starts directing more cars on a different route.

Matt Sterenberg (23:48.953)
Yeah, that’s something that’s come up as a theme in other episodes too is on ramps and off ramps, right? You know, how do I get to where I want to go? And if I’m interrupted or I change my mind, how do I steer off? How do we stack these things on top of each other? Or this credential led to this job, but 10 years later, actually don’t want to continue this. Do I have to change industries entirely or what’s kind of adjacent to the thing that I’m doing? What are the transferable skills?

Matt Sigelman (24:19.918)
And part of that’s the responsiveness you just described. And part of it is also being able to have the foresight to see where roots lead. I mentioned before some of the research that we were doing, looking at the career trajectories of people coming straight from high school to work. And one of the things that distinguishes those good first jobs from others is that it’s less about the good first job. And they do pay better. And they

are more likely to offer benefits and all those kinds of things. But the big deal about those jobs is where they take you from there. And often that’s hard for people to see. And so one of my aspirations here is that we have more ways for workers and learners to be able to anticipate the transitions that they’re making, to be able to see where different steps they take could lead them and what

the array of options will be because we tend to feel kind of siloed or trapped in a silo when often we’re a lot closer than we may think to a broader array of opportunities.

Matt Sterenberg (25:34.563)
So here at Instructure, the parchment side is so dedicated to credentialing. That’s what we live and breathe. We started at parchment, was transcripts. And then we did diplomas. And now it’s certificates, badges, micro-credentials. And we’re heavily invested in that. And part of our ethos is learning is going to be lifelong. We need to better communicate skills, competencies, and all that.

I want to give you an opportunity. What do you want us to know as educational technology providers, as people that are trying to build the technology that serves students and hopefully creates more opportunities? What do you want us to know in this space as we’re building the technology for credentialing?

Matt Sigelman (26:30.554)
I would urge you to look at not only how you can be faithful recorders of what people have accomplished, but also to the potential to step in to be advisors in the process as well. Most schools have these days a

a degree audit system, something that will tell you when you haven’t fulfilled the requirements. But the process by which students are picking courses is also deeply aligned to the outcomes they experience after they graduate. We’re saying at the top of this conversation that unfortunately, a lot of students wind up

with disappointing outcomes. Over half of students, according to a study we did together with the Strada Institute for the Future of Work last year, over half of students wind up in jobs that they didn’t need to go to college to get. And the work that you’re doing to chart the courses that people take and the credentials that they’ve earned is also the ability to keep them

not only on track to graduate, but on track to build a portfolio of skills that can differentiate themselves. It’s the insight that people need in order to be able to say, how do I fill my course dance card? What are the courses that will help me build skills that set me apart? What are the credentials I can add to my degree that will make sure that at the end of the run, I can really stand out?

Matt Sterenberg (28:25.984)
One thing that’s come up in this podcast too is.

how you graduate from college and you know how impactful it was, you know, I can speak personally. It’s like, man, college was an amazing experience. I learned so much. I grew a lot. And when someone says, what’d you learn? You’re like, it’s actually hard to define. And so, you know, think about someone going in for their first job interview or trying to communicate what they learned or communicating the value of college in general. You know, you took all these classes, you know, your GPA.

Matt Sigelman (28:48.538)
Yeah.

Matt Sterenberg (29:00.29)
but you actually don’t have a really good accounting of the skills and competencies you developed. And obviously that’s where the micro credentialing and badging can hopefully enumerate everything that you learned in a specific course. And so some people are really trying to tackle this, but I do think that’s an interesting thing where you know personally how impactful it was. It’s actually really hard to define.

Matt Sigelman (29:04.762)
Yes.

Matt Sigelman (29:25.552)
I think, look, I think first of all, couldn’t agree more. I also think that a lot of this is gonna come down to how we aggregate things up. On the one hand, it is very valuable to able to look at a skill transcript as I know you guys have been working on in addition to just a credential transcript. But I would also encourage us to take a look at this from an employer lens.

It will be very hard for an employer to look at a 35 course transcript and say, okay, you know, here’s what this person knows or to be able to distill it down to a broader evaluation of readiness. be sort of like looking at a boy scout’s chest at all the little ribbons and saying, you know, I’m sure, you know, he’s proud of them, but for the rest of the world, that’s right.

Matt Sterenberg (30:18.976)
And it’s an economy of information too, right? Like the more that we measure, you know, it’s like, well, what do I send? What do they care about cutting through the noise of an employer? Like we have to be able to do that as well. Yeah.

Matt Sigelman (30:30.392)
And I think that’s the real opportunity to work at this backwards and say, hey, you know what? Let’s start not from what the student knows. We hope the student knows a lot of things if they’ve had a rich college experience. But let’s instead turn this around and say, for the opportunity that they’re targeting, what is it they need to know? And how can we bring the right sets of proofs that enable the student

to gain that employer’s confidence. That’s going to be especially important in an age of AI. We’ve been working a lot at what we call the expertise upheaval. We’ve seen concrete evidence that this is already happening, that employers are in many of the fields, the knowledge economy fields that have been the most valuable sources of opportunity for college graduates. Employers are shifting preferences toward hiring people with more experience.

and reducing the amount of opportunity they created at the entry level. That means that to win one of those entry level slots, students need to be able to stand out. And the best way to do that is to able to give employers confidence that a student has the capabilities they need, that they’re operating at a high level of proficiency, the extent to which we can bring that focus.

is going to be the extent to which we can really support students in not only completing their degree, but in making a great success of their start.

Matt Sterenberg (32:18.134)
Really appreciate the conversation. Thanks for joining me.

Matt Sigelman (32:21.506)
Enjoyed this. Thanks so much, Matt.

There’s always more to learn.