The End of Resumes? Skills-Based Hiring and AI with Tigran Sloyan of CodeSignal

Tigran Sloyan is the Co-founder and Chief Executive Officer of CodeSignal, the leading skills platform empowering teams and individuals to discover and develop the skills that will shape the future. Tigran is a recognized voice in the industry—a TED speaker, active Forbes Technology Council member, frequent keynote, and contributor to major publications. He is passionate about the intersections of technology, education, and talent acquisition, and his innovative insights are regularly featured in Forbes, Fast Company, Morning Brew, and more. Before co-founding CodeSignal, Tigran was part of the technology management team at Google, where he spearheaded initiatives like Google Hangouts for Education and Google Login for sectors like Travel and Publishing. 

💡 5 Things You’ll Learn in This Episode:

1. How AI and simulations are transforming skills-based hiring and eliminating resume bias.
2. Why resumes are outdated and how companies can move toward more accurate and fair assessments.
3. The role of Bloom’s Two Sigma Problem in shaping CodeSignal’s AI tutoring innovations.
4. How Generative AI is enabling real-world, high-fidelity skill simulations for non-technical roles like sales and leadership.
5. Why universities struggle to keep up with fast-changing skills and how private sector partnerships can fill the gap.

✨ Episode Highlights:

[00:02:10] From Google to CodeSignal, Tigran Sloyan’s journey into skills-based hiring.
[00:05:42] Why resumes are a poor hiring signal and what should replace them.
[00:11:38] Hands-on learning vs. passive learning and the power of AI-driven practice.
[00:15:16] AI-powered simulations for training sales, leadership, and technical roles.
[00:24:11] Can AI replace executive coaches and leadership mentors?
[00:30:15] Why traditional universities struggle to keep up with changing skill demands.
[00:41:17] AI-driven skills assessments as an alternative to applicant tracking systems.
[00:49:46] The need for collaboration in solving the global skills gap.

😎 Stay updated with Edtech Insiders! 

• Follow our Podcast on:
  • Apple Podcasts
  • Spotify
  • Amazon Music
• Sign up for the Edtech Insiders newsletter.
• Follow Edtech Insiders on LinkedIn!

🎉 Presenting Sponsor:

This season of Edtech Insiders is once again brought to you by Tuck Advisors, the M&A firm for EdTech companies. Run by serial entrepreneurs with over 25 years of experience founding, investing in, and selling companies, Tuck believes you deserve M&A advisors who work as hard as you do.

Chapters

• 2:28 From Google to CodeSignal, Tigran Sloyan’s journey into skills-based hiring.
• 6:00 Why resumes are a poor hiring signal and what should replace them.
• 11:56 Hands-on learning vs. passive learning and the power of AI-driven practice.
• 15:57 AI-powered simulations for training sales, leadership, and technical roles.
• 24:52 Can AI replace executive coaches and leadership mentors?
• 30:56 Why traditional universities struggle to keep up with changing skill demands.
• 41:58 AI-driven skills assessments as an alternative to applicant tracking systems.
• 50:27 The need for collaboration in solving the global skills gap.

Transcript

[00:00:00] Tigran Sloyan: The fact that I thought Bloom's two sigma problem now, 40 years later. It can actually be solved and that's where Cosmo comes in. Cosmo is our AI tutor and guide that's baked into every learning experience we create. Because you've talked about the technical piece. Yes, there's feedback in technical, but it's still not enough. Right? Sure. The ID will tell you things don't run, but sometimes you gotta wonder why.

[00:00:32] Alex Sarlin: Welcome to EdTech Insiders, the top podcast covering the education technology industry from funding rounds to impact to AI developments across early childhood K 12 higher ed and work. You'll find it all here at EdTech Insiders. Remember to subscribe to the pod, check out our newsletter, and offer our event calendar and to go deeper, check out EdTech Insiders Plus.

Where you can get premium content, access to our WhatsApp channel, early access to events, and back channel insights from Alex and Ben. Hope you enjoyed today's pod. Tigran

Sloyan is the co founder and CEO of CodeSignal, the leading skills platform empowering teams and individuals to discover and develop the skills that will shape. The future Tigran is a recognized voice in the industry. He's a Ted speaker, an active Forbes technology council member, a frequent keynote speaker, and a contributor to major publications.

He's passionate about the intersections of technology, education, and talent acquisition, and his innovative insights are featured in Forbes, fast company, morning brew, and more. I met Tigran a long time ago in a funny context at general assembly in New York when he was a Google engineer. But before co founding code signal, he was part of the technology management team at Google for a while, where he spearheaded initiatives like Google Hangouts for education, Google login for sectors like travel and publishing.

Tigran Slyan, welcome to edtech insiders. Hey Alex, so happy 

[00:02:10] Tigran Sloyan: to be here. Thank you for having me. 

[00:02:11] Alex Sarlin: I'm really excited to speak with you. You and I have crossed paths a couple of times over many years of being in the ed tech world, and I've been really in awe of what you've been doing at Code Signal for a long time, but I don't think we've had a chance to really sit down and catch up and talk shop for a while.

So let's get into it before we get into our, our personal stuff. I'd love for listeners who are not yet familiar with Code Signal. Give us a little bit of an overview of who you are, what brought you into sort of the education and assessment space from Google and what CodeSignal is all about. Absolutely.

[00:02:46] Tigran Sloyan: It's like you said, it's been a long time, so I'm really excited to connect and talk shop a little bit, but CodeSignal, we are on a mission to discover and develop the skills that will shape the future. What that means to us is we believe that talent is the most, most precious resource humanity has, because if you think about it, every single piece of technology, every single problem that humanity has and will face.

has been or will be solved by skilled and talented humans. And unfortunately, majority of that human talent and human skill doesn't get realized. Most humans don't even get the chance to live up to their full potential. And I personally started realizing that idea you've mentioned at Google, that was more than 10 years ago.

More than 10 years ago, I was working at Google. I was doing a lot of interviews. I was, you know, I felt like on top of the world. And at some point as I was doing those interviews, I started to realize that Part of the reason, actually a big part of the reason where I thought it was my skills, it was because I had gone to MIT.

Because I realized that most of the people that were going through those interviews had a very, very specific background. And when I asked the recruiting team, they said, yeah, that's how we screen the candidates that find themselves in the hands of engineers getting interviewed. And I knew a lot of incredible people.

who I grew up with, who I went to school with, who never had a chance to go to MIT. And even for me, it was a complete coincidence. It was a lucky coincidence. I grew up in Armenia. I had never heard of MIT. Most people who I grew up with had never heard of MIT. And I only accidentally at the math competition that I was at, found out that it was a thing because a random guy said, I'm going to MIT.

And I said, what is that? And he took a sticky note and wrote MIT. edu. and said, go apply. They love nerdy international math kids. And that realization that if not for that random incidents, I wouldn't be sitting where I was sitting and I wouldn't be doing what I was doing was a stalker and a wake up call for me.

Now on the flip side, on the flip side, I also knew that Many people never even had the chance to find their way of learning, to realize their skills and potential, let alone get into a school that would make them have a great resume. And this concept of the talent problem, which is that talent is the most precious resource humanity has, that most people never even get a chance to realize their talent and realize their potential.

And even sometimes when they do. Unless it's reflected on their resume, companies do not have a way of understanding or noticing it. Really gave me that, okay, I'm going to just stop doing what I'm doing. I'm going to quit and I'm going to go jump off this plane and figure out how to make this parachute on the way and fix this problem.

And that's been the past decade of my life. Two 

[00:05:42] Alex Sarlin: things jump out to me when I hear you tell that story. One is it's really admirable to look at your own experience and say, yes, you obviously had a lot of intelligence to be doing math competitions and all this, but there was an element, a real meaningful element of luck in finding out about MIT and getting in and going.

And then the fact that MIT, it became this pipeline to amazing technical jobs, such as Google to not say, well, I just deserve that. I'm amazing. But to say they're part of this was opportunity that I was presented to me. Part of it was luck. And there are so many people who don't have that opportunity. A lot of people don't think like that.

And I think it's really, it's, it's admirable to see it that way. I totally agree. And I try to think that way as much as I can as well. The other thing I love is that code signal in its very name. I think. Indicate something. I've always thought this was so clever and interesting that you talk about signal, right?

Signal is such skills is the word everybody uses skills based hiring It's it's important in a lot of ways. What we're really talking about is signal. How can you show? Your skills, how can you signal your expertise your experience your ability to do the job? So let's talk signal How do you think about signaling skills and why is why a resume is such a?

Bad way to do it. 

[00:06:58] Tigran Sloyan: Yeah, it is at the end of the day about signal in many ways, cause for many humans, education is a pathway to a better career and a better life and a better economic outcome for themselves. And unfortunately, in many ways, the resumes have made sure that education is primary purpose, uh, secondary educations, post secondary education's primary purpose has become in many ways, the signaling factor.

Which is also why it's so hard and it's so competitive and parents will do anything to get their kids into the right schools. But when many of the schools put all of their educational resources freely available online, you'd find that very few people actually do it, which proves the point that that degree is a signaling mechanism and not necessarily the goal being like, Oh, I'm going to go and get a very high quality education, build my skills.

And. When I first started the company, I started it with the same mission we have today, which is to discover and develop the skills that will shape the future. And I had a very educational focus, but I wanted to do it all. I wanted to fix skills based hiring. I wanted to fix education. I wanted to fix it for Early, primary, middle school, high school, college, post secondary, all of it at the same time.

The big realization we had very early on is that without assessing skills, you can't teach anybody anything. You can teach beginners because by definition, once they come in, beginners have zero. In terms of skills. So you've already got the signal. You need it to match them with the content and the practice and the things they need to do, but once they progress a little bit, you start to lose them.

And it's very easy to lose them because you either start to match them with the type of learning content that is too hard, or you start to match them with the type of learning content that's too easy. And traditional and also non traditional education systems struggle with this day in day out. And then lastly, to come back to the signal, if you're going to create an alternative learning system, one that's more inclusive and one that gives more people opportunity, that opportunity has to be reflected in the outcomes in the job, in the economic opportunity, in that potential.

And if companies are not doing skills based hiring, that is just not going to work. And that was a tough realization because I was As a naive entrepreneur, like entrepreneurs have to be, when I came out of Google, I was like, give me a year, I'll fix it. And then we realized the depth, the depth of the problem.

And we had to make a choice to say, you know what, we're just going to put this big dream on a semi pause and go fix the thing that is really fundamentally broken. And it's going to be a roadblock. For us and for everybody else in the ecosystem, which was building a high signal direct assessment system that assesses people not using multiple choice questions.

You talked about signal, right? Yes, resumes are a horrible signal, but multiple choice questions are not that much better because there's so many examples in our daily lives, right? You wouldn't want a doctor operating on you who did a multiple choice exam, and that was basically the credential. And there is a reason we don't give out driver's licenses by just doing the written test.

We get you behind the wheel, and that's kind of how we feel comfortable that you can do this, right? Now, we had that philosophy from day one, that if we're gonna get a high signal measure of skill, we need to create simulations, we need to create hands on experiences that actually get at the root of, could you do this, or could you not?

On the flip side of it, when it comes to, well, first, getting companies to trust that signal of skill, Right. They need to know that it has to have that very high fidelity from ensuring integrity to fully measuring what matters. And then when it comes to teaching people, the skills that you're trying to teach them, again, you come back to this concept of if you can't understand what, what I can and cannot do already, right.

If you're showing me a video, like imagine teaching your kid how to ride a bike by showing them a video. And not actually letting them touch the bike, you got to get them on the bike, right? And 90 percent of their time is spent on doing it. And as they're doing it, you're both assessing their ability to do this.

and giving them guidance and correction and mentoring. But it's the process of doing. That's how our brain works. That's how our brain creates synapses. That's how our brain wraps the myelin around those synapses that make us good at doing something. It's not by passive observation. 

[00:11:38] Alex Sarlin: Yep. Learning is doing active learning.

100%. I love the metaphor of the driving and the bike riding. And I think they're, they're powerful. It's very natural for us to see physical activities as something that you're, you have to do and do in front of an expert and do many times and practice your free throws or practice your, with your training wheels or all of those things.

But we often don't translate that to, to mental activities like coding. I know that coding is, is only one part of what you do. You're, you're, again, the name of the company is code signal. Signal is in there, but coding because of your technical expertise and because coding is a relatively new field of which there haven't been that many or almost any really good validated assessments and sort of skills based signals that was sort of how you've been started.

But I understand you're expanding far beyond coding at this point. Tell us about that expansion. That's really interesting. 

[00:12:30] Tigran Sloyan: Absolutely. So the funny you mentioned, while the coding part goes to the roots of that, we started with technical roles, primarily the thing that I knew, and we knew quite well, in some ways, part of a reason, because we knew we were going to do way more than coding and way more than technical skills all along, because those things were Technical skills are not the only skills that are going to shape the future.

And part of the reason we thought the name will work regardless, because another way to think about it is that it's about leveraging code, which is software to get you to signal. All right. So now you're absolutely right. We, like I said, the mission has always been to do this for more than software. The problem was always, how do you do education and assessments in a hands on way?

For non technical roles, because for technical roles, part of the reason many engineers tend to be introverts, it's really easy to just sit in your bedroom, grab a laptop and practice your skills, right? It's one of the beautiful things about coding is that. There are tools that allow you to practice on your own and get better at doing now, how do you do that for business skills?

How do you do that for sales skills? How do you do that for management skills? And I've been thinking about this for the past seven years, as we've been building the company time after time, I'll come back to, I really wish, and I really hope we can do this and extend this method of hands on practice focused learning and assessments.

Into non technical domains, but I don't know how we're going to pull it off because technology isn't there yet. And then the Gen AI revolution started. The Gen AI revolution was a music to my ears. And from day one, as soon as I saw like GPT 3, GPT 3. 5, and we had worked closely with the OpenAI team, even during the training process, when I saw the early versions of it, I was like, this unlocks so many things.

Agreed. Because the first thing it unlocks is simulations for other roles. So today, if we're trying to teach a salesperson how to be a better salesperson or a manager, how to be a better manager, we don't just show them a video. We actually have them in a simulated role play with a conversational AI. So if you're a manager, you would be practicing, for example, giving tough feedback, constructive feedback to a direct report.

It was played by an AI and that AI will not play nice, right? Because you want that just like in, you mentioned sports, it's kind of crazy how in sports and every other physical domain, it's just obvious to us that you have to engage in it. But if you think about in knowledge work, most of the time, you're not just sitting there and thinking you're either speaking, which is a physical activity or typing, which is also a physical activity, 

[00:15:16] Alex Sarlin: right?

[00:15:17] Tigran Sloyan: So in the same way that you can't get good at shooting a ball. If you've never done it before, you cannot get good at doing these things if you've never done it before. So our philosophy is, if you've got to get good at giving tough feedback and managing a tough conversation with your direct report, we're going to get you to practice it.

And we're going to get you to practice it over and over and over again until you feel that confidence to be go in a real game in a real situation and handle it. Same goes for sales. For sales, we customize it more, right? When we're working with organizations, they normally don't want just general sales.

Obviously, if this is for a consumer learner, we will teach you general sales skills on a general hypothetical scenario. When we work with companies, they want their sales scenarios, their customers, which is. even more fascinating because we can't just simulate, we can simulate exactly what happened on that call last week and let you get better at doing it.

[00:16:13] Alex Sarlin: I share exactly your excitement for that the power of generative AI to unlock simulation based learning at a level we've just never imagined before. A word you said several times in passing there that I think is, is one, I personally think is one of the keys and I. Think you would agree is feedback, right?

When you talk about the fact that coding is done in an environment that has natural feedback to it, right? It tells you when things don't compile. It shows you when it works. It shows you when there are errors. It tells you what the error is. It shows you what line it's on. Like, I mean, it's, it's a closed system.

And that when you talk about the introverted engineer stereotype of somebody sitting there, I always think of Bill Gates at his 10, 000 hours, right? Bill sitting there back and forth with the computer. It's that is not us. It's not as solipsistic as. It sounds because you're actually in a conversation.

It's like shooting free throws or practicing your, your biking to continue that physical metaphor. And we have not had that in almost any other field in other fields. It's, it's relied on, on people. Sales trainings tend to be with people or difficult conversation training tends to be with people. And.

Coaches giving you feedback. And that's just been hard to scale. It's been executives have had that kind of training for a long time, but very few of us have had the ability to, to have that rapid feedback cycle around the things that we do every day. And suddenly it's not only possible, it's possible in for almost any tool in almost any context, as specific as you'd like, based on any kind of previous data.

It's thrilling. It really is really exciting. So let's back up for a second, because there's something you did in the past that I think is a good precursor to this. And there's something we actually, one of the places we started CrossPaths, which is that one of the projects you did at Google was you were the technology management lead, I believe for the Google Hangouts for Education project.

People may or may not remember the Google Hangouts for Education. It was a really exciting project that. Honestly should have taken off. It was really all about pulling people together and enabling people to work with mentors or coaches or experts or teams through Google Hangouts, which is sort of a teleconferencing platform in an educational context.

And actually there was even the ability to sort of charge by the minute or by the hour, almost like to enable tutoring really exciting project. That was basically a way to scale. Peer feedback to scale that kind of human feedback, but didn't take off the way, you know, I think you and I both would have wanted tell us about your experience in that project and how human feedback, even though it's probably the gold still the gold standard, at least for now, just doesn't always scale the way we want it to.

[00:18:49] Tigran Sloyan: It's funny. That's. Last time I saw you, it's crazy how hard it is to not fall apart, but it was more than 10 years ago. You were at Coursera, I was at Google, and we were working together to get this idea of, could you do remote tutoring, remote mentoring for individuals? Because, and this is when I learned about the Bloom's two sigma problem.

And at that point, it's been 30 years, right? The Benjamin Bloom, who's better known for the Bloom's taxonomy, but his other more significant finding with his PhD students. Is that if you take a group of learners and you get split it into two parts at random and you give one part mentorship and tutoring and the other part, just traditional classroom instruction learning, the ones that had one on one tutoring and mastery focused learning, which is what we already talked about.

They perform two standard deviations above. compared to the control group. And I mean, you know, in educational psychology, these kinds of experiments are done day in and day out. Most of them don't work. Most of them come back saying it has no effect. We're sorry. This is one of the rare ones where like, it doesn't just have an effect.

It has two standard deviations and it has been dubbed as the two sigma problem because Of what you said, great one on one tutoring and mentoring really works. How do you scale that? And Google Hangouts in some ways was an attempt to say, okay, well, what if we scale it from, you have to be in close proximity with somebody to doing that remotely.

But even that, there's only so many tutors, and there's only so many hours, and it's still very, very expensive and inaccessible. You will never scale that to billions and billions of people, which is why it didn't take hold. And ever since leaving Google and starting the company, that's been the other piece, which has been at the back of my mind of like, I know that just assessments and mastery and just practice and hands on, Is not going to be enough.

Those are a must have without it. We don't even have a shot, but the last missing component that is going to make this possible is one on one tutoring and personalized learning. And when I said seeing generative AI revolution come to life, that was the second piece for me. The fact that I thought Bloom's two sigma problem now, 40 years later.

It actually be solved. And that's where Osmo comes in. Cosmo is our AI tutor and guide that's baked into every learning experience we create. Because you've talked about the technical piece. Yes, there's feedback in technical, but it's still not enough, right? Sure, the IDE will tell you things don't run, but sometimes you got to wonder why.

So even in the technical domain, sometimes you go on this Case of, Oh my God, I've been Googling this for three hours and I'm still not quite sure why, and by every Googling, I get more confused, you know, just like you're used to look up, you're having some symptoms, what's wrong with you? And then 15 minutes of Googling makes you think you are about to die.

That effect exists in technical learning, where you go on this rabbit hole and you get more depressed by every Googling. And it's really easy to be discouraged and just give up on that journey, especially if you're early. Right. Because people have been doing this for decades, likely have gotten used to that.

Like, okay, that's okay. I'm going to get confused. I'm going to get worried. I'm going to get, see, like, I don't know anything anymore, but I'm going to bounce back. People who are early in that journey who haven't developed that skill, they, it's just very easy to give up. And you hear those stories left and right, but then you come to business and non technical skills.

And that's even a bigger issue. Because like you said, at least there you get an error. And you know something is wrong here. You don't get an error. That's why every person has had a bad manager. Imagine what that means. How many bad managers are there? It's not because people are trying to be bad managers.

They most of the time, they don't know. That they're being bad managers because when you're being a bad manager, your direct report doesn't throw an error and say, not working, you got to cut this out and go Google how to be a better manager. 

[00:23:14] Alex Sarlin: That's hilarious. Wouldn't that be nice? So that would be great.

If that, uh, 

[00:23:18] Tigran Sloyan: yeah, like a red flashing light, right? 

[00:23:21] Alex Sarlin: Exactly. But this is exactly what is possible now. I mean, you could have a managerial simulation where you say, okay, this person isn't doing X, Y, Z. You said this, whoops. You just made a big mistake. They now hate you. They're about to give you a bad review on Glassdoor and, and they're out the door.

Like a hundred percent. It is such an exciting change. So, so talk to me about that relationship between the one on one tutoring and the AI, because this is something, you know, I've talked to a number of different tutoring companies in the three years during this podcast, and I think. Everybody is just really wrestling with the sort of human, like actual human versus simulated human companion, mentor, coach, AI based being, and who plays what and what's possible and, and what's moral and all of those things.

How do you think about the sort of human versus simulated human issue? 

[00:24:11] Tigran Sloyan: Absolutely. So, I mean, this Also has been a solved problem. It's just been solved in not a scalable way. And the reason I say it's a solved problem is there are leadership coaching firms, many of them, many successful ones, just exclusively for high level executives.

And the way they approach it is they write detailed scenarios, detailed scenarios that outline. What the actors, because they literally hire actors, right? This is the level of budget we're talking about. But if you've been a high level executive, you've been in this coaching session where the coaching firm will bring in the actors.

The actors have no idea who you are, what you want, but they've got a script and the script is not like a movie script, but it's a scenario. Like a murder mystery game type of scenario that says, this is who you're playing. These are your objectives. This is what you know, and this is what you don't know.

Right now, of course, you're going to leverage your brain in case some weird situation comes up to handle it, but you're supposed to stay within the confines of this scenario, and this is your role. In that, right? That's how you just take that and you bring it, bring it to a, an AI powered one, instead of having an actor.

So if you bring a generative AI model, and then a little lamb, obviously you need voice. You need many other things to create a converse true conversational simulation. Cause you don't want to be typing. That is not the interaction that you're having. You need emotions, which is another thing that is really hard to do.

It took us. So long to not just have an LLM there you can chat with, but an LLM that talks, that has emotions, that gets frustrated, that gets mad, that gets loud, and how's that type of interaction with you, but is playing a predefined role. Now, this could be in an assessment scenario in which you don't really get feedback, right, because we're essentially assessing.

Your ability to do this, for example, we have a customer interaction assessment, which is a general customer interaction assessment that is measuring if you're a customer interviewing for a customer facing role or learning to be a customer facing person, how well do you handle frustration? Because many cases you get on a call and the customer is Matt, right?

Cause guess what You really doesn't work and that gets customers mad and you have to, if you're a customer facing, you have to know how to manage that frustration and having the LLM actually be able to get frustrated and say, going to quit. This is it. Are you kidding me? They're not delivering. And you being able to stay composed, respectful, and turn it around and build it into a trust moment.

Now you go to the learning side. No, that's not going to cut it. That's great practice. Yay. The error is there in some ways, but you need feedback, right? And this is where Cosmo comes into play. And that's a whole different AI mentor and guide that is observing what's happening. So, as you talk, as you go, Cosmo, like the coaches, right, in the executive coaching scenario, you will have the actors who are role playing with you, but you'll also have the coaches.

And after every short session, you say, okay, pause. This is what you did well, and this is what you did not do well. And then you don't just say, okay, goodbye. You say, go again. Try again. Yeah. Because you talked about sports in the beginning. We've known in sports and physical activities is all along. You don't just go to a practice, play a whole game.

And your coach says, this is what you did. Well, you, this is what not did well. And then say, okay, go home. We'll see you next week. For drills, right? It's short drills. You go in, you do it, you get instant feedback and then you do it again. You get another feedback. This is, it's one of my favorite books, Daniel Coyle, The Talent Code.

He's done this research across musicians, across artists, across sportsmen, and he's shown that it's called deliberate practice. It's pushing yourself to the limit of your ability in this carefully designed practice sessions, getting instant feedback on what you are and aren't doing well, and then going back and repeating it.

over and over again. And guess what? In addition to being highly effective at building skill, this is also way more fun than just sitting there for hours and watching somebody talk at you. That's boring. 

[00:28:36] Alex Sarlin: Yes, you're speaking my language. I totally agree. Anders Ericsson, the deliberate practice psychologist who came up with the concept of deliberate practice, there's another great book about this called Talent is Overrated by Jeff Colvin.

But based on the same theory, I think the deliberate practice theory is Your, you know, concept is something that is so vital for education and so infrequently followed for a variety of reasons, sometimes technical limitations. And I couldn't agree more. The other thing that you're alluding to here that I think instructional designers and our listeners may recognize is the difference between formative and summative assessment, right?

The assessment use case is assessment of learning, right? Can you get the job because you know how to handle frustration and you don't get immediate feedback or maybe never get feedback because if you're going to come test again, you don't, you want to come in fresh versus formative feedback assessment for learning, which is exactly how you, you know, you just framed it.

It's like, well, that practice and formative assessment, we know to be so, so effective in learning, but it's just, it's something that's very hard to implement. Until now, frankly, I think until now, I mean, AI can change assessments so deeply, make it so much more authentic, make it so much more deliberate. I guess my, my follow up question, and I mean, I know, I sort of know how I feel about this, but I'm not sure if it's the most popular opinion is can.

The deliberate practice coach, the thing that plays a deliberate practice coaching role, which is part of the deliberate practice theory, you have an expert coach giving you feedback about how to improve, especially on your exact where to practice. Can an AI play that role? I think definitely yes, but I know that some people who are a little more cautious about the AI's capabilities are not sure.

What do you think? 

[00:30:15] Tigran Sloyan: Well, I think it absolutely can, but not in isolation. Again, we go back to, we know how this is done at the top notch executive leadership teams that would charge literally a thousand dollars an hour for every person involved to deliver this. They don't just say, because they try to scale too, not to billions, but they try to hire maybe like a hundred coaches.

Many of these coaches. Come from a theoretical background, haven't done this before. So they don't just send and say, Hey, go be charming and figure this out on the spot. No are clearly defined rubrics. And this is not a secret. And funny enough, you talked about like all of these pieces, right? We talked about hands on learning by doing, we talked about mastery and assessments.

We talked about personalization, customization, mentoring. These are not secrets. We talked about Bloom's two sigma problem. These are things that have been known in the education space for decades. But we have the ostrich effect. The ostrich effect is that because it hasn't been technologically possible for many, many decades, we've just put our hands in the sand.

We're like, we're just not talking, not going to talk about it. We know this is the best way to learn. We're just not going to talk about it. We're going to pretend. We're going to pretend it doesn't exist and forget about it. But we have to pull our head out of the sand. It's time because now it's technologically possible.

And I do not see enough people building this, embracing this, and understanding that we know the best way to teach, assess, and learn. And we have to keep that bar. For everybody, instead of continuing to pretend that nothing has changed and that these things are still not possible and whatever we used to do is okay, because it's not okay.

The skills gap keeps on getting wider, more and more humans are finding themselves in a situation where they're not living up to their potential and the dark side of AI is that it's going to displace many, many jobs. Millions and millions of people are going to find themselves unable to provide for their families and more than ever, that continuous education and continuous learning and giving people a shot at building their skills, living up to their potential and living up to the economic opportunity that they Can't have and deserve it's on our shoulders, right?

Like if we're not going to do it, if the educational entrepreneurs, the educators of the world who understand this space deeply, aren't going to do it. Who is, so there's no hiding away from it. That's a great 

[00:32:45] Alex Sarlin: point. So, so let's talk about that future, because I think you mentioned the skills based hiring.

basically movement and how when you started CodeSignal, you said, Hey, a year from now, we got it. We'll make this happen. And then sort of started to realize, wait, there's a lot of pieces at play, a lot of major status quo that makes recruiting and hiring and the evaluation of skills, something that sort of has been stuck for quite a long time.

And I think we've, I think we've come a long way. And I think CodeSignal has been part of that change to start to recognize that skills based hiring is Is possible and is, is more and more of a, of a real thing. I had a formative conversation many years ago with, with Ryan Craig, one of my ed tech heroes.

And I said, why is this happening is when I was at Corsair, why, what is the blocker here? And he had an answer that always stuck with me. He said, it's the applicant tracking systems. He's like, look, companies hire with these machines that look at resumes and they have to evaluate signal from afar. And what do they do?

They look at the. MIT, hey, MIT, you're in, or they look at the experiences. They look at various things on your resume. It's just been, I mean, that's, it's been so goofy. There's a lot more to go, but I think we've come a long way. Can you tell us about what you've seen in these decade of how skills based hiring has evolved and what you think is next?

[00:34:01] Tigran Sloyan: So first of all, assessment is hard, like real high quality, high signal assessment is incredibly difficult to do because you don't just need simulations that cover a vast range of the business space of skills, but you also need content. And honestly, content is the hardest part because at least the other one is just pure software.

You build the software. Great. It works, right? It's a one time investment. Content is very, very difficult because a, it requires a lot of hands on building of that content. B, it requires constant rotation and update. So in many ways, you're in like a Netflix. And less like a Slack, right? Like users are not supposed to bring their own content.

And this has been tried before. There's been assessment vendors that came in and said, you know what? We're going to give you the platform and the simulation. You're going to go figure out how to do the content. They can't. It's like, you know, going to a restaurant and them saying like, Oh, here, here's the kitchen.

Here's the ingredients. You got this, but they don't got this. What ends up happening is they will build assessments that are not predictive, that are severely biased. And do not produce the signal in many ways end up harming the overall experience. So, and honestly, I think applicant tracking systems have always had the best of intentions, but the real villain here, the real villain is the resume, right?

And the resume has been around for Some argue for 500 years with Leonardo da Vinci writing the first resume. I think it's a bit of a gimmick, but Hey, it looks like a resume. I don't know if you've seen it where da Vinci is writing to the Duke of Milan and saying, I can make weapons that will destroy your enemies.

Literally. It's like da Vinci writing a list of his accomplishments on what things has done, trying to get a position in the, in the court of the Duke of Milan. And the funny part is that one of the footnotes says, Oh, and I can also paint. That's the most hilarious resume that you would see, but the real rise of resumes is more a 20th century thing, right?

As the industrial revolution came into full swing, as a lot of people moved from villages into the cities, as great depression struck and many people were out of work, they were trying to figure out, okay, how do I go out there and advertise what I can do? So people started writing on literally pieces of paper.

I can dig a hole, I can carry this much weight. And those were like the early resumes where people will write, hold up on the street trying to get a job. And then over time that evolved into writing as the job space changed, right? Hey, I can type, I'm a lawyer, I am a doctor, I can do this, I can do that. And then education was one of the primary signals you can send.

So it basically became, what can you do? Self declared written on a piece of paper. Then we took that with the rise of computers and made it into a PDF form. And then with the rise of internet, we made it into LinkedIn profiles, but the essence of it has not changed, right? The format changed, but in essence, that LinkedIn profile is not very different from the person in 1929 who was holding a piece of paper saying, I can carry this much weight.

Give me a job. How do I know if you can, first of all, and second of all, That is a really low signal way to identify people who lives that leaves a lot of people out of the equation. So in my opinion, though, what has happened over time is best of intentions are a great thing, but you need wider pressures that make people change, right?

Because just best intentions, because you know what? No one's out there trying to say, you know what? We're just going to exclude others. We're not going to give people opportunity. We're going to be biased. No one says that. Everybody has the best of intentions. But what happens is they don't have the chance or the technology to do this.

Or they don't have the pressure to change. So for companies, what has happened over the last decade or two is skills gap has been getting wider and wider and it's been harder and harder to find people to fill those roles, which means they have to go out of the 1 percent that has MIT, Stanford, Harvard on their background.

They just have to, there was no other way. So they were pressured to go look for, how do we go beyond the resume? How do we go find people based on their skills? And like you said, we were at the forefront of that revolution, especially for technical skills when it's been changing so much and so frequently.

And part of the reason why that pressure has been building up even more is skills with new kinds of technologies, new kinds of skills get created. Like without a typewriter, typing is not a skill without a computer, without internet, being a web developer is not a relevant skill, just like prompt engineering is not relevant if there's no LLMs.

So technology creates new types of skills and with technology accelerating faster and faster. The new skills and things that we have to know to be effective keep on changing faster and faster and traditional education is not designed to update curriculum at that pace traditional education. I mean, when I was at MIT, MIT was still teaching Lisp.

And it's been a decade since less possible language used in any modern application, and that's MIT, right? And it takes decades to go through that change. And nowadays, within a year, you might have a new set of popular skills. I mean, mobile engineers, look, mobile engineering is one of the hottest, most independent jobs right now.

It's only been around for 15 years. Right 15 years ago, I mean, the iPhone came out, what, 2007, the app store came out around 2009. And all of these companies like Uber, like Instacart, like Airbnb, they were all created on the back of that mobile revolution and mobile engineering becoming a job. You'll be hard pressed to find a single traditional large scale education university system that teaches mobile engineering as a major doesn't exist.

And it's going to continue being a problem. So what I'm trying to say is that there are forces that are much bigger than us that are putting pressure on, you have to go the skills based route because skills are changing too fast and skills gaps are getting bigger and bigger. So you have to adapt and then just going the skills based hiring route is also not enough.

Because there's not enough skilled humans right now. So you have to develop the people you have instead of just constantly trying to close skills gaps through hiring. That's no longer works. And right now, many companies are trying to say, okay, we've got to be at the forefront of AI. We've got to hire machine learning engineers.

Guess what? Machine learning engineers are all happily employed making more money that can, you can imagine. You cannot go hire good ones. You just can't, the only solution is you have to go look inside and you will find many incredible people who are excited, ecstatic to learn. You just got to give them a chance and a system and encourage them, give them the time to build that skin.

[00:41:17] Alex Sarlin: Yeah. I love, I mean, fascinating history of the resume. I love that. I've never, I've never heard almost any of that. And so such an interesting way to look at it. You know, you see this sort of like the resume is this. Piece of paper, literally, whether it's a sign in 1929 or, or, or, you know, something printed out now or a LinkedIn page.

Now, LinkedIn is used much more by college graduates than not college graduates, by the way. So there's already issues in there. It's sort of this, this thin little bridge between here. I'm going to try to showcase everything. done in a way that, that an applicant tracking system or a hiring manager or can understand.

And the signal is very messy. One thing I find really interesting, you mentioned, you know, that it's hilarious, right? Putting painting as like on the bottom side as a, as a skill for Leonardo da Vinci, but skills. I mean, part of what I wrestle with, with skills is that Skill. There's been a skill section of the resume for a very long time, and it has traditionally been the absolute most silly, unverified, goofy, last thing you think about on a resume for a long time.

You put skills at the bottom. You say, Oh, I speak conversational French. And this it's, it's, it's so goofy. And Instead, we've turned it around and said, no, no, no. Not only is that not an afterthought, that's the center of the resume or not even say the resume. That's the center of the decision. That's the center of the signal, which is really, really exciting.

So, and then to your point, once you have that kind of system in place, then it becomes hopefully more meritocratic, less biased, more about what can you actually do. And then it exposes, frankly, the fact that not enough people have the skills and it creates, puts pressure on the education system. Now, the fact that the traditional education system is still, even in 2024, that slow to pick up the slack there, right?

Even when people publish here, the top 10 in demand skills like mobile engineering or, you know, machine learning, it still takes a long time for universities or certainly high schools to start teaching those things. What does that lead to? It feels like it creates pressure, but also opportunity in the in the private sector to say, okay, there's a gap here.

People are desperate. They have a huge return on investment if they can learn mobile engineering or machine learning or data analysis quickly. So for a company like CodeSignal and your You're now really moving into learning with both feet. You know, you have the assessments in place. Now the learning is in place.

You mentioned your Cosmo AI tutor. It feels like that creates an incredible opportunity for you and the few, not that many others who are really starting to put the pieces together. I agree with that. I totally agree with that, but I'd love to hear it from your perspective. Like, do you think that the trend of the traditional education system sort of Continuing to somehow just miss this and the private sector continuing to rise and fill the gap, do you think that will continue or will there have to be some adjustment where traditional education starts to say, you know what, we are, we have to be part of this or we're going to be dinosaurs, actually.

[00:44:17] Tigran Sloyan: Yeah, that's a great question. Well, first of all, I think going back to the question of intention, it's not like universities don't see this, right? They do. Yeah. It's not like they don't want to change. It's not like they don't want their students to be highly successful, get the jobs they want, get the skills they want.

But the system of universities has been designed hundreds of years ago, and it has not been designed for this. Pace of skills change and this space, and that is what's causing the problem. Now, any people, like you said, would look at that and say, incredible business opportunity. Let's go compete with all these universities.

Let's create an alternative version. Let's just destroy everybody in that system. But that's wrong because universities play a much, much bigger role in our society than just skills builders. Yes, skills building is one of the things that universities are supposed to do, but they do so much more from literally being an adult daycare, right?

Because like, you know, kids get to 18, they can't stay at your house. Like they become a socialization place for humans to become mature and become adults to being large scale sports tournaments and sports systems to just being a place for community and research. There is a lot. Now that's also part of the problem, right?

How can you expect an institution to do so much to do so much and also keep up with an insane rate at which skills are changing? So I think the responsibility when it comes to the private sector is to work with universities. And we've already started this. We've already started actively engaging and working with different universities to say, you're amazing at doing so many different things.

But you don't have to carry the burden of assessing skills of allowing practice of figuring out how you scale tutoring across the scale, a software solution, a vendor can come in and become that solution embedded in there, where you can do all the rest, the community, the research did all the amazing things that I'm sure you think back to your university years, and there's so much you think about that you want to keep, but the burden of Building, measuring, evaluating, and developing skills cannot.

continue to lay with traditional education system. It has to be provided and helps through the private sector. 

[00:46:46] Alex Sarlin: I think that's a very fair assessment. So in some ways you're talking about basically what sometimes they call public private partnerships or the bootcamp world has done some of that, or there's sort of been a few different attempts, generations of attempts to sort of combine the, well, the amazing things that universities do with, uh, career focused skills building that has been traditionally done outside of it.

I love hearing you talk about everything that you're doing at CodeSignal and I feel like you have a very, very good opportunity to jump into that space. We are coming off of the news just, just this week that 2U had bought Trilogy Education, which is, is trying to serve exactly as one of those solutions that working with universities, both of them, 2U and, and Trilogy, but 2U is.

You know, has declared bankruptcy. They're now closing down the trilogy bootcamp sector of their company. It feels like there's a really amazing opportunity to sort of reset that university partnership ecosystem. Sometimes they call them OPMs or, you know, there's sort of different aspects of this. And I really hope that it is informed by the kind of hardcore, thoughtful, informed and data driven work that you're doing at CodeSignal.

It's really exciting. So where can people find 

[00:47:59] Tigran Sloyan: Codesignal online? Yes. Codesignal. com, as simple as that. That would be the hub to take you to different places. And we serve many different audiences, right? We serve individuals who just want to get started, want to learn. And it's free too. That's the other thing that's been a core part of the mission.

I've always said, we're going to make free world class education available to everybody in the world, which means the business model has been set up in a way where similar to Duolingo, we don't even have apps, but similar to Duolingo. A few small percent of our users pay, and that paves the way for a majority of them to use it for free with Cosmo with access to every single course we have from business to leadership to technical skills across the board.

But then we also work with universities and companies, and the hub is basically to just go to code signal dot com and see what else we do. But at the end of the day, I think. The opportunity is massive and it's going to take a village and probably more than a village to pull it off. So I'm always very actively talking to partners to say, who else is passionate about fixing this?

Cause I do not want to compete. I want to collaborate because we need more skilled and talented humans and entrepreneurs and executives who want to fix this and just find ways to do it together. And that is also why we've also been building things in a more platform way where it's not just like, Oh, it's mine.

It's great. And I'm going to keep it. It was like, no, you want assessments in your learning experiences. We will deliver it to you. You want tutoring? We'll find a way to make it work in your system as well, so that we can rise all together instead of competing for a smaller pie. 

[00:49:46] Alex Sarlin: That's an amazing note to end on and I think extremely hopeful.

I agree. You know, I feel like you're, we have a shared techno optimist. You see all the issues clearly. But the potential is there and it will take a village. That's why our community of education, technology, and workforce technology, entrepreneurs and experts and investors and operators really needs to sort of be thinking about this collectively.

Tigran Slayan is the CEO of. CodeSignal and the co founder, they've been doing amazing work in skill assessment and skill based hiring and now skill teaching for almost a decade. Thanks so much for being here with us on EdTech Insiders. Thank you, Alan. Thanks for listening to this episode of EdTech Insiders.

If you like the podcast, remember to rate it and share it with others in the EdTech community. For those who want even more EdTech Insider, subscribe to the free EdTech Insiders newsletter on Substack.