AI Doesn’t Learn Alone: Dr. Magdalena H. Gross of Mercor on Human Intelligence

Dr. Magdalena H. Gross of Mercor is a Stanford-trained scholar and a leader in AI education. With a unique ability to understand how people think, she helps experts translate deep human knowledge into AI-ready systems. She designs AI curricula and training programs that empower individuals and organizations, while also leading in community arts and muralism. Learn more about her work at www.magdalenagross.com.

💡 5 Things You’ll Learn in This Episode:

1. How human experts actually train AI through structured thinking and feedback 
2.  Why making “invisible thinking visible” is key to both learning and AI development 
3.  The parallels between reinforcement learning in AI and how humans learn 
4.  How breaking down reasoning into steps improves both teaching and model performance 
5.  Why interdisciplinary thinking is essential for solving complex problems with AI 

✨ Episode Highlights:
[00:02:48] What “human data” really means: experts training AI through deep domain knowledge
[00:05:30] From binary labeling to reinforcement learning: how AI training now mirrors human learning
[00:07:47] Spiral curriculum at scale: how experts and AI both improve through repetition and complexity
[00:10:44] Cognitive modeling: making expert thinking visible to accelerate learning
[00:13:28] Breaking down reasoning: categorizing how humans think across domains
[00:16:28] Why interdisciplinary thinking drives breakthroughs in humans and AI
[00:18:58] The future of human data work: expanding alongside the growth of human knowledge 

😎 Stay updated with Edtech Insiders! 

Follow us on our podcast, newsletter & LinkedIn here.

🎉 Presenting Sponsor/s:

Every year, K-12 districts and higher ed institutions spend over half a trillion dollars—but most sales teams miss the signals. Starbridge tracks early signs like board minutes, budget drafts, and strategic plans, then helps you turn them into personalized outreach—fast. Win the deal before it hits the RFP stage. That’s how top edtech teams stay ahead.

Innovation in preK to gray learning is powered by exceptional people. For over 15 years, EdTech companies of all sizes and stages have trusted HireEducation to find the talent that drives impact. When specific skills and experiences are mission-critical, HireEducation is a partner that delivers. Offering permanent, fractional, and executive recruitment, HireEducation knows the go-to-market talent you need. Learn more at HireEdu.com.

Cooley LLP is the go-to law firm for education and edtech innovators, offering industry-informed counsel across the 'pre-K to gray' spectrum. With a multidisciplinary approach and a powerful edtech ecosystem, Cooley helps shape the future of education.

Transcript

[00:00:00] Alex Sarlin: Innovation in pre-K to grade learning is powered by exceptional people for over 15 years. EdTech companies of all sizes and stages have trusted higher education to find the talent that drives impact when specific skills and experiences are mission critical. Higher education is a partner that delivers offering permanent, fractional, and executive recruitment.

Higher education knows the go-to-market talent you need. Learn more@hireedu.com. That's HIRE edu. 

[00:00:33] Dr. Magdalena H. Gross: And often it takes that moment of thinking outside of the box, outside of the domain or collaboration with other domains to break through and have creative thinking and revolutionize. And I'm thinking even in the medical field, right?

People who have curious cases, I know I've had curious medical cases, they're solved by teams across multiple different. Domains, right? The bone doctors working with the blood doctor, who's working with the heart doctor, who's working with the whatever. So if you think about really solving humanities complex problems, I would imagine it takes the ability for these AI models to be interdisciplinary thought partners.

[00:01:19] Alex Sarlin: Welcome to EdTech Insiders, the top podcast covering the education technology industry from funding rounds to impact to. I developments across early childhood K 12 higher ed and work. You'll find it all here at EdTech Insiders. 

[00:01:35] Ben Kornell: Remember to subscribe to the pod, check out our newsletter, and also our event calendar.

And to go deeper, check out EdTech Insiders Plus where you can get premium content access to our WhatsApp channel, early access to events and back channel insights from Alex and Ben. Hope you enjoy today's pod.

Hello, Ed Tech Insider listeners. Today I'm joined by Dr. Magdalena H. Gross, a Stanford trained scholar and a leader in AI education with a unique ability to understand how people think. She helps experts translate deep human knowledge into AI ready systems. Magdalena designs AI curricula and training programs that empower individuals and organizations.

While also leading in community arts and Muralism, welcome to the pod, Dr. Gross. 

[00:02:28] Dr. Magdalena H. Gross: Hey, nice to meet you, Ben. I'm so honored to be here. Thank you so much. 

[00:02:33] Ben Kornell: I know you and Alex go way back. I'm really excited to learn more about your work and what you're working on. So first, let's start. For listeners who are new to the human data industry, how would you explain it and where do humans fit into training modern AI systems?

[00:02:48] Dr. Magdalena H. Gross: Yeah, so I work for, or I have worked for two of the largest providers of human data operations to labs. So when you think about using ChatGPT and then ChatGPT, it's model four and then model five, and it kind of gets smarter and smarter. Mm-hmm. Behind those LLMs are humans. Those humans are quantum physicists.

They're expert shoppers. They are real estate agents. They are just experts in their field, whatever the field is, and they are working with the models on the backend to make them smarter in that field. Many of them do not have CS degrees, right? They don't actually know necessarily what coding is, but what they've done is they work with a company that I work with like Meco.

There are other companies like Scale or Surge, and they work hourly to make the models smarter. Now the question is, how do you make sure people aren't just like wasting time or doing the wrong stuff or giving it bad information? Well, I lead training programs and certification programs for those experts to learn how to interact with the models before they ever get.

To those big labs. What does that mean? Usually it means starting with incredible conversations with really smart people, hundreds, thousands, sometimes tens of thousands of them, and helping them figure out what they know in the first place. I'll give you an example. I have now a friend who was on an early project with me, he's a stem cell scientist.

He does like fruit fly genetics, and he literally couldn't train the model 'cause he didn't really know how. He knew what he knew. Mm-hmm. And so we had to work on a model adjacent to his actual expertise. And I said, well, if you were gonna work with another insect, what would you do first? What would you do second?

What would you do third anyway? He became an expert tasker. That's what they're often called, annotator, a human annotator or tasker. And he's now actually a research scientist at one of the labs because he got so good at working with the models. 

[00:04:54] Ben Kornell: So I think there's like a stereotype that people are basically being shown this and this, yes, no, it's like a very binary ones and zeros kind of experience.

But what you're describing is a much deeper. Training mechanism between the AI and the human being where there's knowledge transfer and there's scaffolding of knowledge. So you're teaching them to run a masterclass in how experts think, and then having those conversations with ai. Can you tell us a little bit about what viscerally is that experience for one of these taskers?

[00:05:30] Dr. Magdalena H. Gross: It's incredible. So great question. So what you referenced is called preference labeling, and in some cases people are still doing that. Some labs and in some fields. And the fields. Remember, these LLMs aren't growing linearly. In every field. So there are some fields that are, the LLMs are a little better in them than in others, mostly because of their access to experts in those fields.

What we have now is we've moved mostly from preference labeling or annotation or tagging to something that most people call some form of. Reinforcement learning. And what I love about that term is it's actually how humans learn. So what they're calling an AI reinforcement learning, and there's many different ways the labs all do it in their own proprietary ways, but if you think about just that term reinforcement, a little baby is trying to like.

Get up upstairs, you know, and it's wobbling, and then you're saying, great job. Great job. Oh no, don't do that. Or whatever it is. You're doing the same thing with an AI model. You're having conversations, very complex conversations with a model in a particular environment. Right, and then you're going back to those answers and you're saying, yes, this response was excellent in this, this, and this way.

This response missed the mark in this, this, and this way. And you do it hundreds and hundreds, sometimes thousands of times, so that the model is then generalizing and learning from tons and tons of conversations and data that's like, I'm not a Cs. Person. So that's my understanding on the back end of what's happening and what I'm teaching experts to think about doing.

[00:07:07] Ben Kornell: It's so incredible because it's so parallel, like you said, to how teachers teach students and this idea of giving rigorous feedback, it's not only introducing conceptual knowledge and scaffolding like complex thought, but it's also reviewing the outputs and then giving feedback and saying a little bit more of this, a little bit or less of that.

Here's where you got a concept. Exactly right. Here's where you caught it wrong on the micro level. I totally understand how that works, but you've actually worked at scale where it's not just a handful of people, it's hundreds if not thousands of people doing this type of training. 

[00:07:44] Dr. Magdalena H. Gross: Yeah. 

[00:07:45] Ben Kornell: What have you learned from that experie?

[00:07:47] Dr. Magdalena H. Gross: I've learned that much like in any classroom, both AI and the experts training it learn through a spiral curriculum. Jerome Bruner doing the same thing over and over again at increasing levels of complexity. And if you allow. Tens of thousands or hundreds of thousands of folks to low stakes try out this kind of training and response before they, it ever goes public, meaning it's ever data that's sold or whatever.

I can't speak to it directly 'cause every lab does it a little bit differently. But if you have a low stakes environment where you allow them to practice those conversations first, before they go and input those conversations into the model. You end up decreasing their error time and increasing the quality of those conversations almost exponentially.

[00:08:40] Ben Kornell: Mm-hmm. And so now each model advances significantly every few months. Has the playbook been roughly the same, or have you had to evolve? 

[00:08:53] Dr. Magdalena H. Gross: I've had to evolve. And what happens too is remember, I am not actually on a research team. The research teams often evolve first and then say, Hey, this thing has either hit a wall or we need to advance the model faster, and we're thinking about doing it in X, y, and Z way.

Can you try it? And so it's an iterative process and it comes back to those of us in the human data world, and we try new things and new kinds of trainings. And sometimes also it's different based on the field, the domain. So mathematics, anything that has a lot of numerical inputs and outputs is a little bit different than humanities, which is different than the sciences.

The fields right now is again. This is, don't quote me 'cause I'm not the computer scientist, but what it seems like to me is that the fields that are advancing the fastest are the ones that still have answers that are unequivocal. So think of a chess, so math 

[00:09:57] Ben Kornell: versus like a 

[00:09:58] Dr. Magdalena H. Gross: language arts. Versus the language arts, which can be open and interpretive.

We're still struggling to understand what does that mean? And I think that's where human ingenuity and human creativity will always triumph. Because of course that's right now at least it's very hard to like program. 

[00:10:15] Ben Kornell: Mm-hmm. So from a training standpoint, there's a lot of ingestion of data sets. And that's a non-human interactive layer.

But then there's this like expertise layer where there's experts training, and then there's just average human quality control and checking. How much of your work focuses on that standard human, or are you pretty much uniformly in the expertise area? 

[00:10:44] Dr. Magdalena H. Gross: Both. So I think what's really important is as you teach experts to make the invisible visible, to make their thinking visible, it's basically cognitive modeling, right?

So Sam Weinberg and I at Stanford, 

[00:10:57] Alex Sarlin: yeah, 

[00:10:57] Dr. Magdalena H. Gross: I learned under him, and he has this framework of cognitive modeling. It's not exactly his, but it was coined in the nineties where you think out loud about what you know. And it turns out when experts do that, novices can learn much faster. So same principle. Once folks get very good at making the invisible visible, they typically become good at checking other people's work too.

So it's often one and the same. That kind of the expert that's making great quality data, we're then later typically advancing into a reviewer. Role where they're going back and reviewing and making sure that other people's submissions are excellent too. Something I like the World Economic Forum says, you know, a lot of jobs are gonna be lost, but there are gonna be all of these jobs.

We don't even know the titles of them that are gonna be created. And I feel like Reviewer of Human Expertise is one of those jobs that even when I started in this field in March, which by happenstance I landed in, you know, we didn't even have, it was really labs. That we're reviewing and researchers that we're reviewing.

But researchers aren't fruit fly geneticists, right? Right. So then they were saying like, Hey, can you have the geneticists also just review? Because they're all reviewing for slightly different things. So yeah, it's a flywheel. Is that what it's called? When you kind of are internally. Creating jobs and then going back and making the jobs you already created better and so on.

Yeah. 

[00:12:24] Ben Kornell: Right. And this idea of moving from like intuitive knowledge to metacognition, I think that strikes me as. Both like essential and also could make AI even better because if AI can explain how it rationalized to get to something, you can find the errors in the logic. 

[00:12:46] Dr. Magdalena H. Gross: Yes. 

[00:12:46] Ben Kornell: How does that connect to the work that you've done and how have you been seeing this in practice?

What are some stories that resonate around this idea of, you know, intuitive or automatic learning and making it visible? 

[00:12:58] Dr. Magdalena H. Gross: So one of the really interesting terms that happened, I'm thinking about this same researcher now that I worked with. It was just really fun to develop materials with him as we started creating matrices of reasoning in all the domains.

So we started categorizing. Physics reasoning. 10 different kinds of reasoning. Spatial reasoning, numeric reasoning. You know, hypothetical reasoning, this kind of reasoning. 

[00:13:28] Ben Kornell: You know, like in math there's geometric and algebraic and combinatorics. These are all. Frameworks of thinking. Yeah. 

[00:13:37] Dr. Magdalena H. Gross: Frameworks of thinking.

Legal reasoning has 500 different kinds of reasoning, right? I'm not an expert in all of those. But then you would find kind of team leads within domains who would create these matrices because you still need to be able to identify and tag did. Where did the reasoning breakdown, you know, how many steps of reasoning did this model do?

How many did I do? You know, what kind of reasoning am I doing intuitively? What reasoning is being layered? And so what would happen, especially in the early days when I was working with a lot smaller groups of experts, is they would write me testimonials, emails, sometimes slacks. I would say, oh my God, I went back to my freshman class that I'm teaching as a PhD student, as a TA, or an early prof, and I completely revamped my curriculum.

'cause I didn't even know that it took this many steps of thinking to get to this answer. Or, you know, this. Quantum physics idea, or whatever it was. So while these are vendors often, you know, who are working in their spare time 10 to 40 hours a week for an AI lab or for a vendor, they're really in an ideal situation.

They're learning about what they know, how they know it, and they're taking it back to their research practice and they're taking it back to the classroom. And they're that much more able to help others learn quickly as well. Mm-hmm. That's kind of when it works really well. 

[00:15:08] Ben Kornell: I don't know if you can share this.

I imagine you've worked with different models. What's the difference in terms of working with them? How do you think about knowledge transfer and training maybe differently with an open AI versus an Anthropic versus a Google, or is that something you can't go into? 

[00:15:26] Dr. Magdalena H. Gross: I can't really go into it, but also I often am even farther top of.

Meaning a lot of the time, the place where my initial training programs and certification programs come in is even before they're assigned to any kind of a client. And it could be a lab or somebody else. And so think of it even before where they're getting the kind of ab. Seas of human data training and then later being funneled into like a domain world where they're being vetted by some domain specialists, and only then are they being passed off.

It's pretty rigorous work. 

[00:16:06] Ben Kornell: So one of the areas that you've been passionate about is just bringing interdisciplinary thinking into AI and the future of ai. A lot of the folks that you're working with, like you say, are not computer coders. Why is that so important to you and why do you think that's important for this moment that we're in where AI is changing everything?

[00:16:28] Dr. Magdalena H. Gross: I think in general, what interested me my whole life. Is folks who have thought really broadly about narrow topics, people who have brought in other perspectives and therefore revolutionized their fields. Anyone from Albert Einstein to. You know, even if you think about Sam Weinberg's work, he wasn't really even working in history, education.

He was thinking about how do people think? And often it takes that moment of thinking outside of the box, outside of the domain, or collaboration with other domains to. Breakthrough and have creative thinking and revolutionize. And I'm thinking even in the medical field, right? People who have curious cases, I know I've had curious medical cases, they're solved by teams across multiple different domains, right?

The bone doctor working with the blood doctor, who's working with the heart doctor, who's working with that? Whatever. So if you think about really solving humanities complex problems, I would imagine it takes the ability for these AI models to be interdisciplinary thought partners. It's just a hypothesis.

I could be wrong. 

[00:17:42] Ben Kornell: No, no. But this idea of basically connecting the dots between all of these different disciplines, which in fairness is, you know, we're often limited. I mean, healthcare's a great example where when it's fragmented or fractionalized, you get worse outcomes and. Many times it's human, human time and space that gets in the way of bringing those different perspectives.

There's an opportunity for ai, one to connect the dots, but two, maybe to ask better interdisciplinary questions that allows you the consumer of the AI to go get those answers and to go connect those dots. 

[00:18:18] Dr. Magdalena H. Gross: A thousand percent. I think actually Perplexity AI does some of that, right? They might. It says something like, I don't know the answer to that, but here's what I would think about.

I love that. I love that kind of approach, right? 

[00:18:30] Ben Kornell: Yeah. We've seen this be a real boom in economically, that idea that. AI needs all of this training inputs and that there needs to be human trainers scale. AI famously acquired by meta, basically with the idea that it's all BA to train meta's ai. Long term, do you feel like this is going to be an enduring industry and an enduring economic opportunity, or is it like a narrow.

Yeah, 

[00:18:58] Dr. Magdalena H. Gross: I actually don't know the answer to that. My, my suspicion is that it's fairly infinite just because human knowledge is pretty infinite. And so as the AI systems get more complex and they're able to work together more in agents and you know, it really does become more interdisciplinary. You can't see why we wouldn't just kind of infinitely grow.

Right. I'm thinking of the Renaissance. I mean, anything that's happened after major technological revolutions historically of. It has been some kind of surprising creative moment in human history, so I can confidently say I don't know, but I can't see why not. 

[00:19:37] Ben Kornell: Well, I'm very excited to keep following this trend.

Thank you, Dr. Gross, for joining us today here at EdTech Insiders. Fascinating conversation and I feel like you're actually bringing so much value to all the store to human knowledge that we all have. I think it's very validating for any of us who've. Many years in, in deep academic pursuit. So thank you so much, Dr.

Gross, for joining us. If people wanna find out more about your work or connect with you, what's the best way for them to reach out? 

[00:20:08] Dr. Magdalena H. Gross: I just launched a website of my own. I think there might still be some lorem ipsum on it, but I'm working on it. And it's magdalenagross.com or my LinkedIn profile, which is.

LinkedIn, Magdalena H. Gross, I think. 

[00:20:22] Ben Kornell: Wonderful. 

[00:20:22] Dr. Magdalena H. Gross: So yeah, reach out to me. I'm usually really, really fast. I'll respond. I found myself in this field serendipitously, so I do wanna share what I know. 

[00:20:31] Ben Kornell: Awesome. Well thanks so much, Magdalena for joining us today and Ed Tech Insiders. Check her out on LinkedIn or magdalenagross.com.

Thanks so much for joining us. 

[00:20:40] Dr. Magdalena H. Gross: Thanks. 

[00:20:42] Alex Sarlin: Thanks for listening to this episode of EdTech Insiders. If you like the podcast, remember to rate it and share it with others in the EdTech community. For those who want even more, EdTech Insider, subscribe to the Free EdTech Insiders Newsletter on substack.

This season of EdTech Insiders is brought to you by Cooley. LLP Cooley is the go-to law firm for education and EdTech innovators offering industry informed counsel across the pre-K to gray spectrum. With a multidisciplinary approach and a powerful EdTech ecosystem, coolly helps shape the future of education.

This season of EdTech Insiders is brought to you by Starbridge. Every year, K 12 districts and higher ed institutions spend over half a trillion dollars, but most sales teams miss the signals. Starbridge tracks early signs, like board minutes, budget drafts, and strategic plans, and then helps you turn them into personalized outreach, fast, win the deal before it hits the RFP stage.

That's how top Ed tech teams stay ahead.