Building an AI Tutor That Gets Students to Think with Sue Khim of Brilliant

Sue Khim is the Co-founder and CEO of Brilliant, the interactive learning platform that has helped millions of learners build deep understanding in math, science, and computer science. In this episode, she shares how Brilliant's new AI tutor, Koji, is designed to develop critical thinking rather than simply provide answers.

💡 5 Things You'll Learn in This Episode

1. Why the best AI tutors ask questions instead of giving answers.
2. How Brilliant applies active learning and game design to STEM education.
3. The philosophy behind designing AI that builds intelligence rather than dependence.
4. What personalization should look like in the next generation of AI tutoring.
5. Why learning should feel like a "climbing wall" instead of an answer machine.

✨ Episode Highlights
[00:02:32] The origin story of Brilliant and its active learning philosophy
[00:05:31] Introducing Koji: an AI tutor designed to help students think.
[00:11:07] Why concentration, challenge, and flow are essential for learning
[00:15:48] Designing AI that guides students without giving away the answer
[00:19:40] Early reactions to Koji and why parents are embracing this approach
[00:25:30] What game design can teach us about building better learning experiences.
[00:35:05] Sue's vision for the future of personalized AI tutoring.

😎 Stay updated with Edtech Insiders! 

Follow us on our podcast, newsletter & LinkedIn here.

🎉 Presenting Sponsor/s:

Every year, K-12 districts and higher ed institutions spend over half a trillion dollars—but most sales teams miss the signals. Starbridge tracks early signs like board minutes, budget drafts, and strategic plans, then helps you turn them into personalized outreach—fast. Win the deal before it hits the RFP stage. That’s how top edtech teams stay ahead.

Tuck Advisors was founded by entrepreneurs who built and sold their own companies, frustrated by other m and a firms, they created the one they wished they could have hired but couldn't find. One who understands what matters to founders and whose North Star KPI is the percentage of deals closed. If you're thinking of selling your ed tech company or buying one contact Tuck advisors now.

Innovation in preK to gray learning is powered by exceptional people. For over 15 years, EdTech companies of all sizes and stages have trusted HireEducation to find the talent that drives impact. When specific skills and experiences are mission-critical, HireEducation is a partner that delivers. Offering permanent, fractional, and executive recruitment, HireEducation knows the go-to-market talent you need. Learn more at HireEdu.com.

This season of Edtech Insiders is brought to you by Cooley LLP. Cooley is the go-to law firm for education and edtech innovators, offering industry-informed counsel across the 'pre-K to gray' spectrum. With a multidisciplinary approach and a powerful edtech ecosystem, Cooley helps shape the future of education.

Transcript

[00:00:00] Alex Sarlin: This season of Edtech Insiders is brought to you by Starbridge. Every year, K-12 districts and higher ed institutions spend over half a trillion dollars, but most sales teams miss the signals. Starbridge tracks early signs like board minutes, budget drafts, and strategic plans, and then helps you turn them into personalized outreach fast.

Win the deal before it hits the RFP stage. That's how top edtech teams stay ahead. 

[00:00:31] Sue Khim: A good tutor should not be an express train to the answer. A good learning product should not be spoon-feeding you insights. We think about it as more like a climbing wall. We are giving you holds. We are giving you a route.

Maybe the tutor can add some more holds if you need them on the first few climbs, but you still have to climb. So we want the tutor to ask the student to be an active participant. And so you'll see the tutor asking the student to reason through things themselves, try and test ideas, draw things, make predictions, explain how they got where they are, and design an experience that keeps the student in the work

[00:01:15] Alex Sarlin: Welcome to Edtech Insiders, the top podcast covering the education technology industry. From funding rounds to impact to AI developments across early childhood, K-12, higher ed, and work, you'll find it all here at Edtech Insiders. 

[00:01:31] Ben Kornell: Remember to subscribe to the pod, check out our newsletter and also our event calendar.

And to go deeper, check out Edtech Insiders Plus, where you can get premium content, access to our WhatsApp channel, early access to events, and backchannel insights from Alex and Ben. Hope you enjoyed today's pod.

[00:01:55] Alex Sarlin: Welcome to Edtech Insiders. We are speaking with a terrific guest, really a legendary edtech founder who we've never talked to on the podcast before. We are talking to Sue Khim, the co-founder and CEO of Brilliant. Brilliant is an AI tutor that helps students actually think, and it's a community of STEM and math aficionados all over the world.

She will tell you all about it. Born in South Korea, she moved to the United States as a child and grew up in Chicago, and later earned a degree in mathematics from the University of Chicago, which stemmed some of this incredible work with Brilliant. Sue Khim, welcome to Edtech Insiders. 

[00:02:30] Sue Khim: Thanks for having me, Alex.

It's really great to be here. 

[00:02:32] Alex Sarlin: It's great to chat with you. So before we get into all of the incredible AI work you are doing right now, give us a little bit of the origin story of Brilliant. You are a math person. You realized math education really wasn't doing what it could be doing, and Brilliant has created opportunities for millions of kids to learn math at very high levels.

Tell us about what it looks like. 

[00:02:51] Sue Khim: Yeah. You did attribute a degree to me that I don't have. I didn't actually graduate with a math degree. I dropped out midway to start the company. 

[00:02:59] Alex Sarlin: Gotcha. Did not know that. 

[00:03:01] Sue Khim: Yeah. And at the time, we were really focused on creating avenues for students to be able to do active learning, and we were inspired to think about, now that everyone has the ability to do active learning, which is so much more impactful, how might we create interactive world in which students can actually learn math, science, computer science in the way that it was meant to be learned, like in a very hands-on way.

[00:03:32] Alex Sarlin: And so Brilliant was really designed around that. It has a whole series of different courses in math, science, computer science. They go quite deep into the conceptual understanding behind some of these things, and then are also highly interactive. They are not multiple choice. They're really designed for people to be virtual manipulatives, and looking at patterns and trying to figure things out.

What is the sort of pedagogical philosophy behind the active learning inside Brilliant? 

[00:03:56] Sue Khim: Yeah. We've always been focused on getting people to actually think. There are a lot of parents at Brilliant, and there is this wonderful, exciting world of math that someone introduced to us at some point in time, and that we want to introduce to students.

And it is not this drudgery that we do every day because we're forced to. It is something where, taught properly, it is the most interesting thing in the world to think about. And so I think the initial product came out of a shared passion from the team to help the world see math as problem-solving and active learning.

[00:04:37] Alex Sarlin: It makes sense, and it comes right out of the research literature, especially about math and physics learning. Carl Wieman famous work about active learning. But the idea that math education can be so drill and kill, it can be so formula-based, it's really about going through the motions, almost like doing your reps.

And mathematicians are like, "That's not what we do." Yeah. We don't do our reps, right? We don't do problem sets as mathematicians. We discover, we, like, explore, we go really deep into these unknown problems that these-- problems that have been unsolved for 100 years that people have been working on. It's like it's such a different mindset.

It makes a lot of sense. And then you are now launching a really, really interesting and very also active learning pedagogically based AI tutor named Koji that's come out, I think you just announced it a few weeks ago, and it's like absolutely the big new thing on the block in Brilliant because it builds exactly on your philosophy and on your data and on your system.

Tell us what Koji is and why you're excited about it. 

[00:05:31] Sue Khim: Yeah. The adoption has been crazy. So we launched an AI tutor a few weeks ago, and it's the first AI tutor that gets kids to actually think. And this was very explicitly the goal that we built toward as we were working on this AI tutor. And the way we thought about it was that kids have had access to tools that can give them the answers for a long time.

This part is so solved. More solutions are not needed here So we had pre-AI homework solvers like Wolfram Alpha, that's been around for a long time. Then you had crowdsourced answers from sites like Brainly, and then you had PhotoMath, you could just take a photo. And now you have ChatGPT, you can paste in a math problem, you can get a step-by-step solution instantaneously.

And the thing that we felt really strongly about when we were building an AI tutor is that the promise of AI for students is so much more than this. When you use AI as essentially an answer vending machine, what did you actually learn? Like, you are skipping some really important parts, and a key message of our launch is that AI could be making our kids into geniuses.

The really important part that we wanted to protect in this era of AI answers is that time and effort that you spend thinking for yourself. You stare at the problem. You don't know what to do yet. You try something. That doesn't work, so you stay calm and you- ... think a little harder about, okay, what do I know?

Why am I stuck? What else can I try? And maybe after minutes of thinking, you notice a new pattern and you try something else. That's the thing that we were most worried about losing with AI because it's now so easy and instant to skip all of that But as we were talking a little bit about before this started, the nice part about AI is that it is a really general purpose technology.

So you can use it to spit out an explanation, or you can use it as a tool to train the part of your mind that thinks. With Koji, we were very intentional about that. We didn't want a tutor that just explains something at you and then asks you to repeat it back. 

[00:07:47] Alex Sarlin: Mm-hmm. 

[00:07:47] Sue Khim: We thought that it should feel like an experienced tutor sitting next to you saying, "Okay, what do you notice?

What have you tried? Can you draw it? Can you test a smaller case? What would happen if this number changed?" And it's giving the kind of help that keeps the student doing the thinking. So that's what we mean when we say get kids to actually think. We don't want the AI to perform the intelligence for the student, we want to build the intelligence in the student.

[00:08:16] Alex Sarlin: Absolutely. And we've just been reviewing some of the research on AI and mathematics and AI tutors and mathematics, and you are so aligned, so unbelievably aligned with what all the research is suggesting and saying, and what people are finding that there's all this performance effect where people do better on tasks when they have access to AI, and as soon as they lose access to it, their performance goes down because they have created what researchers call, like, a crutch.

And there's need for expert pedagogical insights to be baked into the AI and it to be designed with instructional design. Like, so many of the things you're saying, and of course goes without saying, giving the answer deprives students of all of the thinking process, and the metacognition, and all sorts of things.

They just don't even get to make sense of what's going on. So you are very well aligned there. I wanna double-click on one thing you said, because I think it's a really important part of Brilliant's story, at least from my understanding from the outside, which is that a lot of EdTech products are sort of designed to raise the floor, right?

They're designed for the lowest common denominator, maybe for remedial education or for every student, and has all sorts of supports and scaffolds. Not that Brilliant doesn't do this, but I think Brilliant has always been built, it's called Brilliant, and it's really about genius. It's really about raising the ceiling as well as raising the floor.

And I think it's an interesting stance that I think you've taken. You mentioned that you've gotten students into MIT at 15. There's, like, helping students excel beyond their wildest dreams, and I'd love to hear you talk about that a little because I think it feeds into this, exactly this concept of what AI tutors can unlock.

[00:09:41] Sue Khim: Yeah. No one's ever asked me that question before. As much as Brilliant was started by people who in fact did become mathematicians, one of the things that we really believe is that math should not be taught because it is practically useful. Like, most kids are not gonna factor polynomials at work. Like, there is just no job that requires this of you.

Vanishingly few kids are going to become mathematicians, and we feel that this is extremely freeing when you tell people the point of learning math isn't because you need to learn math. The point of math is because it trains a part of your mind that's really important, and if it's carefully structured, it is a great challenge.

It's teaching you to reason. It's teaching you to decompose problems. It's teaching you about abstractions and about what to do when the path isn't obvious. And it's very different from how most kids learn math in school, which is it's really important for you to learn all these formulas and apply all these formulas.

And we didn't set out when we started this company to get kids into MIT when they were 15. Like, we were not focused on that outcome. We were focused on teaching a new way to think about math as a problem-solving gymnasium. 

[00:11:06] Alex Sarlin: Mm. 

[00:11:07] Sue Khim: And I think when you put kids in that environment, one of the maybe very controversial things that we believe is that kids love to concentrate.

Hmm. And like, maybe that doesn't seem true when you look around you and see all the brain rot and see the mindless scrolling and all of the ways that technology has been wearing away at our attention, but something that we talk about a lot internally is that the reason that games are so popular with students isn't because they're entertainment.

It is because they are a form of managed concentration, and that feels really pleasurable if you execute it well. And when we talk about learning and getting kids to go much farther than they ever thought that they could- And we talk about creating that environment of flow and excitement and having a conversation with this rich material.

It's not gamification in the sense of bing bongs and game mechanics. It's not like children as Skinner boxes. It is managing that state of struggle and flow and concentration so that you see this activity as something that transports you into a mental state that you can feel yourself on the cusp of being able to do something that you couldn't do before.

And then when you finally overcome that next obstacle, it feels amazing, and you're ready for the next one. And the entire journey of Brilliant has been about building that path brick by brick, and then letting students go through it at whatever pace makes sense for them. And some students, there's a lot of gaps.

We need to focus a lot on reviewing things that they may have missed many grades before. And other students find some area, maybe it's geometry, maybe it's algebra, they get really immersed in it, and then suddenly you have the fourth grader who's doing eighth grade material. And we don't think about this in terms of Grade levels and ages.

I think we consider it, when you buy a game, it doesn't have an age range on it usually. That's right. And we think about it as we should build the best learning dojo- Yeah ... and then invite all comers, and we want everyone to succeed to the potential that they have and to create a great learning environment for each individual.

[00:13:32] Alex Sarlin: That's really one of the most exciting and cohesive philosophies of learning, especially in math, but in any subject really, that idea of a learning gymnasium, a learning dojo, not leveling by grades necessarily, but just creating an environment where people can concentrate, can immerse, can go deep, can make sense of subjects, can have that thrill, that dopamine high, but not the negative dopamine high, not the, like, addictive dopamine high, the true deep dopamine high of solving a problem they didn't know they could solve.

That's like why we have dopamine in, in many ways, right? This feeling of, "Oh my God, I didn't know this was possible, and I made my way through. I worked at it. I feel incredibly accomplished, and now give me more. I just can't..." That growth mindset, right? And that's a wonderful philosophy. And as you say, when you create an environment like that, it doesn't have an age limit, it doesn't have prerequisites, and it's designed not to be punitive so people aren't put off by it and say, "Oh, I tried a few problems.

I didn't get them right, and I'm gonna go do something else. I'm gonna go back to my scrolling." They say, "Okay, wait, I'm into this. I'm in it. My concentration is high. My excitement is high. I feel like I'm actually there." We can talk about the game elements of this for days. This is, like, one of my favorite subjects, but when you look at really deep games, what they do is they start surface level.

They give you little chances to master certain mechanics, and then they go deeper and deeper and deeper. And after a while you're, like, so immersed that there's like 100 things happening on the screen in some of these games, and students are managing every single one, or players I should say, are managing every single one.

And I feel like in an ideal world, that's exactly how it feels to learn. It's an exciting vision. 

[00:15:03] Sue Khim: Totally. We think that success looks like students coming out of the experience really believing, because they've done it so many times, that- They are unafraid to do hard things. 

[00:15:15] Alex Sarlin: Yeah. And then I think a relevant piece of this, it may feel like a little bit of a curved segue here, but I think it's super relevant, is because you built this learning dojo, this gymnasium where so many students, you have millions of students have used Brilliant over the years, you have this incredible data set of problems, of attempts, of pathways, of choices, of all this data that comes from all of these students engaging in all of these ways, which becomes an incredibly powerful training set for Koji.

I'd love to hear how you have used that proprietary data to make something that is unique in the field. 

[00:15:48] Sue Khim: Parents, I think, are very worried about the habits that products create in their kids. You know, like the default use case for AI for students right now is cheating. And so students use it to write their essays, solve their problem, make this easier, and they see it as a shortcut machine.

And when we designed Koji, we thought a lot about behavior. And we knew that watching students go through the product, it definitely increases engagement to make things easier, at least in the short term. Like the usage metrics are gonna go way up And so a very simple example of a kind of design decision that we had to make is what do we do when a student says, "Just tell me the answer?"

[00:16:31] Alex Sarlin: Mm-hmm. 

[00:16:31] Sue Khim: We've gone back and forth with them several times, and they're like, "Just tell me the answer. I just need the answer." And the easiest thing to do would be to tell them. It feels really helpful in the moment. It's definitely gonna make engagement go up. It might even make the product feel magical, but for us it's the wrong kind of magic because it's not the goal that we have for our learners.

A good tutor should not be an express train to the answer. A good learning product should not be spoon-feeding you insights. We think about it as more like a climbing wall. We are giving you holds. We are giving you a route. Maybe the tutor can add some more holds if you need them on the first few climbs, but you still have to climb.

So we want the tutor to ask the student to be an active participant, and so you'll see the tutor asking the student to reason through things themselves, try and test ideas, draw things, make predictions, explain how they got where they are, and design an experience that keeps the student in the work. And the value of the data that we get is we already know in the curriculum itself, we've tested a lot what are the best ways that we can do that.

And now with all of the tutoring session data that we have, we are getting better and better at our ability to keep students in a zone where they're the ones doing the thinking. 

[00:17:49] Alex Sarlin: So many points that just resonate so deeply with me, but the phrase that I really love here, because I feel like it spans the entire tension of AI right now, is that idea of like the wrong kind of magic, the right kind versus the wrong kind of magic, because it does feel magic.

And we're all still getting used to the magic of AI, the magic that it can answer any question. 

[00:18:07] Sue Khim: Yeah. 

[00:18:08] Alex Sarlin: I mean, that's the magic. That's why ChatGPT was the fastest growing tech tool of all time because it was like, this thing can answer any question. No matter what you ask it, it can give you an answer. That is a kind of magic, and we have to admit that.

But that's not what teaching magic looks like. That's the exact wrong kind of magic when you're trying to learn something. Yeah. And I think the field is trying to really get its head around that because it's complex, and students at any given moment are often choosing to take the general purpose tools.

They're choosing to go to ChatGPT or Gemini or Claude or Argoth, right, out of the App Store and say, "Give me the answer. You can give me the answer. This is magical. This is amazing. It's saving me so much time." Like, you can easily understand why they feel that way. At the same time, everybody in education says, "Hold on a second."

So I, I love hearing you talk about having all of this data, the tutoring data, the curricular data. Some people call it grounded in curriculum, right? That's a popular phrase. Your AI is grounded in your core curriculum, meaning, like, that's the ground truth, and that's incredibly powerful. There's so many insights in what you're doing that I think are relevant to the entire field right now.

And as you've seen uptake over the last few weeks, what has the response been? You have a huge community of really brilliant Brillianters. I don't know what you call them, but people on the Brilliant platform. Brilliantines? I don't know what you call them. But when they're suddenly seeing this Koji tutor, which is literally engaging with them, watching what they're doing inside the platform, seeing where they're putting that line, seeing where they're putting that circle, seeing how they're balancing that fraction or equation, and then saying, "Hey, may- you might wanna try this," or, "Here are some ideas," what has the reaction been?

[00:19:40] Sue Khim: Great question. We were very nervous about launching this product. We were gonna launch when it was ready to go. That was the Friday of a week where at least three commencement speakers were booed off the stage- Yes ... for mentioning AI. And there was this, like, sinking feeling of, oh man, consumers hate this.

Teachers are very worried. AI is deeply uncool and unpopular- ... and everyone's going on summer break. What kind of tech company launches on a Friday? And these are the conversations that we were having, like, 10 minutes before I pushed the button, and I was like, "Well, whatever. We're gonna have to push the button, so let's just do it."

And the response was so much greater than what we thought it would be. And I think that what we learned is that a positive view of AI in learning really resonates with people. I think that people have a very clear sense of how AI is making our kids dumber right now, and there's been a lot of concern about that.

And this idea that there is a concrete positive vision for- Mm-hmm ... how can AI create incredible learning outcomes for students. Like, how can we harness this technology for good? Parents know, students know, like, everyone understands very deeply that this shortcut is a shortcut to nowhere. Like, what did you shortcut?

There is no value to having a better and better homework machine. As you were saying, like, anyone who works in education looks at these new tools for finishing your homework, and they're like, "Wow, this doesn't solve anything at all." Like, this- Right ... actually makes my job a lot harder and a lot worse and is making it harder for students to understand that the effort is so much of what teachers are trying to elicit And so I think that we were very pleased by how people responded positively.

And another thing that we thought might be some skepticism that came with the launch is, you know, chatbots can already provide Socratic hints. You know, there's lots of stuff that generates quizzes. Will people be able to tell? One of the things that we say a lot internally is people can tell. Consumers can tell, learners can tell.

They can tell when something was crafted with care. They can tell when a pedagogical sequence makes sense because people can tell when they actually understand something. And this launch was gonna be the true test of that. Are people going to be able to tell the difference between Cogi and everything else that claims that it is going to teach you?

And, you know, we saw so many very powerful models where they were prompted to be Socratic. If you do that, it will sound Socratic. It'll ask questions, it'll explain things, it'll generate some questions, and that is definitely not the same thing as a learning experience. Like, if you ask a teacher who has taught the subject, "Was this good?"

They will say, "No, and it's really subtle, but I promise you the student is not gonna learn from this." A tutor is constantly making decisions about the student's next best move. It's not, can I explain this to them better? It's, what is the student confused about? Are they ready for a hint, or should I let them sit with it for another minute?

Is this a small computational mistake, or is there a deeper misconception here? Should I ask them to draw a picture, or should I give them a simpler version of the problem? And that judgment is the pedagogy that we're building into product. This is a very complicated thing to explain to a consumer who does not spend all day thinking about what is the best way to teach math.

And our bet was we're gonna build all this into the product, and somehow people will be able to tell. And I think that we were vindicated in that. And all of the years that we've spent building these incredibly tailored, crafted interactive learning experiences where the student is doing that problem-solving, they're manipulating things, they're making predictions, they're testing their ideas, it did shine through as Cogi is not chat plus math.

And we have been getting so much email and DMs from parents who do see Cogi as this coach that is integrated into a learning environment that was already carefully Designed. So I think that that has been the most rewarding part of launching is really vindicating that people can tell the difference.

[00:24:16] Alex Sarlin: That's really amazing to hear. As I hear you talk about what's a, is a shortcut to nowhere, one metaphor that people sometimes use, I always find really interesting, and I feel like it's very illustrative in some ways, is like, you know, if you went to the gym and you want a personal trainer, well, the last thing you want is the personal trainer to start lifting the weights for you, right?

Yeah. 

[00:24:35] Sue Khim: Or 

[00:24:35] Alex Sarlin: start doing the stretching. Like, as soon as they do that, it's a shortcut to nowhere, right? If you're like, "I'm trying to do 10 reps, but you know what? I'm, I'm tired. I'm gonna have my robot physical trainer do the 10 reps for me." It's like, hold on a second. You have reified- Well, you 

[00:24:47] Sue Khim: lifted the weights, but did you Yeah, the 

[00:24:49] Alex Sarlin: weights were lifted, but why were the weights supposed to be lifted?

Yeah. What was that for? It's the same thing with homework, right? I mean, why were you doing a problem set? It wasn't to get those answers because those answers are gonna be so important in saving the world. It's, like, it, those answers are all practice, and they're all about your own brain. And I feel like there's something really funny about that metaphor, but I think it's so relevant to what you're saying there because what is it, you, you mentioned the climbing wall metaphor before, a similar kind of thing.

It's like if somebody's trying to get somewhere, you need to create an environment that lets them get there. It, getting them there, giving them, warping them to the top of the climbing wall gets you nowhere. I feel like I say it, I'm like, "Yes, of course," but at the same time, I don't think this is how a lot of people think.

[00:25:30] Sue Khim: Yeah, you have to build for the medium, you know? An AI tutor is not a human tutor. I think that there are things that human tutors do beautifully that you should seek to borrow in an AI tutor. You know, you should try to notice and listen. You should be able to tell when a kid is pretending that they get it, but they don't actually get it.

Right. You know? You should know here's when I encourage them, here's when I push them. And so, you know, there are things that human tutors do that AI tutors absolutely should learn from. But AI is, like, fundamentally a different medium. Like, digital medium is completely different from a human sitting across from you talking because you can have the conversation within a graphical learning world that is responding to you.

And you had a great visual picture there. We also often talk about kids working on geometry. Very important to have the visual there. And instead of a tutor saying, "Think about the angle," they can facilitate observing a student who's dragging the points around, seeing what changes, you know, ask them to make predictions, and then they can watch the visuals react, and you can facilitate them playing with the idea.

And this is where we think, like, games are a much better reference point than lecturing or one-on-one human tutoring. A good game would never say, "Okay, let me just give you a two-minute explanation of how to beat this level." It would figure out, okay, how can I construct the appropriate next thing for this player to do?

It would give you feedback. It would give you more attempts. It would make it feel safe to fail if you're really struggling. It would figure out, like, where is the edge of this person's ability? This is only possible now, and I think it's a really beautiful model for learning, where a human tutor could not do this one-on-one every day for every student.

And I think the best AI tutor is able to provide a fundamentally new kind of learning experience 

[00:27:34] Alex Sarlin: Yes. Makes me think of two things. Bear with me for a second, I swear these will be relevant. One is, you know, if we look back at the model of the climbing wall or the physical trainer, the AI physical trainer, it's like, so what is the move if you're a, you know, a robot standing next to a person, and the person is doing six reps and they need to be doing 10, and they're just like, "I just don't know.

I just can't do it." Like, what can you do to keep that person engaged, motivated, excited, immersed, and actually, you know, try harder? Well, what you probably are gonna do is give them some incentive, give them some structure, and say, "You know what? Let's think about it differently. Let's imagine you're lifting up your child to put them on the top bunk.

Let's make it look like that, and just do it a couple more times." It's like what you're doing is getting into their head and helping them understand the value, understand the excitement. That's exactly what games do, right? To your point about, like, training in games, there's a famous story, and it's like it's one of my favorite stories in gaming, from the original Mario Brothers.

I can't tell you if this is 100% true, but this is sort of apocryphal and, I think, very important story about the game world, which is that in the original Mario Brothers, they said that the first little enemy you approached originally was the turtle, and you would jump on the turtle's head and it would turn into a shell, and then you jump on the shell and it would go shoot away.

And they said that when they played it, people just didn't know what to do with that. It was like too complicated a concept. So they created this Goomba, and they created it, it was literally this, almost the same shape as the turtle shell, and they, like, made it. Apparently it was like, the story is like it was like the last bit of memory they had to make this incredibly simple enemy, but it was to teach you the core mechanic of the game- Yeah

which is jump on things' heads. Yeah. Jump on things from below or above and y- and things will happen. And it was the first villain. It comes up to you the first second you start the game, and it's exactly your point about how do you train somebody to learn the mechanics of something in a game. You do not tell them, right?

Mm-hmm. You give them a scenario. You let them try it, and you let them try it in a way that's easy to fail, and if you walk right into the front of that Goomba and you die in Mario, which probably 95% of people did in their life, no problem. Yeah. Do it again. It's still exciting. You never give up at that point.

I don't think anybody in the history of the world has given up because they walked into the Goomba and died. And I feel like, you know, these lessons are so simple psychologically, but they're so deep, and they span, I mean, that game's like 30 years old now. They span so many areas of human psychology, and I feel like what, everything I'm hearing you say about immersive environment, about a learning dojo, about engagement, it just reminds me so much of these core psychological principles.

You've talked about managed concentration. I find this approach to designing AI tools so compelling compared to what a lot of people are doing right now, even though there's a lot of interesting tools happening. I feel like you're thinking about it at a deeper level, and I really appreciate that. I'm sure our listeners are fascinated too.

[00:30:14] Sue Khim: The thing that is really compelling to me about games is you don't see A lot of 10-year-olds wandering around with this burden that, ugh, they're just not good at Mario. 

[00:30:24] Alex Sarlin: Right. Right. Mario anxiety. 

[00:30:27] Sue Khim: Mario anxiety, yeah. Like, "Ugh, I don't even wanna try because I didn't get it last time." They're gonna keep trying.

They're gonna try 20 times until they 

[00:30:36] Alex Sarlin: get that level. The opposite, right? They can't wait to get back to it to try again, and that's what you want for learning. 

[00:30:40] Sue Khim: Yeah. And it's really, really hard to do. Like, game designers are geniuses. They are really, really good at ... I was talking to someone who invests in match three games, and she talked about how these match three game designers, you know, when you clear some gems, the stuff that comes in is heavily algorithmically optimized to keep you right at that edge of motivation.

And there is just so much thought going into every single interaction that you're taking and what the appropriate response is to keep you in that flow of that game. And if we could take, you know, 10% of that energy and put it into learning instead of Candy Crush, I think that we are going to see a tremendous amount more learning.

[00:31:29] Alex Sarlin: We've had this dream for a long time now, and we've never quite gotten over the hump with it in, in the edtech world of making learning experiences as engaging and exciting and motivating as games. I really feel like we may be on the verge of it. I couldn't be more on the same page. We did a webinar about a year ago now with all of these people trying to do various types of game mechanics in learning, and trying to build tools to build learning games, all sorts of amazing things.

I really feel like that dream, which has been ... It's still alive. I mean, Duolingo is still the most popular edtech tool, and it is pure that. I feel like we are about to see that on this incredibly large scale, and it's really, really amazing to see. 

[00:32:05] Sue Khim: Yeah, I think as long as you treat the game as the content itself- Yes

and not all of the dressing and bing bongs around the game. What we have seen is that what you would typically think of as, like, the Skinner box game mechanics barely made a dent in engagement. Like that. When you made the Next piece of content, the next challenge that the user is doing really tailored for them, like that is when engagement skyrockets.

[00:32:30] Alex Sarlin: I literally just had like a, I think an hour-long conversation with Claude yesterday about exactly this, about the core loop, right? The core game loop of games has to be really great. If in match three games, that matching is so satisfying that you can build this whole world around it because the core mechanic is really fun.

Jumping is fun. You know, fighting is fun. Yeah. All these things are fun. And what we, I think, have made a mistake in many generations of EdTech game designers is not taking the moment to be like, "Well, you can't keep the core thing just being a regular learning experience that doesn't have any, that it doesn't have any fun built into it."

Like if you have to find the element that's really satisfying and make that the core mechanic. It's, it's exactly what I'm hearing you say, right? Like, keep the learning as the core of the game, not just the content of which all the game surrounds. And that's not that easy to do because sometimes those core things you're doing, if you're balancing a chemistry equation or finding the slope of a line, that can feel off-putting, it can feel scary, it can make students freeze up.

But if you find a way to make that itself fun, then everything goes from there. I feel like you're brilliant. You've really done a good job of that, and I see that identity you'll continue to build it. 

[00:33:36] Sue Khim: Yeah. The magic of Candy Crush isn't the bright lights and the sounds when you got it right. The magic of Candy Crush is the keeping you at the appropriate level of challenge.

And I think in math, you know, because it's like so grade-based, you miss something, and then everything after that doesn't make sense, but you just keep going and it's like not the appropriate level of challenge anymore. People get frustrated, then there's all this pressure of tests and grades and, you know, you're being asked to do things that you can't, then people just give up.

[00:34:03] Alex Sarlin: Exactly. I would even say that part of the core of Candy Crush is just that you're matching, right? You're matching things in a row. You're connecting, you're doing a very simple type of pattern matching and sort of adjusting things so that patterns match, and that itself is so satisfying. It feels so good.

People do it for a long time. They wouldn't do it without the surrounding game mechanics for hours. They would do it for a few minutes, but it's still fun to do. 

[00:34:24] Sue Khim: Yeah. 

[00:34:24] Alex Sarlin: And I feel like finding the core fun inside the learning is really like one of our big challenges as an EdTech community, because then everything can span from there.

But I think AI can really make that possible in a way that we, we haven't seen before. But as you look forward to the next Let's say year. Cogi is now live, you are expanding it, you're offering some really interesting free tutoring for the summer for students. I don't know if that's all already given out yet or not, but you're doing all sorts of really interesting things to build this entirely new deep thinking AI tutoring interface community.

In your wildest dreams, what does it look like a year from now? What has changed? What has grown? How is Cogi being received in the Brilliant community, but also in the edtech community at large? 

[00:35:05] Sue Khim: Yeah. We still feel really early. We just launched. I think that this first version has shown us there's a lot of opportunity.

The obvious next steps are exactly what you'd expect, more subjects, more grade levels. One thing about the launch that really surprised us is how young students are starting. We are built around middle school and high school curriculum, and, you know, we definitely have kids now as young as I think like seven years old signing up.

And so that has put earlier grades very much on the roadmap. Eventually more learning contexts, so there's a lot of work going on there. But the most exciting opportunity by far is personalization. And, you know, it's not in the shallow sense of, "Well, you like soccer, so here's a soccer word problem," but much more the tutor actually understanding how you think.

You know, really subtle things like does it know that you tend to rush? Does it know that you excel at visual reasoning, but, you know, this kind of symbolic manipulation you really struggle with? You know, does it know that you can solve a problem when the numbers are small, but then when it gets abstract, you get lost?

That's the stuff that I'm really excited about because I think it will help us provide, again, like just the right next level of challenge to the student to unlock that next insight. And the thing that we are working very hard on now is can we build some of these tutoring instincts into open model weights and make a model that is specifically very, very good at tutoring?

And I think the best version of Cogi is building a really concrete understanding of each individual learner and continuing down this path of, you know, based on that, we are able to provide more and more natural live feeling coaching that is very tailored To you. And what we've learned from launch is that students of all ages are willing to be challenged by a little green mascot.

Again, like huge unknown. The challenges being well-constructed is really important. They don't need everything to be easy. In fact, easy is boring. And for us, you know, the bar is building an AI tutor that makes students feel okay to keep trying, supported to unblock themselves, unlock new insights, and proud of the accomplishments along the way so that they want to do the next hard problem.

So, you know, making students unafraid to do hard things and proud to do hard things, like that's what gets us up in the morning. 

[00:37:38] Alex Sarlin: Amazing. I think that's a great note to end on. It feels so promising and exciting. It's all about the psychology that you're really baking into the product and baking into your learners.

Sue Khim is the co-founder and CEO of Brilliant. Brilliant is a big community for math, computer science, science courses, and teaching, and they just launched Koji, an AI tutor that helps students actually think. Thanks so much for being here with us on Edtech Insiders. 

[00:38:02] Sue Khim: Thanks, Alex. 

[00:38:04] Alex Sarlin: Thanks for listening to this episode of Edtech Insiders.

If you like the podcast, remember to rate it and share it with others in the Edtech community. For those who want even more Edtech Insider, subscribe to the free Edtech Insiders newsletter on Substack.

This season of Edtech Insiders is brought to you by Cooley LLP. Cooley is the go-to law firm for education and edtech innovators, offering industry-informed counsel across the pre-K to gray spectrum. With a multidisciplinary approach and a powerful edtech ecosystem, Cooley helps shape the future of education