This is a transcript of Lex Fridman Podcast #496 with FFmpeg & VLC.
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
- Go back to this episode’s main page
- Watch the full YouTube version of the podcast
Table of Contents
Here are the loose “chapters” in the conversation.
Click link to jump approximately to that part in the transcript:
Click link to jump approximately to that part in the transcript:
- 0:00 – Episode highlight
- 2:17 – Introduction
- 5:35 – Weirdest things VLC opens
- 9:59 – How video playback works
- 19:20 – Video codecs and containers
- 30:07 – FFmpeg explained
- 51:07 – Linus Torvalds
- 55:46 – Turning down millions to keep VLC ad-free
- 1:10:04 – FFmpeg & Google drama
- 1:29:18 – FFmpeg developers
- 1:35:55 – VLC and FFmpeg
- 1:40:29 – History of FFmpeg
- 1:43:46 – Reverse engineering codecs
- 1:57:01 – FFmpeg testing
- 2:01:08 – Assembly code (handwritten)
- 2:25:26 – Rust programming language
- 2:34:42 – FFmpeg and Libav fork
- 2:43:04 – Open source burnout
- 2:50:51 – x264 and internet video
- 3:04:07 – Video compression basics
- 3:11:04 – CIA and fake VLC
- 3:21:39 – Ultra low latency streaming
- 3:39:07 – AV2 codec and video patents
- 3:48:59 – VLC backdoors
- 3:59:14 – Video archiving
- 4:05:51 – Future of FFmpeg and VLC
Episode highlight
Jean-Baptiste Kempf
The important thing is, is your code good? We care about excellent code. We don’t care who you are. Like maybe you’re a dog. I don’t care, right? I don’t care where you come from. I need to look at your code. Oh, yeah, but I’m an engineer at this very large company in Italy, in Germany, in the US. We don’t care. We care about the quality of your code because this is what defines our community, which means that we have a lot of people who contribute who are from very different backgrounds and very introverts. Sure. But that’s okay, right?
The important thing is, is your code good? We care about excellent code. We don’t care who you are. Like maybe you’re a dog. I don’t care, right? I don’t care where you come from. I need to look at your code. Oh, yeah, but I’m an engineer at this very large company in Italy, in Germany, in the US. We don’t care. We care about the quality of your code because this is what defines our community, which means that we have a lot of people who contribute who are from very different backgrounds and very introverts. Sure. But that’s okay, right?
Lex Fridman
FFmpeg is probably one of the biggest CPU users in the world. Everything we’ve just said in the past couple of minutes, every sentence is someone’s lifetime’s work. There are books about every sentence. So the level of complexity in many cases is inordinate.
FFmpeg is probably one of the biggest CPU users in the world. Everything we’ve just said in the past couple of minutes, every sentence is someone’s lifetime’s work. There are books about every sentence. So the level of complexity in many cases is inordinate.
Jean-Baptiste Kempf
FFmpeg has one hundred thousand lines of assembly for all the codecs.
FFmpeg has one hundred thousand lines of assembly for all the codecs.
Lex Fridman
For all codecs.
For all codecs.
Jean-Baptiste Kempf
And just this one has two hundred and forty thousand. Every cycle matters. We are talking about probably three billion devices which are going to decode video nonstop because, for example, thirty percent of the video from Netflix are now in AV1, fifty percent of YouTube.
And just this one has two hundred and forty thousand. Every cycle matters. We are talking about probably three billion devices which are going to decode video nonstop because, for example, thirty percent of the video from Netflix are now in AV1, fifty percent of YouTube.
Lex Fridman
This is what peak video codecs should look like. Seventy-nine point nine percent assembly, nineteen point six percent C, and zero point five percent other.
This is what peak video codecs should look like. Seventy-nine point nine percent assembly, nineteen point six percent C, and zero point five percent other.
Jean-Baptiste Kempf
And what’s incredible is with those tweets, which are factual, people go crazy.
And what’s incredible is with those tweets, which are factual, people go crazy.
Lex Fridman
For the last two years, they go crazy. No, intrinsics is fine. The compiler-
For the last two years, they go crazy. No, intrinsics is fine. The compiler-
Jean-Baptiste Kempf
You can optimize your compiler. Auto-vectorization, it’s your fault. You don’t understand. And we’ve tried that forever, right?
You can optimize your compiler. Auto-vectorization, it’s your fault. You don’t understand. And we’ve tried that forever, right?
Lex Fridman
For two years, and two years later, showing hundreds of examples of handwritten assembly. No, no, no, you’re doing it wrong. The compiler can do this. The intelligence agencies tried to, like, say, “Can you put a backdoor in VLC?”
For two years, and two years later, showing hundreds of examples of handwritten assembly. No, no, no, you’re doing it wrong. The compiler can do this. The intelligence agencies tried to, like, say, “Can you put a backdoor in VLC?”
Jean-Baptiste Kempf
Yes. Two of them.
Yes. Two of them.
Lex Fridman
Well, what did you say?
Well, what did you say?
Jean-Baptiste Kempf
No. Well, I was a lot less polite.
No. Well, I was a lot less polite.
Lex Fridman
Basically saying, “Hell no.”
Basically saying, “Hell no.”
Jean-Baptiste Kempf
Like, if we had to compromise our software, we would shut it down. This is clear.
Like, if we had to compromise our software, we would shut it down. This is clear.
Lex Fridman
Any tweets, Kieran, you regret?
Any tweets, Kieran, you regret?
Kieran Kunhya
Tweets I regret?
Tweets I regret?
Lex Fridman
Or is it like that, how does the French song go? Regret nothing.
Or is it like that, how does the French song go? Regret nothing.
Jean-Baptiste Kempf
Don’t regret anything. No, it’s because regrets are attacks on your mind.
Don’t regret anything. No, it’s because regrets are attacks on your mind.
Introduction
Lex Fridman
The following is a conversation all about FFmpeg and VLC with Jean-Baptiste Kempf and Kieran Kunhya. FFmpeg is an open source software system that is the invisible backbone behind YouTube, Netflix, Chrome, VLC, Discord, and basically every platform that touches video or audio on the internet. It can decode, encode, transcode, stream, and play almost any video or audio format ever created. To me, it is one of the most incredible software systems ever developed, and it’s all done by volunteers. VLC is also a legendary piece of software. It is an open source media player that plays basically anything you throw at it, any format, any platform, no ads, no tracking.
The following is a conversation all about FFmpeg and VLC with Jean-Baptiste Kempf and Kieran Kunhya. FFmpeg is an open source software system that is the invisible backbone behind YouTube, Netflix, Chrome, VLC, Discord, and basically every platform that touches video or audio on the internet. It can decode, encode, transcode, stream, and play almost any video or audio format ever created. To me, it is one of the most incredible software systems ever developed, and it’s all done by volunteers. VLC is also a legendary piece of software. It is an open source media player that plays basically anything you throw at it, any format, any platform, no ads, no tracking.
Lex Fridman
It has been downloaded over six billion times, and again, for me, it has been one of my favorite pieces of software ever, with the most legendary logo, which I, of course, had to honor in this conversation by wearing the VLC traffic cone hat the whole time. So again, above all else, thank you to the incredible volunteer engineers who put their heart and soul into this code that has been used and loved by billions of people. Thank you. And about the two great engineers and human beings I’m talking to in this episode, Jean-Baptiste is the president of VideoLAN and is a key figure behind VLC and FFmpeg.
It has been downloaded over six billion times, and again, for me, it has been one of my favorite pieces of software ever, with the most legendary logo, which I, of course, had to honor in this conversation by wearing the VLC traffic cone hat the whole time. So again, above all else, thank you to the incredible volunteer engineers who put their heart and soul into this code that has been used and loved by billions of people. Thank you. And about the two great engineers and human beings I’m talking to in this episode, Jean-Baptiste is the president of VideoLAN and is a key figure behind VLC and FFmpeg.
Lex Fridman
Kieran is a longtime codec engineer, FFmpeg contributor, and the man behind the now infamous FFmpeg account on Twitter/X that I recommend everybody follow for the memes and for the unapologetic celebration of open source and great low-level software engineering. Let me also say that it’s inspiring and humbling that so much of modern civilization rests on software built by people who are not chasing fame or money, but are obsessed with the craft of engineering. We live in a world where billions of people consume video every day without ever thinking about the invisible machinery underneath it. But that machinery matters. Open source infrastructure matters.
Kieran is a longtime codec engineer, FFmpeg contributor, and the man behind the now infamous FFmpeg account on Twitter/X that I recommend everybody follow for the memes and for the unapologetic celebration of open source and great low-level software engineering. Let me also say that it’s inspiring and humbling that so much of modern civilization rests on software built by people who are not chasing fame or money, but are obsessed with the craft of engineering. We live in a world where billions of people consume video every day without ever thinking about the invisible machinery underneath it. But that machinery matters. Open source infrastructure matters.
Lex Fridman
It is one of the great examples of human beings quietly collaborating across borders to build something useful, durable, and elegant for the rest of us. And so this conversation is not just about codecs and media pipelines. It is also about the deeper spirit of engineering and generosity that makes projects like FFmpeg possible. Again, I can never say it enough. Thank you. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Jean-Baptiste Kempf and Kieran Kunhya.
It is one of the great examples of human beings quietly collaborating across borders to build something useful, durable, and elegant for the rest of us. And so this conversation is not just about codecs and media pipelines. It is also about the deeper spirit of engineering and generosity that makes projects like FFmpeg possible. Again, I can never say it enough. Thank you. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Jean-Baptiste Kempf and Kieran Kunhya.
Weirdest things VLC opens
Lex Fridman
So the legend goes VLC can open everything. What’s the weirdest thing that you know that it can open?
So the legend goes VLC can open everything. What’s the weirdest thing that you know that it can open?
Jean-Baptiste Kempf
You know, there is a ton of people who are using VLC to record VHS videos, right? Like, it’s just like you plug it with a capture card and you can basically record VHS video.
You know, there is a ton of people who are using VLC to record VHS videos, right? Like, it’s just like you plug it with a capture card and you can basically record VHS video.
Lex Fridman
Well, how does that work?
Well, how does that work?
Jean-Baptiste Kempf
Basically, it’s, you know, those type of capture cards where you can put a Peritel in or- … or RCA, and you put that, and actually VLC can play those type of cards, and there is a module which allows to control directly some of those VCR camcorders. We support DVD-Audio lately, right? We spent the summer working on DVD-Audio support, and like there is no, no one’s making any DVD-Audio support. There are custom encryption schemes.
Basically, it’s, you know, those type of capture cards where you can put a Peritel in or- … or RCA, and you put that, and actually VLC can play those type of cards, and there is a module which allows to control directly some of those VCR camcorders. We support DVD-Audio lately, right? We spent the summer working on DVD-Audio support, and like there is no, no one’s making any DVD-Audio support. There are custom encryption schemes.
Lex Fridman
What about Lucasfilm?
What about Lucasfilm?
Jean-Baptiste Kempf
Oh, yeah, and there is of course all the weird codecs support, game codecs supported by FFmpeg.
Oh, yeah, and there is of course all the weird codecs support, game codecs supported by FFmpeg.
Lex Fridman
The one Star Wars video game, the first ten-second opening sequence, someone has gone and implemented that and made sure that’s bit exact on one disc that existed at one time of one little sequence in the game.
The one Star Wars video game, the first ten-second opening sequence, someone has gone and implemented that and made sure that’s bit exact on one disc that existed at one time of one little sequence in the game.
Jean-Baptiste Kempf
And then funnily, there was a… At one VideoLAN conference, we made a competition to make the weirdest and most horrible file ever- … and see if VLC could play it.
And then funnily, there was a… At one VideoLAN conference, we made a competition to make the weirdest and most horrible file ever- … and see if VLC could play it.
Lex Fridman
What did it end up being? What’s the file?
What did it end up being? What’s the file?
Jean-Baptiste Kempf
It was an MKV file made by Derek- … which each of the frame was changing resolution, aspect ratio, rotation, and… and it was like-
It was an MKV file made by Derek- … which each of the frame was changing resolution, aspect ratio, rotation, and… and it was like-
Lex Fridman
Did it work?
Did it work?
Jean-Baptiste Kempf
Yes. And there was another one where the whole video was actually animated subtitles, right? SSA, right? So-
Yes. And there was another one where the whole video was actually animated subtitles, right? SSA, right? So-
Lex Fridman
Yeah. I remember that, yeah.
Yeah. I remember that, yeah.
Jean-Baptiste Kempf
… each, each, this one was-
… each, each, this one was-
Lex Fridman
It-
It-
Jean-Baptiste Kempf
And so each frame was a black frame, but on top of that there was a subtitle that was animated for each frame.
And so each frame was a black frame, but on top of that there was a subtitle that was animated for each frame.
Lex Fridman
There was a file that’s a valid ZIP and a valid MP3 at the same time or something like that, so.
There was a file that’s a valid ZIP and a valid MP3 at the same time or something like that, so.
Jean-Baptiste Kempf
So yeah, we made a competition of stupid files.
So yeah, we made a competition of stupid files.
Lex Fridman
And it worked. It opened all of the stupid files.
And it worked. It opened all of the stupid files.
Jean-Baptiste Kempf
Yes.
Yes.
Lex Fridman
By the way, for people who are not familiar, I am wearing a hat. Would it be fair to say this is the best worst logo of all time, the cone?
By the way, for people who are not familiar, I am wearing a hat. Would it be fair to say this is the best worst logo of all time, the cone?
Jean-Baptiste Kempf
Yeah, by far, right? The logo of VLC is so iconic, right? Like we are a team with a small number of people and the icon is known everywhere. I go to middle of nowhere in India or in China, people know the cone, right? And 25% of the website traffic that comes to our main website is cone player, right? So many people don’t know VLC, right? They know the cone player.
Yeah, by far, right? The logo of VLC is so iconic, right? Like we are a team with a small number of people and the icon is known everywhere. I go to middle of nowhere in India or in China, people know the cone, right? And 25% of the website traffic that comes to our main website is cone player, right? So many people don’t know VLC, right? They know the cone player.
Lex Fridman
That’s the thing they Google for is “cone player.”
That’s the thing they Google for is “cone player.”
Jean-Baptiste Kempf
Yeah. They go on Google and they put “cone player” and they download VLC, right? So that’s iconic. And once we tried to change it as a joke, right? We said it was going to be a type of caterpillar construction and we said that during April 1st- … and we had around 10,000 emails saying, “No, don’t change the logo,” and so on, right? So it’s so iconic, right? It’s so distinctive, right? If you want to do a video player, you’re going to put a play button on a TV, right? And that’s a YouTube logo, right? It’s unoriginal. This one is orange, right? It’s very bright and it’s weird.
Yeah. They go on Google and they put “cone player” and they download VLC, right? So that’s iconic. And once we tried to change it as a joke, right? We said it was going to be a type of caterpillar construction and we said that during April 1st- … and we had around 10,000 emails saying, “No, don’t change the logo,” and so on, right? So it’s so iconic, right? It’s so distinctive, right? If you want to do a video player, you’re going to put a play button on a TV, right? And that’s a YouTube logo, right? It’s unoriginal. This one is orange, right? It’s very bright and it’s weird.
Lex Fridman
And it’s ridiculous and it’s absurd and it’s hilarious. It becomes meme and meme becomes culture. Yeah.
And it’s ridiculous and it’s absurd and it’s hilarious. It becomes meme and meme becomes culture. Yeah.
Jean-Baptiste Kempf
And you keep it and you know about it and you know that in 20 years, like you still have, going to have the cones and remember, oh yeah, that was a video player.
And you keep it and you know about it and you know that in 20 years, like you still have, going to have the cones and remember, oh yeah, that was a video player.
Lex Fridman
Yeah. And we’ll talk about, you know, the mission of FFmpeg being a kinda the archival aspect of it. So you can think about 1,000 years from now we’ll have all these videos that only VLC can open. Humans, human civilization has already destroyed itself multiple times and the only thing that will remain is this like, you know, the cockroaches will be crawling around and it’ll be the VLC logo- … with our, some of the archival footage that VLC can open. And the aliens will show up and they’ll press play and they’ll get to see it all ’cause-
Yeah. And we’ll talk about, you know, the mission of FFmpeg being a kinda the archival aspect of it. So you can think about 1,000 years from now we’ll have all these videos that only VLC can open. Humans, human civilization has already destroyed itself multiple times and the only thing that will remain is this like, you know, the cockroaches will be crawling around and it’ll be the VLC logo- … with our, some of the archival footage that VLC can open. And the aliens will show up and they’ll press play and they’ll get to see it all ’cause-
Jean-Baptiste Kempf
Well, really, really hope so, right? But there is also so many memes where people say, “Well, I’m sure I can put a pancake inside my DVD drive and VLC will play it.” Like-
Well, really, really hope so, right? But there is also so many memes where people say, “Well, I’m sure I can put a pancake inside my DVD drive and VLC will play it.” Like-
Lex Fridman
Can they?
Can they?
Jean-Baptiste Kempf
No, we tried. It doesn’t.
No, we tried. It doesn’t.
Lex Fridman
Doesn’t
Doesn’t
Jean-Baptiste Kempf
… but we actually have a video of us trying that. Didn’t work.
… but we actually have a video of us trying that. Didn’t work.
Lex Fridman
A codec for physical reality, I don’t know what that would even look like.
A codec for physical reality, I don’t know what that would even look like.
Jean-Baptiste Kempf
There was a guy who did that, right? He printed a small cone, right? Like the ones we distribute as goodies and inside he put an RFID chip which was his way of playing a movie, right? And so he- … put this on a RFID player and when he put that it was playing like The Last Star Wars and so on. So instead of having like DVD boxes, he had like VLC cones all around and he plugged that and that was like physical objects.
There was a guy who did that, right? He printed a small cone, right? Like the ones we distribute as goodies and inside he put an RFID chip which was his way of playing a movie, right? And so he- … put this on a RFID player and when he put that it was playing like The Last Star Wars and so on. So instead of having like DVD boxes, he had like VLC cones all around and he plugged that and that was like physical objects.
How video playback works
Lex Fridman
So the thing that we’re talking about is everything around video codecs, video encoding, video decoding, video streaming, video player client that I’m wearing on my head, the entire ecosystem enabling free media. We’ll talk about FFmpeg, we’ll talk about VideoLAN, VLC and all the other incredible video technology that is used probably by billions of people. So JB, you’re the lead developer behind the legendary VLC player. Kieran, amongst many other things, you’re lead developer behind the legendary FFmpeg handle on Twitter. And both of you have spicy opinions I would say. So today we wanna talk about FFmpeg and VLC.
So the thing that we’re talking about is everything around video codecs, video encoding, video decoding, video streaming, video player client that I’m wearing on my head, the entire ecosystem enabling free media. We’ll talk about FFmpeg, we’ll talk about VideoLAN, VLC and all the other incredible video technology that is used probably by billions of people. So JB, you’re the lead developer behind the legendary VLC player. Kieran, amongst many other things, you’re lead developer behind the legendary FFmpeg handle on Twitter. And both of you have spicy opinions I would say. So today we wanna talk about FFmpeg and VLC.
Lex Fridman
For context for people who are not aware—and I’m sure basically everybody listening to this have used these two technologies probably regularly without knowing it. So FFmpeg underlies basically most video on the internet including YouTube, Netflix, Chrome, Firefox, of course VLC and countless other video platforms. It is estimated that over 90% of video processing workflows online and offline involve FFmpeg. VLC has been downloaded at least 6.5 billion times. But likely that number, ’cause it’s impossible to really count the number, is much higher than that. Virtually any operating system supports virtually any media format. The limitation being it can’t open pancakes. So can we just lay out some of the basics to, to help people understand what’s involved in all of this?
For context for people who are not aware—and I’m sure basically everybody listening to this have used these two technologies probably regularly without knowing it. So FFmpeg underlies basically most video on the internet including YouTube, Netflix, Chrome, Firefox, of course VLC and countless other video platforms. It is estimated that over 90% of video processing workflows online and offline involve FFmpeg. VLC has been downloaded at least 6.5 billion times. But likely that number, ’cause it’s impossible to really count the number, is much higher than that. Virtually any operating system supports virtually any media format. The limitation being it can’t open pancakes. So can we just lay out some of the basics to, to help people understand what’s involved in all of this?
Lex Fridman
So when we press play on a video player like VLC, what happens? How does it go from the file or the stream to the pixels on the screen and the sound on the speaker? What are the big stages to be aware of?
So when we press play on a video player like VLC, what happens? How does it go from the file or the stream to the pixels on the screen and the sound on the speaker? What are the big stages to be aware of?
Jean-Baptiste Kempf
So there are several stages, right? The first stage is to get from an address, right, which is the type of URL, to give you a byte of streams, right? So this would be, for example, HTTP, file, DVD, right? You give the path to the media, and it gives you a stream of data.
So there are several stages, right? The first stage is to get from an address, right, which is the type of URL, to give you a byte of streams, right? So this would be, for example, HTTP, file, DVD, right? You give the path to the media, and it gives you a stream of data.
Kieran Kunhya
The stream needs to be cut up by what’s known as the container, the demultiplexer or demux. We’ll try and keep the jargon light throughout this, but it needs to go and start demarcating video and audio frames. So it just gets data from the operating system blocks at a time and needs to start cutting these frames up into compressed data. It then needs to start doing simple parsing of the video frames-
The stream needs to be cut up by what’s known as the container, the demultiplexer or demux. We’ll try and keep the jargon light throughout this, but it needs to go and start demarcating video and audio frames. So it just gets data from the operating system blocks at a time and needs to start cutting these frames up into compressed data. It then needs to start doing simple parsing of the video frames-
Kieran Kunhya
… mainly to figure out whether that codec is GPU decodable or needs to fall back to software. We’re very sort of used to assuming the GPU will play all of these things. There’ll be hardware acceleration. I think it’s up to forty-five percent of files are not GPU decodable. So these need to be probed. They need to be detected. There can be variants of a given codec, some of which are decodable on the GPU. Different vendors of GPU might have different capabilities, so those need to be detected. So if, if it’s GPU capable, you pass it through to the GPU black box. So now if there’s a software fallback, that means in the beginning is, is to first do deentropy coding, so removing the mathematical coding of the bit stream.
… mainly to figure out whether that codec is GPU decodable or needs to fall back to software. We’re very sort of used to assuming the GPU will play all of these things. There’ll be hardware acceleration. I think it’s up to forty-five percent of files are not GPU decodable. So these need to be probed. They need to be detected. There can be variants of a given codec, some of which are decodable on the GPU. Different vendors of GPU might have different capabilities, so those need to be detected. So if, if it’s GPU capable, you pass it through to the GPU black box. So now if there’s a software fallback, that means in the beginning is, is to first do deentropy coding, so removing the mathematical coding of the bit stream.
Kieran Kunhya
So this uses capabilities such as Huffman coding or arithmetic coding to actually decompress the mathematical layer of the bit stream. We then need to start reading the syntax elements for intra prediction. So intra prediction are like still images of the video, so your I-frames. So this works and operates in the spatial domain. So you do your intra prediction in spatial domain. You have a residual because your prediction isn’t quite matching that of reality. So you’ve made a prediction, but then there’s a little bit left, and that’s what’s known as the residual. This is stored in the frequency domain, and these are quantized to decompact their space. We then need to do the inverse transform to bring them back to the spatial domain and apply these residuals.
So this uses capabilities such as Huffman coding or arithmetic coding to actually decompress the mathematical layer of the bit stream. We then need to start reading the syntax elements for intra prediction. So intra prediction are like still images of the video, so your I-frames. So this works and operates in the spatial domain. So you do your intra prediction in spatial domain. You have a residual because your prediction isn’t quite matching that of reality. So you’ve made a prediction, but then there’s a little bit left, and that’s what’s known as the residual. This is stored in the frequency domain, and these are quantized to decompact their space. We then need to do the inverse transform to bring them back to the spatial domain and apply these residuals.
Lex Fridman
So a lot of the process of the decoding is this thing is compressed. And you have to predict the highest quality thing that’s supposed to go there. I-frame- … is the best representation you have spatially.
So a lot of the process of the decoding is this thing is compressed. And you have to predict the highest quality thing that’s supposed to go there. I-frame- … is the best representation you have spatially.
Kieran Kunhya
Yes.
Yes.
Lex Fridman
And then there’s a lot of temporal compression that can happen depending on the codec, and then you’re predicting. You’re predicting what the reality that was captured in this rawest form.
And then there’s a lot of temporal compression that can happen depending on the codec, and then you’re predicting. You’re predicting what the reality that was captured in this rawest form.
Jean-Baptiste Kempf
Yeah, because what people don’t realize is that the compression on video and audio is one hundred times, right? Like, people don’t realize how compressed we do, right? For audio, you move, you compress by, when you go from normal audio to MP3, you compress by ten times, right? When you move to video, you need one hundred times, two hundred times, right? So you need to remove all the details, but that you don’t care about because all the compressions that we do, and that’s very important, people forget about that, is to be viewed by humans, right?
Yeah, because what people don’t realize is that the compression on video and audio is one hundred times, right? Like, people don’t realize how compressed we do, right? For audio, you move, you compress by, when you go from normal audio to MP3, you compress by ten times, right? When you move to video, you need one hundred times, two hundred times, right? So you need to remove all the details, but that you don’t care about because all the compressions that we do, and that’s very important, people forget about that, is to be viewed by humans, right?
Jean-Baptiste Kempf
So all the codecs, either for audio, mimic basically how your ear works, right? And a lot of things about, like, the response on the ear and same for your eyes, right? And so, for example, on video, we don’t work on RGB, right? Everyone expect to work in RGB. We don’t, right? We move to YUV, which is basically one is luminance, brightness, and the other are colors. And this matches your eyes, where inside your eyes you have the cones and the rods, right? With some of them look on brightness and more on the other on colors, right? So we need to compress a lot, and so we need to degrade. But in order to degrade, we need to match the human perception, and this is why it’s so difficult.
So all the codecs, either for audio, mimic basically how your ear works, right? And a lot of things about, like, the response on the ear and same for your eyes, right? And so, for example, on video, we don’t work on RGB, right? Everyone expect to work in RGB. We don’t, right? We move to YUV, which is basically one is luminance, brightness, and the other are colors. And this matches your eyes, where inside your eyes you have the cones and the rods, right? With some of them look on brightness and more on the other on colors, right? So we need to compress a lot, and so we need to degrade. But in order to degrade, we need to match the human perception, and this is why it’s so difficult.
Jean-Baptiste Kempf
And then we need to use the maximum power, mathematical power, very complex technologies. We move to the frequency domain, as Kieran said. We do a ton of dequantizing in order to get the best compression, but it still looks good.
And then we need to use the maximum power, mathematical power, very complex technologies. We move to the frequency domain, as Kieran said. We do a ton of dequantizing in order to get the best compression, but it still looks good.
Lex Fridman
You’re trying to compress in order to maximize the highest quality thing for human perception.
You’re trying to compress in order to maximize the highest quality thing for human perception.
Jean-Baptiste Kempf
That is correct. And this is very important, right? Compression is not like a ZIP, right? A ZIP, you have data in, you get data out, right? And you try with all the ZIP compression to arrive with the limit. Here we are degrading the signal, right? And so we need to degrade both the audio and the video signal in the best way possible. And we can do that, but it involves, first, a lot of theoretical knowledge about how the eye works, but a lot of mathematical change, a lot of mathematical tricks, right? For example, when you move to RGB and you go to YUV, for example, what we do very often is that we scale down the resolution of the color compared to the brightness.
That is correct. And this is very important, right? Compression is not like a ZIP, right? A ZIP, you have data in, you get data out, right? And you try with all the ZIP compression to arrive with the limit. Here we are degrading the signal, right? And so we need to degrade both the audio and the video signal in the best way possible. And we can do that, but it involves, first, a lot of theoretical knowledge about how the eye works, but a lot of mathematical change, a lot of mathematical tricks, right? For example, when you move to RGB and you go to YUV, for example, what we do very often is that we scale down the resolution of the color compared to the brightness.
Jean-Baptiste Kempf
And most of the time, and just this without compression, it divides the size by two, but most people don’t see it, right? And so on and so on, right? And then you go to very complex mathematical change. So of course Fourier transforms, which de facto are not Fourier transforms, they are like discrete cosine transforms, but that’s the same idea. So frequency domain we split the video by blocks, right? So that’s why when it’s wrongly decoded, you see those blocks, and badly encoded, you see those blocks, and so on, to arrive to compression states that are insanely high, right? And each generation of the codec is like thirty percent less- … for the same quality, right? And this requires amount of power, of computational power, that are huge.
And most of the time, and just this without compression, it divides the size by two, but most people don’t see it, right? And so on and so on, right? And then you go to very complex mathematical change. So of course Fourier transforms, which de facto are not Fourier transforms, they are like discrete cosine transforms, but that’s the same idea. So frequency domain we split the video by blocks, right? So that’s why when it’s wrongly decoded, you see those blocks, and badly encoded, you see those blocks, and so on, to arrive to compression states that are insanely high, right? And each generation of the codec is like thirty percent less- … for the same quality, right? And this requires amount of power, of computational power, that are huge.
Kieran Kunhya
No, no, but you should, you should elaborate. It’s thirty percent better, but an order of magnitude, perhaps, perhaps even two orders of magnitude more compression power. That’s the big difference.
No, no, but you should, you should elaborate. It’s thirty percent better, but an order of magnitude, perhaps, perhaps even two orders of magnitude more compression power. That’s the big difference.
Lex Fridman
What do you mean by compression power?
What do you mean by compression power?
Kieran Kunhya
Sorry, CPU power to achieve that level of compression.
Sorry, CPU power to achieve that level of compression.
Lex Fridman
Oh, yeah. So and you have to be able to leverage the CPU and sometimes GPU, like you mentioned. And then we should mention that a lot of this programming is done at the lowest possible- … stack, whether it’s C and of course, as, as the legendary- … Twitter handle re-emphasizes over and over, a lot of assembly.
Oh, yeah. So and you have to be able to leverage the CPU and sometimes GPU, like you mentioned. And then we should mention that a lot of this programming is done at the lowest possible- … stack, whether it’s C and of course, as, as the legendary- … Twitter handle re-emphasizes over and over, a lot of assembly.
Jean-Baptiste Kempf
So what happens globally is that you have an address, right? Which gives you with the operating system, a stream of bytes, a stream of data, right? And this is the first step. And the second step arises with demuxing, where you’re going to separate audio, video, subtitle in type of different tracks. And then on each of those tracks, you’re going to decompress them, decode them, either audio with an audio codec, video to video codec, and subtitle to subtitle codec. And once you’ve decompressed those type of things, you have raw images, raw, and then you’re going to talk with your graphic card in your screen and display that. And same for the audio, you’re going to talk to your audio card, which then is going to go in analog to your audio speakers.
So what happens globally is that you have an address, right? Which gives you with the operating system, a stream of bytes, a stream of data, right? And this is the first step. And the second step arises with demuxing, where you’re going to separate audio, video, subtitle in type of different tracks. And then on each of those tracks, you’re going to decompress them, decode them, either audio with an audio codec, video to video codec, and subtitle to subtitle codec. And once you’ve decompressed those type of things, you have raw images, raw, and then you’re going to talk with your graphic card in your screen and display that. And same for the audio, you’re going to talk to your audio card, which then is going to go in analog to your audio speakers.
Kieran Kunhya
And everything we’ve just said in the past couple of minutes, every sentence is someone’s lifetime’s work. There are books about- … every sentence. So the level of complexity in many cases is inordinate. You know, it’s… Every sentence has thousands of people working on this- … in industry as a whole, books written about it. So there’s a lot of detail, there’s a lot of subtleties, there’s a lot of both academic and practical realities, both of which matter.
And everything we’ve just said in the past couple of minutes, every sentence is someone’s lifetime’s work. There are books about- … every sentence. So the level of complexity in many cases is inordinate. You know, it’s… Every sentence has thousands of people working on this- … in industry as a whole, books written about it. So there’s a lot of detail, there’s a lot of subtleties, there’s a lot of both academic and practical realities, both of which matter.
Video codecs and containers
Lex Fridman
We mentioned codecs, but I don’t think you mentioned containers. So what are the actual containers for some of the stuff we’re talking about? So people are familiar with MP4, MOV, MKV. So anyway, what are containers versus the thing that goes inside?
We mentioned codecs, but I don’t think you mentioned containers. So what are the actual containers for some of the stuff we’re talking about? So people are familiar with MP4, MOV, MKV. So anyway, what are containers versus the thing that goes inside?
Jean-Baptiste Kempf
So the container is what we call also the muxer, right? When I say demuxing, it means decontainerizing, right? So actually, if you look, mux means multiplexer and demultiplexer, right? Mux and demux are those. And same, a codec is actually coder, decoder, right? And so containers are this collection of multiple tracks, right? So it’s a, what normal people call the file format, but it’s a bit more subtle than that. But the most known one, of course, is MP4, but when I started, it was AVI, right? AVI was the video format from-
So the container is what we call also the muxer, right? When I say demuxing, it means decontainerizing, right? So actually, if you look, mux means multiplexer and demultiplexer, right? Mux and demux are those. And same, a codec is actually coder, decoder, right? And so containers are this collection of multiple tracks, right? So it’s a, what normal people call the file format, but it’s a bit more subtle than that. But the most known one, of course, is MP4, but when I started, it was AVI, right? AVI was the video format from-
Jean-Baptiste Kempf
… from Microsoft, and MOV, M-O-V, which became MP4, was a format from Apple. In the open source community one of the person that is still active on VideoLAN is called Steve Lhomme and sta- started this Matroska format, which is, like, a bit more complex and, and, and more future-proof. And there are so many others.
… from Microsoft, and MOV, M-O-V, which became MP4, was a format from Apple. In the open source community one of the person that is still active on VideoLAN is called Steve Lhomme and sta- started this Matroska format, which is, like, a bit more complex and, and, and more future-proof. And there are so many others.
Lex Fridman
So, I mean, there’s a, it’s a pretty common thing, and maybe it’ll even happen in this conversation, that people confuse container and the codec, right? So confuse MP4 and H.264, for example. Is that a horrible violation?
So, I mean, there’s a, it’s a pretty common thing, and maybe it’ll even happen in this conversation, that people confuse container and the codec, right? So confuse MP4 and H.264, for example. Is that a horrible violation?
Jean-Baptiste Kempf
No, it’s not, because technically the name of H.264 is MPEG-4 Part 10. Because MPEG-4 is actually a meta specification which has several things in it, right? There is the Part 2. So there is, like, audio codecs, right? AAC de facto is MP4 audio- … something. There is actually several video codecs, right, inside the MPEG-4 specification. One of them is MPEG-4 Part 10, called also AVC, called also H.264. Right? So it’s completely the fault of the industry to make things difficult to understand. So that’s very difficult so that people then don’t understand why sometimes you talk about MPEG-4 Part 10, where you mean H.264, and why it’s not MP4.
No, it’s not, because technically the name of H.264 is MPEG-4 Part 10. Because MPEG-4 is actually a meta specification which has several things in it, right? There is the Part 2. So there is, like, audio codecs, right? AAC de facto is MP4 audio- … something. There is actually several video codecs, right, inside the MPEG-4 specification. One of them is MPEG-4 Part 10, called also AVC, called also H.264. Right? So it’s completely the fault of the industry to make things difficult to understand. So that’s very difficult so that people then don’t understand why sometimes you talk about MPEG-4 Part 10, where you mean H.264, and why it’s not MP4.
Lex Fridman
So you can technically shove in all kinds of different codecs inside containers and horribly so.
So you can technically shove in all kinds of different codecs inside containers and horribly so.
Kieran Kunhya
But broadly speaking, though, MP4 is understood to generally be H.264 plus AAC audio. 99% of the time that’s that, and that, the, the rest are de minimis, the small effects, you know, edge effects really compared to that. So it’s not the end of the world. There are people who do get annoyed by that. But also in reality, something like VLC, just to point out, the file may say .MP4, but it may be something completely different, and that’s one of the challenges both FFmpeg and VLC have is the real world is a completely different place to a three-letter file format.
But broadly speaking, though, MP4 is understood to generally be H.264 plus AAC audio. 99% of the time that’s that, and that, the, the rest are de minimis, the small effects, you know, edge effects really compared to that. So it’s not the end of the world. There are people who do get annoyed by that. But also in reality, something like VLC, just to point out, the file may say .MP4, but it may be something completely different, and that’s one of the challenges both FFmpeg and VLC have is the real world is a completely different place to a three-letter file format.
Jean-Baptiste Kempf
And this is very important to say, right? Like, for example, in VLC and in FFmpeg, we discard the file format, right? We look into the file to understand what’s in it because so many people, like, they say, “Oh, it’s a video, it must be MP4,” but technically it’s an MOV or maybe it’s a MKV, right? So we analyze in real time everything that we have, and we don’t trust- … the format.
And this is very important to say, right? Like, for example, in VLC and in FFmpeg, we discard the file format, right? We look into the file to understand what’s in it because so many people, like, they say, “Oh, it’s a video, it must be MP4,” but technically it’s an MOV or maybe it’s a MKV, right? So we analyze in real time everything that we have, and we don’t trust- … the format.
Lex Fridman
So what information does the fact that it’s .MP4 give you?
So what information does the fact that it’s .MP4 give you?
Jean-Baptiste Kempf
It helps, right? It gives you a hint, right? Just like, oh, it’s finished by .MP4. I will start first by opening, probing it with the MP4 container demuxer to see, well, it should be that. But I don’t trust it, and if I’m lost, I say, “Okay, maybe I’m going to try it.” So it bumps the priority of the module.
It helps, right? It gives you a hint, right? Just like, oh, it’s finished by .MP4. I will start first by opening, probing it with the MP4 container demuxer to see, well, it should be that. But I don’t trust it, and if I’m lost, I say, “Okay, maybe I’m going to try it.” So it bumps the priority of the module.
Lex Fridman
So how do you get to—just to take a bit of a tangent there. You know, the dumb thing is if you try the MP4, but it turns out it’s a different codec than you would have expected, most players just break there. And so how do you not break? There’s just philosophically, I’m sure there’s a bunch of stumbling blocks along the way where it’s easy to just break and stop, freak out. That’s it. How does VLC not?
So how do you get to—just to take a bit of a tangent there. You know, the dumb thing is if you try the MP4, but it turns out it’s a different codec than you would have expected, most players just break there. And so how do you not break? There’s just philosophically, I’m sure there’s a bunch of stumbling blocks along the way where it’s easy to just break and stop, freak out. That’s it. How does VLC not?
Jean-Baptiste Kempf
This is why VLC is popular. But the reason is because actually VLC was, is just a client of a streaming solution called VideoLAN from a very long time ago, from the late ’90s. And when you’re playing video which are on UDP, right, in network, they might be damaged, right? So you don’t trust your inputs, and this is very important into the security is that you don’t trust your inputs. So everything in VLC is prepared to work with broken files. And it’s a philosophical idea from the beginning, and everything is engineered into that. And it’s a culture, right? And so, for example… And VLC became very popular on that because a long time ago when people were pirating content which they do a lot less today, um-
This is why VLC is popular. But the reason is because actually VLC was, is just a client of a streaming solution called VideoLAN from a very long time ago, from the late ’90s. And when you’re playing video which are on UDP, right, in network, they might be damaged, right? So you don’t trust your inputs, and this is very important into the security is that you don’t trust your inputs. So everything in VLC is prepared to work with broken files. And it’s a philosophical idea from the beginning, and everything is engineered into that. And it’s a culture, right? And so, for example… And VLC became very popular on that because a long time ago when people were pirating content which they do a lot less today, um-
Lex Fridman
And none of us ever have-
And none of us ever have-
Jean-Baptiste Kempf
No, of course not. Um- … the metadata to play some files like AVI is at, at the end of the file, right? And when you’re downloading, you don’t have that, right? So VLC was just like, “Hey, this file is broken, but I’m still going to try to interpret it,” and this was very useful.
No, of course not. Um- … the metadata to play some files like AVI is at, at the end of the file, right? And when you’re downloading, you don’t have that, right? So VLC was just like, “Hey, this file is broken, but I’m still going to try to interpret it,” and this was very useful.
Lex Fridman
We hinted at the awesomeness of the various different stages. We hinted at the awesomeness of codecs, the depth and the richness and the complexity of everything involved there. What— Let’s try to define what is a video codec? What’s involved there? What does it mean to compress something? You already started to hint at it— … but can, can we elaborate a little bit more?
We hinted at the awesomeness of the various different stages. We hinted at the awesomeness of codecs, the depth and the richness and the complexity of everything involved there. What— Let’s try to define what is a video codec? What’s involved there? What does it mean to compress something? You already started to hint at it— … but can, can we elaborate a little bit more?
Kieran Kunhya
So there’s a huge amount of redundancy in any video, both spatial and temporal, and the point of any video codec is to remove this redundant data, use mathematical properties as part of this reduction process. So more often than not, using several orders of magnitude more compute to compress because that’s more costly, both financially and in CPU resources— … versus the decompression. So it’s asymmetric in that respect. Often the case because compression is done once, but there could be lots of viewers of another file.
So there’s a huge amount of redundancy in any video, both spatial and temporal, and the point of any video codec is to remove this redundant data, use mathematical properties as part of this reduction process. So more often than not, using several orders of magnitude more compute to compress because that’s more costly, both financially and in CPU resources— … versus the decompression. So it’s asymmetric in that respect. Often the case because compression is done once, but there could be lots of viewers of another file.
Kieran Kunhya
So to take that information and compress it by 100x, 200x, removing redundant information and using mathematical properties to make that small, but also have properties such as error resilience. So as JB suggested, VLC in the beginning was used to play UDP network feeds, and UDP network feeds lose packets. And so some of the design goals of a codec is also to be recoverable. You need to actually be able to join a stream. It’s not necessarily a file. You need to join, get on the decoding process, and start decoding.
So to take that information and compress it by 100x, 200x, removing redundant information and using mathematical properties to make that small, but also have properties such as error resilience. So as JB suggested, VLC in the beginning was used to play UDP network feeds, and UDP network feeds lose packets. And so some of the design goals of a codec is also to be recoverable. You need to actually be able to join a stream. It’s not necessarily a file. You need to join, get on the decoding process, and start decoding.
Jean-Baptiste Kempf
And to give a more image to people who are not familiar, right? Like, when you’re going to see any type of movie, right? You’re going to see the camera is going to pan, right, and travel. And you realize that, for example, all the background is the same for like a minute, right? Or— … thirty seconds, right? So you can reuse the cloud that you see on the background, you can reuse that from a frame to another, right? And so it gets the more, the more memory you have, the more power, the more comparisons you can make, right? And so the more compressed you can be. And most of the modern codecs are basically doing that.
And to give a more image to people who are not familiar, right? Like, when you’re going to see any type of movie, right? You’re going to see the camera is going to pan, right, and travel. And you realize that, for example, all the background is the same for like a minute, right? Or— … thirty seconds, right? So you can reuse the cloud that you see on the background, you can reuse that from a frame to another, right? And so it gets the more, the more memory you have, the more power, the more comparisons you can make, right? And so the more compressed you can be. And most of the modern codecs are basically doing that.
Lex Fridman
So just to make it even more explicit. So what is video? Video is a bunch of pixels off an RGB. You have three values, and you have a grid of pixels, and you have, let’s say, twenty-four or thirty or sixty frames a second, and you just have all these pixels repeating and showing different stuff- … thirty times a second. And so the question, the philosophical, the technical question is, how can I compress all of that, store all of that at 100x?
So just to make it even more explicit. So what is video? Video is a bunch of pixels off an RGB. You have three values, and you have a grid of pixels, and you have, let’s say, twenty-four or thirty or sixty frames a second, and you just have all these pixels repeating and showing different stuff- … thirty times a second. And so the question, the philosophical, the technical question is, how can I compress all of that, store all of that at 100x?
Jean-Baptiste Kempf
Yep. Or 1,000x, right?
Yep. Or 1,000x, right?
Lex Fridman
1,000x.
1,000x.
Jean-Baptiste Kempf
The, the target is 1,000x, right?
The, the target is 1,000x, right?
Lex Fridman
And the goal is when you say redundancy, what is redundant? Meaning stuff at best that humans wouldn’t notice if it was missing.
And the goal is when you say redundancy, what is redundant? Meaning stuff at best that humans wouldn’t notice if it was missing.
Jean-Baptiste Kempf
So for example, you have a picture of a cloud, right? And from the next frame, it’s still going to be the same cloud, so it’s redundant. You could just put it once and not do it, right? Or you have a black background behind me, for example. The black is the same on the whole picture, right? So you can say, “Well, you know, in this picture, take the pixels that you have on the top left and the one on the top right. I’m not going to give the value. I’m just going to tell you it’s the same at the top left.” And then you can say for frame one, reuse something from the previous frame or the previous, previous frame, and so on and so on, right? So you could… Basically, it’s unlimited, but then it’s limited in terms of memory or in terms of compute power.
So for example, you have a picture of a cloud, right? And from the next frame, it’s still going to be the same cloud, so it’s redundant. You could just put it once and not do it, right? Or you have a black background behind me, for example. The black is the same on the whole picture, right? So you can say, “Well, you know, in this picture, take the pixels that you have on the top left and the one on the top right. I’m not going to give the value. I’m just going to tell you it’s the same at the top left.” And then you can say for frame one, reuse something from the previous frame or the previous, previous frame, and so on and so on, right? So you could… Basically, it’s unlimited, but then it’s limited in terms of memory or in terms of compute power.
Jean-Baptiste Kempf
Because, for example, if you need to compare pixels on two hundred frames in the past on 4K resolutions, it’s a huge amount of compute.
Because, for example, if you need to compare pixels on two hundred frames in the past on 4K resolutions, it’s a huge amount of compute.
Lex Fridman
And then when you’re showing it, you have to do the decompress of all of that. So is it the codec, has the encoding and the decoding is a coupled process that you’re developing?
And then when you’re showing it, you have to do the decompress of all of that. So is it the codec, has the encoding and the decoding is a coupled process that you’re developing?
Jean-Baptiste Kempf
Yes, exactly, right. And those are two different trade-offs, right? Are you going to compress more? But then it might be more difficult to decode. Are you going to make it a codec that is more complex to encode and easier to decode? Are you going to make a codec that is easier to encode because you need to be fast, but then the client side, the player is going to spend more time? That’s why you have so many different types of codecs, is that it’s not always easy. And to make it even more complex, modern codecs like AV1, AV2, or VVC are actually not codecs. They are a collection of tools, right? There are multiple tools, multiple codecs in the same codec to, depending on the image, get the more compression.
Yes, exactly, right. And those are two different trade-offs, right? Are you going to compress more? But then it might be more difficult to decode. Are you going to make it a codec that is more complex to encode and easier to decode? Are you going to make a codec that is easier to encode because you need to be fast, but then the client side, the player is going to spend more time? That’s why you have so many different types of codecs, is that it’s not always easy. And to make it even more complex, modern codecs like AV1, AV2, or VVC are actually not codecs. They are a collection of tools, right? There are multiple tools, multiple codecs in the same codec to, depending on the image, get the more compression.
Kieran Kunhya
So just to elaborate, codecs like AV1, VVC have a much wide, have a wide audience. It could be a screen share content, it could be video, it could be animation. All of these require different coding tools. So what happens these days is a collection of tools are put in and called AV1 and called AV2, called VVC to allow for different use cases. So you may be on Zoom and sharing your PowerPoint, and then you need to show the audience a video. That codec needs to start changing its tool set depending on the content to compress in a different way.
So just to elaborate, codecs like AV1, VVC have a much wide, have a wide audience. It could be a screen share content, it could be video, it could be animation. All of these require different coding tools. So what happens these days is a collection of tools are put in and called AV1 and called AV2, called VVC to allow for different use cases. So you may be on Zoom and sharing your PowerPoint, and then you need to show the audience a video. That codec needs to start changing its tool set depending on the content to compress in a different way.
Lex Fridman
And like you said, there’s a, a bunch of incredible engineers behind each part of that, each part of the tools that make up AV1, for example.
And like you said, there’s a, a bunch of incredible engineers behind each part of that, each part of the tools that make up AV1, for example.
Kieran Kunhya
Sure.
Sure.
FFmpeg explained
Lex Fridman
So we’ve kind of danced around it. We talked about VLC, the logo, the hat. Let’s talk about FFmpeg. What, what is FFmpeg exactly?
So we’ve kind of danced around it. We talked about VLC, the logo, the hat. Let’s talk about FFmpeg. What, what is FFmpeg exactly?
Jean-Baptiste Kempf
FFmpeg is basically the low-level libraries for codec, so compressions and decompression, muxers and demuxers, and filters. It’s– The core is this, and then you have a several tool which allow you to create a type of pipeline to process any type of video files. And it’s used as a library absolutely inside everything from VLC to Chrome to your smart TVs, to basically any video that you see online you usually use FFmpeg. And FFmpeg in it has all those type of tools, and sometimes depend on other libraries like x264, libvpx, and others, right? So, so it’s really now the de facto tool to process images.
FFmpeg is basically the low-level libraries for codec, so compressions and decompression, muxers and demuxers, and filters. It’s– The core is this, and then you have a several tool which allow you to create a type of pipeline to process any type of video files. And it’s used as a library absolutely inside everything from VLC to Chrome to your smart TVs, to basically any video that you see online you usually use FFmpeg. And FFmpeg in it has all those type of tools, and sometimes depend on other libraries like x264, libvpx, and others, right? So, so it’s really now the de facto tool to process images.
Kieran Kunhya
From a philosophical level, I think it’s incredible that your home videos, your, your grandmother’s home videos and trillion-dollar corporations effectively are on a level playing field using the same technology stack. It’s– it wouldn’t be a surprise, you know, these big companies just have three thousand-line FFmpeg commands. There are some that use the API, but there are some that just have long command lines.
From a philosophical level, I think it’s incredible that your home videos, your, your grandmother’s home videos and trillion-dollar corporations effectively are on a level playing field using the same technology stack. It’s– it wouldn’t be a surprise, you know, these big companies just have three thousand-line FFmpeg commands. There are some that use the API, but there are some that just have long command lines.
Lex Fridman
So yeah, there’s a bunch of tools, like literally command line tool, FFmpeg, of course, FFprobe. There’s libraries, libavcodec, libavformat, libavfilter. But the FFmpeg on the command line- … is, like, legendary because you can cut– Like, there’s so many parameters. You can customize everything to hell.
So yeah, there’s a bunch of tools, like literally command line tool, FFmpeg, of course, FFprobe. There’s libraries, libavcodec, libavformat, libavfilter. But the FFmpeg on the command line- … is, like, legendary because you can cut– Like, there’s so many parameters. You can customize everything to hell.
Jean-Baptiste Kempf
It’s a language. It’s an actual language.
It’s a language. It’s an actual language.
Lex Fridman
It’s an actual– yeah, you could think of it as a programming language.
It’s an actual– yeah, you could think of it as a programming language.
Jean-Baptiste Kempf
Yeah, of course, I’m sure. Because– So most of the people, they’re going to take FFmpeg, file in, file out, and specify the format, right? But you can– We’ve seen thousands of characters, and we’ve seen also, like, people doing programming generation of command lines to make FFmpeg. There is a ton of people who are using AI to generate command lines for FFmpeg because you have no idea what it is. But you can specify so many filters, right on command line, right? So FFmpeg is this collection of toolbox for multimedia processing that everyone, everyone uses. And everyone that is watching your videos are also using, right? You’re on YouTube. Well, it’s FFmpeg on the client side. Well, the server side, on the server side. The client side is probably Chrome. Well, you’re using FFmpeg also.
Yeah, of course, I’m sure. Because– So most of the people, they’re going to take FFmpeg, file in, file out, and specify the format, right? But you can– We’ve seen thousands of characters, and we’ve seen also, like, people doing programming generation of command lines to make FFmpeg. There is a ton of people who are using AI to generate command lines for FFmpeg because you have no idea what it is. But you can specify so many filters, right on command line, right? So FFmpeg is this collection of toolbox for multimedia processing that everyone, everyone uses. And everyone that is watching your videos are also using, right? You’re on YouTube. Well, it’s FFmpeg on the client side. Well, the server side, on the server side. The client side is probably Chrome. Well, you’re using FFmpeg also.
Jean-Baptiste Kempf
And you’re using OBS to record. Well, it’s FFmpeg, right? You’re using a ton of important, like, big box, professional boxes. Well, it’s very possible that inside some part of FFmpeg is running.
And you’re using OBS to record. Well, it’s FFmpeg, right? You’re using a ton of important, like, big box, professional boxes. Well, it’s very possible that inside some part of FFmpeg is running.
Lex Fridman
I mean, there’s like so many, just to give people an idea, like I use FFmpeg a lot on everything. Just trivial stuff like take a video, add an intro video and an outro video, and fade one into the other like what is it called? Dip to black, like where it dips and then shows the next video and does the same thing with audio. There’s like a cross dissolve of the audio. It quiets the audio and makes it loud again. And then there’s a bunch of stuff like showing the captions on screen card, like baking the captions in. You can customize the font. You can do all kinds of layering of audio and video. There’s a million things and of course, all of that works like magically with basically any codec. Like anything you can shove in on the audio and the video side, it works.
I mean, there’s like so many, just to give people an idea, like I use FFmpeg a lot on everything. Just trivial stuff like take a video, add an intro video and an outro video, and fade one into the other like what is it called? Dip to black, like where it dips and then shows the next video and does the same thing with audio. There’s like a cross dissolve of the audio. It quiets the audio and makes it loud again. And then there’s a bunch of stuff like showing the captions on screen card, like baking the captions in. You can customize the font. You can do all kinds of layering of audio and video. There’s a million things and of course, all of that works like magically with basically any codec. Like anything you can shove in on the audio and the video side, it works.
Jean-Baptiste Kempf
But it’s like if you look at, for example, you can do things that you would do with Adobe After Effects- … in command line on FFmpeg, right? It’s, and it’s very interesting because, for example, for images, there is not such tool. There is a few tools, but not with the breadth of FFmpeg.
But it’s like if you look at, for example, you can do things that you would do with Adobe After Effects- … in command line on FFmpeg, right? It’s, and it’s very interesting because, for example, for images, there is not such tool. There is a few tools, but not with the breadth of FFmpeg.
Lex Fridman
So ImageMagick has a similar kind of-
So ImageMagick has a similar kind of-
Jean-Baptiste Kempf
Yes, but you will not-
Yes, but you will not-
Lex Fridman
… spirit, but it-
… spirit, but it-
Jean-Baptiste Kempf
… do some filters, complex filters. You don’t have the equivalent of Photoshop- … in command line, right? But for video, you have FFmpeg in command line.
… do some filters, complex filters. You don’t have the equivalent of Photoshop- … in command line, right? But for video, you have FFmpeg in command line.
Lex Fridman
Yeah. It’s incredible. I mean, it’s like an, it’s an example of a thing when a bunch of great people get together and they get a vision, and they stick by that vision for many years, which is incredible.
Yeah. It’s incredible. I mean, it’s like an, it’s an example of a thing when a bunch of great people get together and they get a vision, and they stick by that vision for many years, which is incredible.
Jean-Baptiste Kempf
And the vision behind, and the same for VLC and FFmpeg, is that we make everything that is very complex easy to use for the normal people, for everyone. Right? Our goal is to make something that is insanely complex technically and make it easy to use, right? And people, they use VLC, they drop a file. They don’t realize how complex the file is, but they play it. Or people put any type of thing inside FFmpeg with complex filters, and it just works like magically, right? And people- And this is our mission, right? Make very complex things.
And the vision behind, and the same for VLC and FFmpeg, is that we make everything that is very complex easy to use for the normal people, for everyone. Right? Our goal is to make something that is insanely complex technically and make it easy to use, right? And people, they use VLC, they drop a file. They don’t realize how complex the file is, but they play it. Or people put any type of thing inside FFmpeg with complex filters, and it just works like magically, right? And people- And this is our mission, right? Make very complex things.
Kieran Kunhya
We wouldn’t be here and you wouldn’t be here if this required, you know, a traditional television studio setup. It’s tools like FFmpeg that democratize this. The podcast and streaming revolution, the YouTube revolution- … was caused. You know, FFmpeg was a big player in that because it democratized this technology that was once in the nineties, for example, you needed equipment that cost hundreds of thousands of dollars to do compression. It was the size of a car, and now everybody has that at almost an exact level playing field, and that’s something that’s so remarkable.
We wouldn’t be here and you wouldn’t be here if this required, you know, a traditional television studio setup. It’s tools like FFmpeg that democratize this. The podcast and streaming revolution, the YouTube revolution- … was caused. You know, FFmpeg was a big player in that because it democratized this technology that was once in the nineties, for example, you needed equipment that cost hundreds of thousands of dollars to do compression. It was the size of a car, and now everybody has that at almost an exact level playing field, and that’s something that’s so remarkable.
Lex Fridman
It gave voice to a lot of people. And just to clarify, we say you wouldn’t be here—not the human, but the podcast.
It gave voice to a lot of people. And just to clarify, we say you wouldn’t be here—not the human, but the podcast.
Jean-Baptiste Kempf
The podcast. Oh, sorry. You as a… Sorry.
The podcast. Oh, sorry. You as a… Sorry.
Lex Fridman
I would still… VLC did not have anything to do on a biological level- … at creating me as a human.
I would still… VLC did not have anything to do on a biological level- … at creating me as a human.
Jean-Baptiste Kempf
But it’s like you realize also everything moved from text to images and images to video, right? Look at social networks. Video is everywhere. It’s the most powerful medium there is, right? And when you see shorts and Reels and TikTok, right? It’s amazingly powerful to give… Video is amazing for that, right? But the complexity is important.
But it’s like you realize also everything moved from text to images and images to video, right? Look at social networks. Video is everywhere. It’s the most powerful medium there is, right? And when you see shorts and Reels and TikTok, right? It’s amazingly powerful to give… Video is amazing for that, right? But the complexity is important.
Lex Fridman
This is what people don’t realize. I mean, this is really—it gave power to the individual all across the world. That’s real freedom. And I think, I can’t believe it, but we still haven’t mentioned the actual obvious thing for people who are not familiar, which it’s open source, and there’s an open source community of users and developers behind it. So it’s really a movement. So, like, we’ll talk in a bunch of different ways about the community behind it. But can you speak to the open source element? So when we say what is FFmpeg, it’s an open source project.
This is what people don’t realize. I mean, this is really—it gave power to the individual all across the world. That’s real freedom. And I think, I can’t believe it, but we still haven’t mentioned the actual obvious thing for people who are not familiar, which it’s open source, and there’s an open source community of users and developers behind it. So it’s really a movement. So, like, we’ll talk in a bunch of different ways about the community behind it. But can you speak to the open source element? So when we say what is FFmpeg, it’s an open source project.
Jean-Baptiste Kempf
Yeah. So FFmpeg, VLC, x264, VideoLAN, everything we do is fully open source. And for the people who don’t understand how open source is, my usual analogy is about a chocolate cheesecake, right? Usually for you, when you want to buy your cheesecake, you go to a bakery, they give you the cheesecake. The other one way of having a cheesecake is have your grandma give you a recipe of how to make that. When we do open source, we give you the chocolate cake, and we give you the recipe to actually remake the same cake, but at the same time tell you how to build the oven and also how you’re allowed to modify the recipe and resell it to someone else.
Yeah. So FFmpeg, VLC, x264, VideoLAN, everything we do is fully open source. And for the people who don’t understand how open source is, my usual analogy is about a chocolate cheesecake, right? Usually for you, when you want to buy your cheesecake, you go to a bakery, they give you the cheesecake. The other one way of having a cheesecake is have your grandma give you a recipe of how to make that. When we do open source, we give you the chocolate cake, and we give you the recipe to actually remake the same cake, but at the same time tell you how to build the oven and also how you’re allowed to modify the recipe and resell it to someone else.
Jean-Baptiste Kempf
And this is because software is just a very long recipe of small instructions. Computers are not very clever. They go very, very fast. So a normal program has tens of billions of instructions instead of the tens when you have your chocolate recipe. So a lot of the software industry was about selling software where you just have the final cheesecake. In open source, we give you everything, and that managed to get a lot of people to work together, right? Because then you decide that you’re going to make the best program, the best recipe for video, and you create communities. In FFmpeg, since the beginning of FFmpeg, probably two thousand to three thousand-
And this is because software is just a very long recipe of small instructions. Computers are not very clever. They go very, very fast. So a normal program has tens of billions of instructions instead of the tens when you have your chocolate recipe. So a lot of the software industry was about selling software where you just have the final cheesecake. In open source, we give you everything, and that managed to get a lot of people to work together, right? Because then you decide that you’re going to make the best program, the best recipe for video, and you create communities. In FFmpeg, since the beginning of FFmpeg, probably two thousand to three thousand-
Lex Fridman
In the thousands, yeah
In the thousands, yeah
Jean-Baptiste Kempf
… people have contributed from the beginning, right? And then it’s exactly like the Linux kernel, right? The Linux kernel has probably ten thousand people contributing everywhere, and they get together, well, mostly online, right? So they virtually get together to create the best tool for something. And on FFmpeg and VLC, it’s just like, well, this codec doesn’t work, so I’m going to work on the codec, and I’m going to add the support for this file inside FFmpeg, so it will be beneficial to everyone. Because again, we work for the greater good. We work for everyone, and that is what open source is.
… people have contributed from the beginning, right? And then it’s exactly like the Linux kernel, right? The Linux kernel has probably ten thousand people contributing everywhere, and they get together, well, mostly online, right? So they virtually get together to create the best tool for something. And on FFmpeg and VLC, it’s just like, well, this codec doesn’t work, so I’m going to work on the codec, and I’m going to add the support for this file inside FFmpeg, so it will be beneficial to everyone. Because again, we work for the greater good. We work for everyone, and that is what open source is.
Lex Fridman
And we should mention, depending on the licensing you could probably build a billion-dollar, maybe even a trillion-dollar company as a wrapper to—
And we should mention, depending on the licensing you could probably build a billion-dollar, maybe even a trillion-dollar company as a wrapper to—
Jean-Baptiste Kempf
Well, yes— … people do. People do, right? There were a lot of problems with mostly cloud providers who are basically running some open source tools in the cloud and just give you the API to access to that. And there was a lot of databases like Mongo or Elastic who changed their license in order to avoid those types of scenarios.
Well, yes— … people do. People do, right? There were a lot of problems with mostly cloud providers who are basically running some open source tools in the cloud and just give you the API to access to that. And there was a lot of databases like Mongo or Elastic who changed their license in order to avoid those types of scenarios.
Kieran Kunhya
This is a question we get a lot in FFmpeg is, “Why don’t you do that?” And you can’t. We have, we have thousands of contributors, some of whom aren’t even alive anymore. It would need all of their agreement to do that, and JB will go maybe a bit later and talk about how challenging that process was in VLC to do the re-licensing.
This is a question we get a lot in FFmpeg is, “Why don’t you do that?” And you can’t. We have, we have thousands of contributors, some of whom aren’t even alive anymore. It would need all of their agreement to do that, and JB will go maybe a bit later and talk about how challenging that process was in VLC to do the re-licensing.
Jean-Baptiste Kempf
The license is a social contract in terms of Rousseau de facto of the community. The community does not agree on much besides the license. People go around, discuss around because of the license, and that also allow those license forks, right? Sometimes the community splits, but it’s possible because of the license and to merge back.
The license is a social contract in terms of Rousseau de facto of the community. The community does not agree on much besides the license. People go around, discuss around because of the license, and that also allow those license forks, right? Sometimes the community splits, but it’s possible because of the license and to merge back.
Jean-Baptiste Kempf
And we’ve seen that so many times, right? GCC and EGCS in the past. We have seen, for example, all the web browsers, right? They started as web, like KHTML, which becomes WebKit and then which becomes Blink, right? So open source license is like the core of the community and people are coming from all around the world, very different types of religion, political borders. They work in the same way on a project to solve a specific problem, and the specific problem we’re working on is to make multimedia easy for everyone.
And we’ve seen that so many times, right? GCC and EGCS in the past. We have seen, for example, all the web browsers, right? They started as web, like KHTML, which becomes WebKit and then which becomes Blink, right? So open source license is like the core of the community and people are coming from all around the world, very different types of religion, political borders. They work in the same way on a project to solve a specific problem, and the specific problem we’re working on is to make multimedia easy for everyone.
Lex Fridman
Looking it up on Perplexity here, looking at the different open source licenses. Most major open source licenses fall into two buckets: permissive, very few conditions, and copyleft, share-alike requirements for derivatives. Below is a brief practical summary of the main ones you’ll see in the wild. MIT license, BSD, ISC Apache GNU GPL, GNU AGPL. Where’s LGPL? Yeah, LGPL. Let’s see. There’s the Mo- the Mozilla Public License. There’s Eclipse Public License. It goes on. There’s a lot of variety. I mean, I think the really popular ones are MIT, GPL, LGPL-
Looking it up on Perplexity here, looking at the different open source licenses. Most major open source licenses fall into two buckets: permissive, very few conditions, and copyleft, share-alike requirements for derivatives. Below is a brief practical summary of the main ones you’ll see in the wild. MIT license, BSD, ISC Apache GNU GPL, GNU AGPL. Where’s LGPL? Yeah, LGPL. Let’s see. There’s the Mo- the Mozilla Public License. There’s Eclipse Public License. It goes on. There’s a lot of variety. I mean, I think the really popular ones are MIT, GPL, LGPL-
Jean-Baptiste Kempf
Yeah. And BSD.
Yeah. And BSD.
Lex Fridman
… and BSD, Apache. Sometimes you’ll see-
… and BSD, Apache. Sometimes you’ll see-
Jean-Baptiste Kempf
Apache as well
Apache as well
Lex Fridman
… Apache. Unlicense, that’s an option. Attempts to dedicate code to the public domain with a fallback permissive license.
… Apache. Unlicense, that’s an option. Attempts to dedicate code to the public domain with a fallback permissive license.
Jean-Baptiste Kempf
There are many licenses for many different things. What people don’t understand is that public domain is something that doesn’t exist worldwide, right? So all the open source licensing use the copyright law, right, the international copyright law, in order to give rights on how you use the software or how you modify. It’s de facto a copyright license contract that you give to the end user or to the developer. And so you have like the first ones, which are basically very permissive, MIT, BSD. You give the code and basically you do whatever you want, right? You take it, you want, you modify, you do what you want. And this is popular for JavaScript and the type of BSD operating system.
There are many licenses for many different things. What people don’t understand is that public domain is something that doesn’t exist worldwide, right? So all the open source licensing use the copyright law, right, the international copyright law, in order to give rights on how you use the software or how you modify. It’s de facto a copyright license contract that you give to the end user or to the developer. And so you have like the first ones, which are basically very permissive, MIT, BSD. You give the code and basically you do whatever you want, right? You take it, you want, you modify, you do what you want. And this is popular for JavaScript and the type of BSD operating system.
Lex Fridman
So some of them, one of the parameters is whether they require attribution, meaning if you use the code, you have to say-
So some of them, one of the parameters is whether they require attribution, meaning if you use the code, you have to say-
Jean-Baptiste Kempf
Yes. So in those types of permissive licenses, some you need to say if you use it, which is called attribution, and some you don’t. And then there is the other part of license which are copyleft, where you need to give back to the community your modifications and with different strings attached. Some weak copyleft licenses, like the Mozilla Public License, to some which are a bit stronger like a GPL, or even very strong like AGPL. So all of those are different types of licensing that depends on what your goals are and how you want to structure your community, which is why I spoke about social contract, because this is very important to understand. FFmpeg and VLC are mostly GPL or LGPL. The Linux kernel is GPL but Android is Apache.
Yes. So in those types of permissive licenses, some you need to say if you use it, which is called attribution, and some you don’t. And then there is the other part of license which are copyleft, where you need to give back to the community your modifications and with different strings attached. Some weak copyleft licenses, like the Mozilla Public License, to some which are a bit stronger like a GPL, or even very strong like AGPL. So all of those are different types of licensing that depends on what your goals are and how you want to structure your community, which is why I spoke about social contract, because this is very important to understand. FFmpeg and VLC are mostly GPL or LGPL. The Linux kernel is GPL but Android is Apache.
Jean-Baptiste Kempf
A ton of JavaScript frameworks that are used are mostly MIT. All the BSD kernels, OpenBSD, NetBSD are of course BSD. And so the… it’s a philosophical change on how you want people to contribute back- … basically.
A ton of JavaScript frameworks that are used are mostly MIT. All the BSD kernels, OpenBSD, NetBSD are of course BSD. And so the… it’s a philosophical change on how you want people to contribute back- … basically.
Lex Fridman
So there’s I think you talked about that you’ve moved at one point from GPL to LGPL on certain parts of the project. What… Can you describe the difference between the two, and what does it take to move to, I guess, a more permissive… So that direction is more permissive. LGPL is more permissive than GPL.
So there’s I think you talked about that you’ve moved at one point from GPL to LGPL on certain parts of the project. What… Can you describe the difference between the two, and what does it take to move to, I guess, a more permissive… So that direction is more permissive. LGPL is more permissive than GPL.
Jean-Baptiste Kempf
Yeah. So you have to realize that you can always go from more permissive to less permissive, right? Because of course, those licenses are basically statements, and so if you restrict, you can always restrict more, right? So in a GPL project, you can take MIT code, but you cannot do the opposite, right? Because they are more constrained to match. Indeed, in fact, I changed the core of LibVLC, which is the engine of VLC- … from GPL to LGPL. And there were two reasons to do that. The first one is that so people can use the VLC engine, LibVLC, into third-party applications. So a lot of applications which are playing video on your phone or on your tablet are actually VLC engine in it- … which is calling FFmpeg in it.
Yeah. So you have to realize that you can always go from more permissive to less permissive, right? Because of course, those licenses are basically statements, and so if you restrict, you can always restrict more, right? So in a GPL project, you can take MIT code, but you cannot do the opposite, right? Because they are more constrained to match. Indeed, in fact, I changed the core of LibVLC, which is the engine of VLC- … from GPL to LGPL. And there were two reasons to do that. The first one is that so people can use the VLC engine, LibVLC, into third-party applications. So a lot of applications which are playing video on your phone or on your tablet are actually VLC engine in it- … which is calling FFmpeg in it.
Jean-Baptiste Kempf
So that was one of the ways to create one of the companies I created, which is doing consulting and integration of those types of applications where you integrate VLC into third-party solutions like inside game engines or stuff like that. With GPL, you couldn’t do that because that means you needed to open source everything, and those are for a lot of, like, commercial companies who don’t want that.
So that was one of the ways to create one of the companies I created, which is doing consulting and integration of those types of applications where you integrate VLC into third-party solutions like inside game engines or stuff like that. With GPL, you couldn’t do that because that means you needed to open source everything, and those are for a lot of, like, commercial companies who don’t want that.
Lex Fridman
So you can create a company with LGPL, you can create a company around it. You can do a commercial thing. You don’t have to open source it. So that’s a big, big leap.
So you can create a company with LGPL, you can create a company around it. You can do a commercial thing. You don’t have to open source it. So that’s a big, big leap.
Kieran Kunhya
So you could play video in your game. The problem is I’m a game developer, and I want to play some videos- … and I don’t want to be forced to open source the entire game just to play those videos. So that’s where the, the consulting business, the libVLC LGPL- … allows you to do that. The LGPL, the library GPL as it used to be known, allows you to do that.
So you could play video in your game. The problem is I’m a game developer, and I want to play some videos- … and I don’t want to be forced to open source the entire game just to play those videos. So that’s where the, the consulting business, the libVLC LGPL- … allows you to do that. The LGPL, the library GPL as it used to be known, allows you to do that.
Jean-Baptiste Kempf
And FFmpeg is exactly the same. It force… LGPL forces you to give back what you change on this component, this- … library, which is why it’s library GPL. And so you can use FFmpeg as LGPL into, like, any type of application, even non-open source, but you need to give back the modification you did on FFmpeg. Same on libVLC.
And FFmpeg is exactly the same. It force… LGPL forces you to give back what you change on this component, this- … library, which is why it’s library GPL. And so you can use FFmpeg as LGPL into, like, any type of application, even non-open source, but you need to give back the modification you did on FFmpeg. Same on libVLC.
Lex Fridman
Is it limiting from an open source perspective to go GPL? Because if your library, if your code is GPL, it means you’re basically discouraging companies from building a business- … around it, right? Is that, is that fair to say?
Is it limiting from an open source perspective to go GPL? Because if your library, if your code is GPL, it means you’re basically discouraging companies from building a business- … around it, right? Is that, is that fair to say?
Jean-Baptiste Kempf
It depends on the company, but for the company whose business model requires the application to be closed source, yes, it’s limited. So that’s why, for example, I moved to LGPL. The second reason is a bit more obscure: it’s that the terms of condition of the App Store, the Apple App Store for iOS, makes it very complex to have GPL applications on it, while it’s easier to have LGPL applications on it. So VLC on Windows and on Mac and on Linux is GPL. The core is LGPL. But on iOS, the iPhone version and the Apple TV version is a type of different license called the MPL. And yes, I went and changed the license and it was a long story.
It depends on the company, but for the company whose business model requires the application to be closed source, yes, it’s limited. So that’s why, for example, I moved to LGPL. The second reason is a bit more obscure: it’s that the terms of condition of the App Store, the Apple App Store for iOS, makes it very complex to have GPL applications on it, while it’s easier to have LGPL applications on it. So VLC on Windows and on Mac and on Linux is GPL. The core is LGPL. But on iOS, the iPhone version and the Apple TV version is a type of different license called the MPL. And yes, I went and changed the license and it was a long story.
Lex Fridman
Yeah. So I think basically to change the license you have to contact all the contributors.
Yeah. So I think basically to change the license you have to contact all the contributors.
Jean-Baptiste Kempf
Yes. It’s very important to understand that open source projects are what we call in the US copyright law joint work, or in civil law collective works or collaborative works. It’s that you work all together in terms of the same goal, and then you create one software, which is one release. But the copyright is kept by all the individuals. Some open source projects don’t do that. They force copyright assignment, but this is not what we do. We’re communities. So everyone has basically copyright on what they changed. And this copyright stays even if at the end your contribution was deleted because the new contribution was based on your previous one, right? So if you want to properly re-license, you need to find all the contributors.
Yes. It’s very important to understand that open source projects are what we call in the US copyright law joint work, or in civil law collective works or collaborative works. It’s that you work all together in terms of the same goal, and then you create one software, which is one release. But the copyright is kept by all the individuals. Some open source projects don’t do that. They force copyright assignment, but this is not what we do. We’re communities. So everyone has basically copyright on what they changed. And this copyright stays even if at the end your contribution was deleted because the new contribution was based on your previous one, right? So if you want to properly re-license, you need to find all the contributors.
Jean-Baptiste Kempf
And at that time, I had to contact more than three hundred and fifty people. And sometimes, well, they’re just an email, right? So it’s… you need to actually track down. I actually, like, travel to some place to go someone that I was like, sorry, that I’d found online to see a– to go to their job and say, “Well, you licensed that. Can you– do you want to change from GPL to LGPL?” Most of the times they don’t even care. They wanted to help VLC. But also it brought me to very complex situation. I arrived to the work of a person who was a factory worker. And I said, “Well, I need you to sign that,” because it was his son who died who actually wrote the code, right?
And at that time, I had to contact more than three hundred and fifty people. And sometimes, well, they’re just an email, right? So it’s… you need to actually track down. I actually, like, travel to some place to go someone that I was like, sorry, that I’d found online to see a– to go to their job and say, “Well, you licensed that. Can you– do you want to change from GPL to LGPL?” Most of the times they don’t even care. They wanted to help VLC. But also it brought me to very complex situation. I arrived to the work of a person who was a factory worker. And I said, “Well, I need you to sign that,” because it was his son who died who actually wrote the code, right?
Jean-Baptiste Kempf
So I had to explain all those types of open source meanings, and no, I was not a company trying to rip out the two lines or five lines that that guy did- … but was useful, and the whole community agreed on that, and he had no idea I was a factory worker. And I was a lot younger, right? Like it was fourteen years ago, and like I was almost in tears, right? It’s very difficult, right? We are talking about lives of people and he explaining, and we talked about the photo of this guy, right? So it’s important to do it right and to do it correctly. But yes, that means tracking down everything because every contribution works. There are some projects who don’t respect that, and we do re-licensing a bit, like, aggressively.
So I had to explain all those types of open source meanings, and no, I was not a company trying to rip out the two lines or five lines that that guy did- … but was useful, and the whole community agreed on that, and he had no idea I was a factory worker. And I was a lot younger, right? Like it was fourteen years ago, and like I was almost in tears, right? It’s very difficult, right? We are talking about lives of people and he explaining, and we talked about the photo of this guy, right? So it’s important to do it right and to do it correctly. But yes, that means tracking down everything because every contribution works. There are some projects who don’t respect that, and we do re-licensing a bit, like, aggressively.
Jean-Baptiste Kempf
But as I said, it destroyed the whole heart of the community because it’s– we only agree on the license, so that’s important.
But as I said, it destroyed the whole heart of the community because it’s– we only agree on the license, so that’s important.
Kieran Kunhya
I would emphasize the community is such a wide-ranging group of people. There’s people in the Syrian war zone with electricity part-time. There’s… there’s people from all walks of life- … rich, poor, young, old. So it’s quite remarkable to get, you know, a group of people aligned on something. And that’s an achievement in itself.
I would emphasize the community is such a wide-ranging group of people. There’s people in the Syrian war zone with electricity part-time. There’s… there’s people from all walks of life- … rich, poor, young, old. So it’s quite remarkable to get, you know, a group of people aligned on something. And that’s an achievement in itself.
Lex Fridman
Yeah. It’s incredible. And a lot of them are introverts, so you coming to find them and getting them and getting them to answer an email might be quite, quite difficult.
Yeah. It’s incredible. And a lot of them are introverts, so you coming to find them and getting them and getting them to answer an email might be quite, quite difficult.
Jean-Baptiste Kempf
Most of us are introverts, right? You need to be more precise. You have extremely introverts, extremely, extremely introverts and introverts, right?
Most of us are introverts, right? You need to be more precise. You have extremely introverts, extremely, extremely introverts and introverts, right?
Lex Fridman
Yeah.
Yeah.
Jean-Baptiste Kempf
It’s just like a whole spectrum of different people. It doesn’t matter. The important thing is, is your code good? Is your code great? Is your technology great? We care about excellent code. We don’t care who you are. Sorry, it’s just like we have no idea to check. We cannot check, right? Like, maybe you’re a dog. I don’t care, right? I don’t care where you come from. I need to look at your code. And this is important because people don’t understand that, and they come to the community and send them some patches, and they get rejected, and they don’t like that because, I mean, you’re just like, “Sorry, it’s not up to our standards.” “Oh, yeah, but I’m an engineer at this very large company in Italy, in Germany, in the US.” We don’t care.
It’s just like a whole spectrum of different people. It doesn’t matter. The important thing is, is your code good? Is your code great? Is your technology great? We care about excellent code. We don’t care who you are. Sorry, it’s just like we have no idea to check. We cannot check, right? Like, maybe you’re a dog. I don’t care, right? I don’t care where you come from. I need to look at your code. And this is important because people don’t understand that, and they come to the community and send them some patches, and they get rejected, and they don’t like that because, I mean, you’re just like, “Sorry, it’s not up to our standards.” “Oh, yeah, but I’m an engineer at this very large company in Italy, in Germany, in the US.” We don’t care.
Jean-Baptiste Kempf
We care about the quality of your code because this is what defines our community, which means that we have a lot of people who contribute who are from some very different backgrounds and, and, and very introverts, sure. But that’s okay, right?
We care about the quality of your code because this is what defines our community, which means that we have a lot of people who contribute who are from some very different backgrounds and, and, and very introverts, sure. But that’s okay, right?
Linus Torvalds
Lex Fridman
So one of the legends of the community is of course, Linus Torvalds, who created Linux and is a longtime maintainer of the Linux kernel. As the legend goes, he can be pretty harsh on this meritocratic process of reviewing the code and saying it’s not good enough. Can you just speak to the legend of Linus Torvalds?
So one of the legends of the community is of course, Linus Torvalds, who created Linux and is a longtime maintainer of the Linux kernel. As the legend goes, he can be pretty harsh on this meritocratic process of reviewing the code and saying it’s not good enough. Can you just speak to the legend of Linus Torvalds?
Jean-Baptiste Kempf
Linus is one of a kind, right? And, and I would even go and say that what he did on Git is more interesting than what he did on the Linux kernel.
Linus is one of a kind, right? And, and I would even go and say that what he did on Git is more interesting than what he did on the Linux kernel.
Jean-Baptiste Kempf
He’s very harsh, but what people don’t see is usually when he’s harsh, it’s to people who are maintainers of part of the kernel, right? So they know him, right? So he’s not very harsh like that to everyone. The thing is, what he created in his room is basically powering every server online, right? Even at Microsoft’s cloud called Azure, I’m quite sure seventy, eighty percent of the servers are running Linux. All your Android phones are running Linux. What he did with the power of open source, sure, is amazing. And yes, the quality of the Linux kernel is very high, and yes, it’s difficult, but we cannot compromise on that. We cannot compromise on quality because in the end—and you have to understand that—the core community of VLC is five people.
He’s very harsh, but what people don’t see is usually when he’s harsh, it’s to people who are maintainers of part of the kernel, right? So they know him, right? So he’s not very harsh like that to everyone. The thing is, what he created in his room is basically powering every server online, right? Even at Microsoft’s cloud called Azure, I’m quite sure seventy, eighty percent of the servers are running Linux. All your Android phones are running Linux. What he did with the power of open source, sure, is amazing. And yes, the quality of the Linux kernel is very high, and yes, it’s difficult, but we cannot compromise on that. We cannot compromise on quality because in the end—and you have to understand that—the core community of VLC is five people.
Jean-Baptiste Kempf
The core community of FFmpeg is ten to fifteen, and we are the ones who are going to maintain your code, right? Because one thousand contributors in the timeline and just ten staying, it’s a one percent chance that someone comes and stays. One percent. So you will have change of job, change of wives, you have children, you have accidents in life. You’re going to change jobs, whatever. You’re not going to come back. It’s most likely. So we are the ones going to maintain your code. It needs to be maintainable. It needs to be excellent. And yes, sometimes that means that you need to rework your work because it was good, but it’s not excellent, and we need excellence because we are very few to maintain something that is critical for the whole.
The core community of FFmpeg is ten to fifteen, and we are the ones who are going to maintain your code, right? Because one thousand contributors in the timeline and just ten staying, it’s a one percent chance that someone comes and stays. One percent. So you will have change of job, change of wives, you have children, you have accidents in life. You’re going to change jobs, whatever. You’re not going to come back. It’s most likely. So we are the ones going to maintain your code. It needs to be maintainable. It needs to be excellent. And yes, sometimes that means that you need to rework your work because it was good, but it’s not excellent, and we need excellence because we are very few to maintain something that is critical for the whole.
Lex Fridman
But we should also mention that there is some spiciness, some harshness to the language that’s sometimes used when you’re keeping this high bar of excellence. Is there something to say to that?
But we should also mention that there is some spiciness, some harshness to the language that’s sometimes used when you’re keeping this high bar of excellence. Is there something to say to that?
Jean-Baptiste Kempf
It’s true, right? It’s also the fact that, for example, what we’re doing is low level. It’s extremely technical. You get into this community. The tone gets very like a type of— It’s a subculture, right? So people who arrive from the external are basically not known to the subculture. Most of those people around FFmpeg and VLC, we do VideoLAN DevDays, VDD every year. They are so fun in real life, and they love it. But it’s true that you’re online and sometimes, like, the tone, you don’t realize how it is. But that’s okay.
It’s true, right? It’s also the fact that, for example, what we’re doing is low level. It’s extremely technical. You get into this community. The tone gets very like a type of— It’s a subculture, right? So people who arrive from the external are basically not known to the subculture. Most of those people around FFmpeg and VLC, we do VideoLAN DevDays, VDD every year. They are so fun in real life, and they love it. But it’s true that you’re online and sometimes, like, the tone, you don’t realize how it is. But that’s okay.
Lex Fridman
It’s a culture. I mean, you get this in the gaming culture. There’s pretty harsh, intense, the way people communicate, and it’s— everyone understands that the way you show love and respect just looks different in different communities. Sometimes people… It depends. If it’s a book club, usually people are going to be much sweeter. If it’s an open source project that’s very high stakes and used by millions of people-
It’s a culture. I mean, you get this in the gaming culture. There’s pretty harsh, intense, the way people communicate, and it’s— everyone understands that the way you show love and respect just looks different in different communities. Sometimes people… It depends. If it’s a book club, usually people are going to be much sweeter. If it’s an open source project that’s very high stakes and used by millions of people-
Jean-Baptiste Kempf
But it’s very not often insults that you see, for example, in the gaming, right? And so Linus’ tone is a bit unusual even for the open source community. It’s more like it’s more harsh on the results, saying, “No, this is not good. This is crap.” Those type of things that you will see.
But it’s very not often insults that you see, for example, in the gaming, right? And so Linus’ tone is a bit unusual even for the open source community. It’s more like it’s more harsh on the results, saying, “No, this is not good. This is crap.” Those type of things that you will see.
Lex Fridman
Try not to make it about the person, make it about the code.
Try not to make it about the person, make it about the code.
Jean-Baptiste Kempf
Yes.
Yes.
Kieran Kunhya
It’s very, very matter of fact, and I think you’ve got to look at it in terms of, you know, the famous FFmpeg is developed almost entirely by volunteers, and that’s true, and you’ve got to imagine someone’s done a hard day’s work at their day job. They come home. You know, terseness might be a thing, you know, it… And that’s not something to take personally.
It’s very, very matter of fact, and I think you’ve got to look at it in terms of, you know, the famous FFmpeg is developed almost entirely by volunteers, and that’s true, and you’ve got to imagine someone’s done a hard day’s work at their day job. They come home. You know, terseness might be a thing, you know, it… And that’s not something to take personally.
Lex Fridman
You’re tired, you’re busy, but you still care about this open source stuff. But you may not be able to explain and handhold someone on every subtle detail.
You’re tired, you’re busy, but you still care about this open source stuff. But you may not be able to explain and handhold someone on every subtle detail.
Jean-Baptiste Kempf
And also you have to realize that most people don’t speak English as native language. And this is especially for open source projects like FFmpeg and VLC, which are mostly centered out of Europe. Sometimes people who are from the US or just are very not happy about the tone, but most of the time it’s also like they don’t know better, right? It’s difficult. The language is—English is a difficult language. There is so many subtleties and tone and so on that you don’t have, right? So often it’s also difficult in those type of communities about different cultures and languages.
And also you have to realize that most people don’t speak English as native language. And this is especially for open source projects like FFmpeg and VLC, which are mostly centered out of Europe. Sometimes people who are from the US or just are very not happy about the tone, but most of the time it’s also like they don’t know better, right? It’s difficult. The language is—English is a difficult language. There is so many subtleties and tone and so on that you don’t have, right? So often it’s also difficult in those type of communities about different cultures and languages.
Turning down millions to keep VLC ad-free
Lex Fridman
So as the legend goes, JB, you repeatedly turned down millions of dollars to keep VLC open source free for everyone without ads. So take me through the reasoning behind that decision of leaving millions of dollars on the table.
So as the legend goes, JB, you repeatedly turned down millions of dollars to keep VLC open source free for everyone without ads. So take me through the reasoning behind that decision of leaving millions of dollars on the table.
Jean-Baptiste Kempf
Yeah, that’s like almost a meme, right, on Reddit or-
Yeah, that’s like almost a meme, right, on Reddit or-
Lex Fridman
There literally is a meme on Reddit.
There literally is a meme on Reddit.
Jean-Baptiste Kempf
9GAG and yeah, yeah. See, there’s-
9GAG and yeah, yeah. See, there’s-
Lex Fridman
You looking like a wizard in the, in the VLC hat on Reddit. This is JB, the creator of VLC media player. He refused tens of millions of dollars in order to keep VLC ads free. Thanks, Jean-Baptiste Kempf. You can even summon him on Reddit.
You looking like a wizard in the, in the VLC hat on Reddit. This is JB, the creator of VLC media player. He refused tens of millions of dollars in order to keep VLC ads free. Thanks, Jean-Baptiste Kempf. You can even summon him on Reddit.
Jean-Baptiste Kempf
Yeah. And usually if you see, right, it’s usually like people tag me, right? And then there is me, and then like I say, “Good morning.”
Yeah. And usually if you see, right, it’s usually like people tag me, right? And then there is me, and then like I say, “Good morning.”
Lex Fridman
Good morning.
Good morning.
Jean-Baptiste Kempf
I got twenty-four K upvotes, which is great, right? My karma on Reddit is amazing, at least on that account. So the question is, needs to be answered first, what is the story about VLC, right? Because yes, this is true, I refuse dozens of millions of dollars, yes, several times. Yes, I could be a multimillionaire and be somewhere on the beach. But I did not do it because I thought it was not moral and it was not the right thing to do. And this is very important for myself, is to be like, I work for the greater good, I work for people, and I don’t want—it’s not just by myself. But the reason is also because I did not feel that I’m completely legitimate to do that, and let me explain you why. VLC’s story is a very weird story.
I got twenty-four K upvotes, which is great, right? My karma on Reddit is amazing, at least on that account. So the question is, needs to be answered first, what is the story about VLC, right? Because yes, this is true, I refuse dozens of millions of dollars, yes, several times. Yes, I could be a multimillionaire and be somewhere on the beach. But I did not do it because I thought it was not moral and it was not the right thing to do. And this is very important for myself, is to be like, I work for the greater good, I work for people, and I don’t want—it’s not just by myself. But the reason is also because I did not feel that I’m completely legitimate to do that, and let me explain you why. VLC’s story is a very weird story.
Jean-Baptiste Kempf
In France, we have university and we have a type of top colleges, and those top of excellency schools are engineering schools, business schools, and basically lawyers and medical, right? But they’re outside of university, and in order to enter those, you spend two years working like crazy math, physics to enter those best engineering schools. One of the school is called the École Centrale Paris. It has changed name since, but it was called the École Centrale Paris. And because it was Centrale, they had to move it because it was too small after the World War II and they moved it, they wanted to move it to the central of France in a place called Clermont-Ferrand. And the alumni decided that this was not okay, right?
In France, we have university and we have a type of top colleges, and those top of excellency schools are engineering schools, business schools, and basically lawyers and medical, right? But they’re outside of university, and in order to enter those, you spend two years working like crazy math, physics to enter those best engineering schools. One of the school is called the École Centrale Paris. It has changed name since, but it was called the École Centrale Paris. And because it was Centrale, they had to move it because it was too small after the World War II and they moved it, they wanted to move it to the central of France in a place called Clermont-Ferrand. And the alumni decided that this was not okay, right?
Jean-Baptiste Kempf
It is the school that Eiffel, right, the one who did the Eiffel Tower, attended to, right? So they said, “No, no, we are amazing, great school. We cannot do that.” And so they bought a piece of land south of Paris very near Paris. And it was a campus managed by a nonprofit of the alumnis, okay? Because of that, everything on the campus was managed by students. The university did nothing, right? So radio, TV, supermarket, library, defining who was going into which rooms. Everything was managed by the students.
It is the school that Eiffel, right, the one who did the Eiffel Tower, attended to, right? So they said, “No, no, we are amazing, great school. We cannot do that.” And so they bought a piece of land south of Paris very near Paris. And it was a campus managed by a nonprofit of the alumnis, okay? Because of that, everything on the campus was managed by students. The university did nothing, right? So radio, TV, supermarket, library, defining who was going into which rooms. Everything was managed by the students.
Lex Fridman
That’s amazing. That’s an amazing experiment, that it all didn’t go to hell quickly. It somehow flourished.
That’s amazing. That’s an amazing experiment, that it all didn’t go to hell quickly. It somehow flourished.
Jean-Baptiste Kempf
It worked great, and I learned so much in my life doing those side activities, right? Because you’re twenty-two and you need to run your campus, else you don’t have electricity, right? So you care about that, right? But anyway, in the ’80s they did a full experiment of deploying a network mostly sponsored by IBM and 3Com, which was a token ring network. So token ring is something that probably almost no one knows about anymore. It’s a networking technology where you don’t have routers, right?
It worked great, and I learned so much in my life doing those side activities, right? Because you’re twenty-two and you need to run your campus, else you don’t have electricity, right? So you care about that, right? But anyway, in the ’80s they did a full experiment of deploying a network mostly sponsored by IBM and 3Com, which was a token ring network. So token ring is something that probably almost no one knows about anymore. It’s a networking technology where you don’t have routers, right?
Jean-Baptiste Kempf
Everyone is linked. It’s like really a ring, and when you want to send a message, you talk to your neighbor who’s going to put the message to the next one, who’s going to put the things to the next one, in terms of ring. The issue with token ring is, of course, is that it’s very slow because every computer on the network needs to open the message, see if it’s okay. Is it for me? No, it’s not, and then send it back, like a token which is traveling around the ring. In the ’80s, you’re doing some Telnet and sending mails as university. That’s okay, right? But starts the ’90s, and the ’90s and start video games, and when you have high latency in video games, basically you die, right?
Everyone is linked. It’s like really a ring, and when you want to send a message, you talk to your neighbor who’s going to put the message to the next one, who’s going to put the things to the next one, in terms of ring. The issue with token ring is, of course, is that it’s very slow because every computer on the network needs to open the message, see if it’s okay. Is it for me? No, it’s not, and then send it back, like a token which is traveling around the ring. In the ’80s, you’re doing some Telnet and sending mails as university. That’s okay, right? But starts the ’90s, and the ’90s and start video games, and when you have high latency in video games, basically you die, right?
Jean-Baptiste Kempf
So in nineteen ninety-four, nineteen ninety-five, around Doom and Duke Nukem coming around, they want a faster network. So the students go and see the university and say, “You know what? We want a faster network. We need to work,” and also play video games. And the university tells them that basically, “Oh, I’m sorry, we cannot help you because you understand the campus is not ours. You manage it, so do something. And you should see some basically partners of this university and basically go away.” And they go, and they actually go and see the CIO of Bouygues, which is a large French company and who’s doing some TVs in France. And he says, “Well, you know what?
So in nineteen ninety-four, nineteen ninety-five, around Doom and Duke Nukem coming around, they want a faster network. So the students go and see the university and say, “You know what? We want a faster network. We need to work,” and also play video games. And the university tells them that basically, “Oh, I’m sorry, we cannot help you because you understand the campus is not ours. You manage it, so do something. And you should see some basically partners of this university and basically go away.” And they go, and they actually go and see the CIO of Bouygues, which is a large French company and who’s doing some TVs in France. And he says, “Well, you know what?
Jean-Baptiste Kempf
The future of video is satellite.” Well, today we know it’s not, but at least it was a good idea. In nineteen ninety-five, the first satellite dish, and he says that instead of having like one satellite dish and a big decoder for each of the students, which are one thousand and five hundred, what about you build, like you put an enormous dish and only one decoder, and you send the video directly on the network. And that required a very fast network. Today, it’s obvious, but at the time was, like, the first to do video streaming. So they built this project, which was called Network 2000. Of course, right, we are in the ’90s, right? Everything futuristic is called 2000, like-
The future of video is satellite.” Well, today we know it’s not, but at least it was a good idea. In nineteen ninety-five, the first satellite dish, and he says that instead of having like one satellite dish and a big decoder for each of the students, which are one thousand and five hundred, what about you build, like you put an enormous dish and only one decoder, and you send the video directly on the network. And that required a very fast network. Today, it’s obvious, but at the time was, like, the first to do video streaming. So they built this project, which was called Network 2000. Of course, right, we are in the ’90s, right? Everything futuristic is called 2000, like-
Lex Fridman
Yeah, 2000, yeah.
Yeah, 2000, yeah.
Jean-Baptiste Kempf
And so they do the Network 2000 project. It’s completely hacked. It crashes after 45 seconds. That’s okay. The demo is 40 seconds. It leaks memory. That’s okay. They put 64 megabytes of RAM instead of the 8 or 16 you have, and the demo should have stopped there. And that was the Network 2000 project by the students.
And so they do the Network 2000 project. It’s completely hacked. It crashes after 45 seconds. That’s okay. The demo is 40 seconds. It leaks memory. That’s okay. They put 64 megabytes of RAM instead of the 8 or 16 you have, and the demo should have stopped there. And that was the Network 2000 project by the students.
Lex Fridman
What was the format of the video that they had to work with?
What was the format of the video that they had to work with?
Jean-Baptiste Kempf
MPEG-2 because satellite is MPEG-2 TS for transport, MPEG-2 video, and MPEG-2 audio at that time. And the project should have stopped there. Everyone was happy. They had, like, amazing ATM network at 155 megabits per second. They had probably one of the best networks in Europe at that time, and they stopped the project. Six months or a year later, two students arrive and say, “Well, you know what? Maybe other people care about video streamed on a local network,” and they create the VideoLAN project, VideoLAN. And one of them is called Christophe Massot, that is a good friend of both Kieran and me, and they start the project. It’s not even open source yet, and they spend around three years to get the school to agree to make it open source.
MPEG-2 because satellite is MPEG-2 TS for transport, MPEG-2 video, and MPEG-2 audio at that time. And the project should have stopped there. Everyone was happy. They had, like, amazing ATM network at 155 megabits per second. They had probably one of the best networks in Europe at that time, and they stopped the project. Six months or a year later, two students arrive and say, “Well, you know what? Maybe other people care about video streamed on a local network,” and they create the VideoLAN project, VideoLAN. And one of them is called Christophe Massot, that is a good friend of both Kieran and me, and they start the project. It’s not even open source yet, and they spend around three years to get the school to agree to make it open source.
Jean-Baptiste Kempf
Because the university wanted to get some– because of the IP and copyright of the students, wanted to basically monetize these MPEG-2 decoders.
Because the university wanted to get some– because of the IP and copyright of the students, wanted to basically monetize these MPEG-2 decoders.
Lex Fridman
Just to be clear, so what was the main application, streaming on a local network?
Just to be clear, so what was the main application, streaming on a local network?
Jean-Baptiste Kempf
It was streaming on a local network.
It was streaming on a local network.
Lex Fridman
By the way, that’s just, like, to state the obvious. This is before YouTube. This is before-
By the way, that’s just, like, to state the obvious. This is before YouTube. This is before-
Jean-Baptiste Kempf
Ten years before YouTube. You have a Pentium 60 or 75, right? You, the main machine was 486DX at 33 megahertz, right?
Ten years before YouTube. You have a Pentium 60 or 75, right? You, the main machine was 486DX at 33 megahertz, right?
Kieran Kunhya
Bear in mind, television was the main form of video at the time. You could get new channels. In the ’90s, having even one new channel when you grew up with four channels, having a fifth or a sixth was a big deal, and so having this satellite service with, you know, dozens, even hundreds of channels was so groundbreaking.
Bear in mind, television was the main form of video at the time. You could get new channels. In the ’90s, having even one new channel when you grew up with four channels, having a fifth or a sixth was a big deal, and so having this satellite service with, you know, dozens, even hundreds of channels was so groundbreaking.
Jean-Baptiste Kempf
Especially because this is university where you had a ton of different nationalities, right? So there was a ton of people who wanted… So in the end, they had, like, several dishes on different types of satellites, right? Because, for example, a lot of people were coming from the Maghreb or the Middle East and they went to different types of satellites. Anyway, the solution worked great, and they started the VideoLAN project. The VideoLAN project has several, and some are completely crazy solutions, like one how to create multicast on a unicast network, but let’s not come to that. It’s too complex. But the VideoLAN Client part is what became VLC.
Especially because this is university where you had a ton of different nationalities, right? So there was a ton of people who wanted… So in the end, they had, like, several dishes on different types of satellites, right? Because, for example, a lot of people were coming from the Maghreb or the Middle East and they went to different types of satellites. Anyway, the solution worked great, and they started the VideoLAN project. The VideoLAN project has several, and some are completely crazy solutions, like one how to create multicast on a unicast network, but let’s not come to that. It’s too complex. But the VideoLAN Client part is what became VLC.
Jean-Baptiste Kempf
Actually, they basically strong-armed the university to force it to open source, because the university did not understand that. And in 2001, it’s still early. But basically, yes, the university agreed early 2001 to make it open source. I joined the project in 2003 because that’s when I joined the university. So the first thing is I’m not the one who created VLC, because actually no one did, right?
Actually, they basically strong-armed the university to force it to open source, because the university did not understand that. And in 2001, it’s still early. But basically, yes, the university agreed early 2001 to make it open source. I joined the project in 2003 because that’s when I joined the university. So the first thing is I’m not the one who created VLC, because actually no one did, right?
Lex Fridman
Just kind of naturally emerged from the VideoLAN project. And we should mention that, like, again, you said it just, but to make it clear, VideoLAN as what it became at the time was a set of technologies around video, and the VLC, what you called the client, that’s the thing that most normies, uh-
Just kind of naturally emerged from the VideoLAN project. And we should mention that, like, again, you said it just, but to make it clear, VideoLAN as what it became at the time was a set of technologies around video, and the VLC, what you called the client, that’s the thing that most normies, uh-
Jean-Baptiste Kempf
That is correct, and-
That is correct, and-
Lex Fridman
… think of, like, as the thing, which is, like, the thing that pops up when you click on a video and you play it.
… think of, like, as the thing, which is, like, the thing that pops up when you click on a video and you play it.
Jean-Baptiste Kempf
So I arrive in 2003, and then I will create the open source nonprofit organization called VideoLAN, and I took everything out of the university to create it as a nonprofit project and something sustainable. It’s, yes, it’s true that I spent more time than anyone on VLC and VideoLAN. That is sure. But it’s a continuity of a previous project, VideoLAN, the student project, which is a continuity of the Network 2000 project, which is a continuity of that and that.
So I arrive in 2003, and then I will create the open source nonprofit organization called VideoLAN, and I took everything out of the university to create it as a nonprofit project and something sustainable. It’s, yes, it’s true that I spent more time than anyone on VLC and VideoLAN. That is sure. But it’s a continuity of a previous project, VideoLAN, the student project, which is a continuity of the Network 2000 project, which is a continuity of that and that.
Lex Fridman
I’m sure there’s moments along the way there you were thinking of, like, what is the future of this from an open source perspective? ‘Cause as, as the internet is blowing up, and there is companies… I mean, for people who don’t remember, like, there’s companies making huge amounts of money.
I’m sure there’s moments along the way there you were thinking of, like, what is the future of this from an open source perspective? ‘Cause as, as the internet is blowing up, and there is companies… I mean, for people who don’t remember, like, there’s companies making huge amounts of money.
Jean-Baptiste Kempf
And I can tell you that in 2005, the project should have died and I made it to continue the project. At some point, we were only two active developers. And I thought it was great technology and was useful, and it will be useful, and I made that my life and my time. And I made that grow from a few hundreds of thousands of users, millions of users to what we have now, which is probably billions of versions of VLC around the world and used everywhere. So that’s a bit the story of VLC. There is a ton of very funny stories around that. Many people from around the world working on it, like you said, in Syria or the middle of nowhere in India. But along the way, I got several offers which were either to bundle toolbars, right? You remember those horrible toolbars-
And I can tell you that in 2005, the project should have died and I made it to continue the project. At some point, we were only two active developers. And I thought it was great technology and was useful, and it will be useful, and I made that my life and my time. And I made that grow from a few hundreds of thousands of users, millions of users to what we have now, which is probably billions of versions of VLC around the world and used everywhere. So that’s a bit the story of VLC. There is a ton of very funny stories around that. Many people from around the world working on it, like you said, in Syria or the middle of nowhere in India. But along the way, I got several offers which were either to bundle toolbars, right? You remember those horrible toolbars-
Jean-Baptiste Kempf
… which were basically spyware, or changing your web browser or your search engine or even, like, advertisement inside VLC. And I didn’t like that, right? I am– and people don’t understand that. It’s not– I’m not against money, right? I’m very happy to make money. I created several startups and one I hope that is going to work very well. It’s the fact that I believe that you need to win money ethically. There is a right way of doing that, and doing sneaky advertisement or stealing data is not the correct way, right? For example, if Netflix arrived at some point and said, “Well, we want to put Netflix inside VLC,” probably the story would have been different, right? But they didn’t. The only people who came to us were shady ads companies.
… which were basically spyware, or changing your web browser or your search engine or even, like, advertisement inside VLC. And I didn’t like that, right? I am– and people don’t understand that. It’s not– I’m not against money, right? I’m very happy to make money. I created several startups and one I hope that is going to work very well. It’s the fact that I believe that you need to win money ethically. There is a right way of doing that, and doing sneaky advertisement or stealing data is not the correct way, right? For example, if Netflix arrived at some point and said, “Well, we want to put Netflix inside VLC,” probably the story would have been different, right? But they didn’t. The only people who came to us were shady ads companies.
Jean-Baptiste Kempf
And if I do that, right, I would have a ton of money, right? And then three years later, the project is gone, right? Someone forks it and something else happens.
And if I do that, right, I would have a ton of money, right? And then three years later, the project is gone, right? Someone forks it and something else happens.
Lex Fridman
So it’s not even necessarily ads or any of that, it’s the shadiness of the- … dishonesty of the– So you had a good radar, you had a good threshold of like, “No, this compromises the spirit of what this is supposed to represent.”
So it’s not even necessarily ads or any of that, it’s the shadiness of the- … dishonesty of the– So you had a good radar, you had a good threshold of like, “No, this compromises the spirit of what this is supposed to represent.”
Jean-Baptiste Kempf
But also it’s for me, right? I’m like very selfishly, I need to go to bed at night and be happy about what I’ve done, right? Maybe it’s my upbringing, maybe it’s my parents’ fault or whatever, right? But I believe there is right and wrong, right? And this was the right decision at the time. It still is. I want to be proud of what I’ve been doing. And like, if I had sold out, I would have betrayed so many other people who work here.
But also it’s for me, right? I’m like very selfishly, I need to go to bed at night and be happy about what I’ve done, right? Maybe it’s my upbringing, maybe it’s my parents’ fault or whatever, right? But I believe there is right and wrong, right? And this was the right decision at the time. It still is. I want to be proud of what I’ve been doing. And like, if I had sold out, I would have betrayed so many other people who work here.
Lex Fridman
Yeah, well, I should say me and most of the internet thank you for that decision. It’s inspiring for others I think that are pushing the open source movement forward, that it’s okay to do these kinds of huge sacrifices if you believe it’s right. And I think in that case it was right and it was the reason that VLC became as successful as it was, ’cause it’s an embodiment, it’s a symbol of like, you know, freedom and what the open source community can create.
Yeah, well, I should say me and most of the internet thank you for that decision. It’s inspiring for others I think that are pushing the open source movement forward, that it’s okay to do these kinds of huge sacrifices if you believe it’s right. And I think in that case it was right and it was the reason that VLC became as successful as it was, ’cause it’s an embodiment, it’s a symbol of like, you know, freedom and what the open source community can create.
Jean-Baptiste Kempf
Yeah, and be a service for so many people around the world, and this is important.
Yeah, and be a service for so many people around the world, and this is important.
Kieran Kunhya
We should emphasize in the 2000s it was really normal to download a program and it secretly installs some spyware. It was, it was buried in very faint text or in the license text box that nobody reads at the bottom- … “Oh, I will be installing this toolbar- … and changing all these things,” and it was very common to have to, you know, you install a program to do something at the time of any sort.
We should emphasize in the 2000s it was really normal to download a program and it secretly installs some spyware. It was, it was buried in very faint text or in the license text box that nobody reads at the bottom- … “Oh, I will be installing this toolbar- … and changing all these things,” and it was very common to have to, you know, you install a program to do something at the time of any sort.
Lex Fridman
To put yourself in the mind of a developer at that time, I think it’s very easy—to everybody listening to this, it’s very easy at that time to convince yourself to take a few thousand dollars- … a few thousand dollars to do it. To say no to much more money- … takes guts and takes vision.
To put yourself in the mind of a developer at that time, I think it’s very easy—to everybody listening to this, it’s very easy at that time to convince yourself to take a few thousand dollars- … a few thousand dollars to do it. To say no to much more money- … takes guts and takes vision.
Jean-Baptiste Kempf
The last offer I had was obscene, and they say, “Yeah, but imagine with all that money you could build something new, open source,” right? It was like the mind trick was… it was difficult. But for me it was just like, “No, this doesn’t work like that or this is not the right thing, so I don’t do it.” And again, right, it’s not that I don’t like money or whatever. It’s just like it wasn’t right.
The last offer I had was obscene, and they say, “Yeah, but imagine with all that money you could build something new, open source,” right? It was like the mind trick was… it was difficult. But for me it was just like, “No, this doesn’t work like that or this is not the right thing, so I don’t do it.” And again, right, it’s not that I don’t like money or whatever. It’s just like it wasn’t right.
FFmpeg & Google drama
Lex Fridman
Well, once again, thank you from me and from the rest of the internet. Let me talk a little bit more about the open source movement, about the fact that, as you say over and over and over and over, FFmpeg is and many open source projects are built by volunteers. So there’s a bit of drama recently, Kieran, on the interwebs, on Twitter. You have a spicy style on Twitter that I think articulates and celebrates all the incredible developers and development and the code, especially assembly, that’s involved in building some of these codecs and building some of this incredible technology. But that brings us to the… a bit of a debacle that happened. Tell me the full saga of what happened with the Google security engineers.
Well, once again, thank you from me and from the rest of the internet. Let me talk a little bit more about the open source movement, about the fact that, as you say over and over and over and over, FFmpeg is and many open source projects are built by volunteers. So there’s a bit of drama recently, Kieran, on the interwebs, on Twitter. You have a spicy style on Twitter that I think articulates and celebrates all the incredible developers and development and the code, especially assembly, that’s involved in building some of these codecs and building some of this incredible technology. But that brings us to the… a bit of a debacle that happened. Tell me the full saga of what happened with the Google security engineers.
Kieran Kunhya
Just to be clear, Google are one of the biggest supporters of open source out there. They have been for a long time. It’s just I think some things kind of went a bit overboard this time. So FFmpeg itself—and this is not like a secret, it’s on the homepage, you know—it processes untrusted data. There can be security issues when you parse untrusted data. That’s very normal. But recently what changed was Google started using AI to create security reports on an open source project, FFmpeg. Volunteers had to deal with that. They did, they provided very limited funding, and they even went to the media first announcing how good their AI was before the issues could be fixed.
Just to be clear, Google are one of the biggest supporters of open source out there. They have been for a long time. It’s just I think some things kind of went a bit overboard this time. So FFmpeg itself—and this is not like a secret, it’s on the homepage, you know—it processes untrusted data. There can be security issues when you parse untrusted data. That’s very normal. But recently what changed was Google started using AI to create security reports on an open source project, FFmpeg. Volunteers had to deal with that. They did, they provided very limited funding, and they even went to the media first announcing how good their AI was before the issues could be fixed.
Lex Fridman
And this is in the public forum.
And this is in the public forum.
Kieran Kunhya
Yeah, this is all public.
Yeah, this is all public.
Lex Fridman
So reporting an issue, using AI to find an issue in the code which is a security vulnerability, and then reporting that publicly before you’re able to fix it.
So reporting an issue, using AI to find an issue in the code which is a security vulnerability, and then reporting that publicly before you’re able to fix it.
Kieran Kunhya
Yeah. It’s announcing how good their AI is, that they provided a standard 90-day industry deadline without really understanding the nature of volunteer-driven development. In addition, this vulnerability was on an obscure 1990s game codec. The way—and let’s look at it from their standpoint to begin with. Let, let’s you know-
Yeah. It’s announcing how good their AI is, that they provided a standard 90-day industry deadline without really understanding the nature of volunteer-driven development. In addition, this vulnerability was on an obscure 1990s game codec. The way—and let’s look at it from their standpoint to begin with. Let, let’s you know-
Lex Fridman
Yeah. Can you steer me in their case?
Yeah. Can you steer me in their case?
Kieran Kunhya
Yeah, sure. They have substantial resources working on the security of open source projects that, you know, are ubiquitous, and they’ve used a lot of compute to do that and very expensive and very capable security researchers to do that. And that’s their viewpoint is they are contributing by doing that. But I think that’s where opinions differ. It opened up a lot of interesting fissures, I would say. It does seem that there’s a portion of the security community that look at themselves a bit like building architects that never have to go to site. You know, going to site is something that is a little bit beneath them, the actual day-to-day construction. They’re there to do their security things and it’s someone else’s problem.
Yeah, sure. They have substantial resources working on the security of open source projects that, you know, are ubiquitous, and they’ve used a lot of compute to do that and very expensive and very capable security researchers to do that. And that’s their viewpoint is they are contributing by doing that. But I think that’s where opinions differ. It opened up a lot of interesting fissures, I would say. It does seem that there’s a portion of the security community that look at themselves a bit like building architects that never have to go to site. You know, going to site is something that is a little bit beneath them, the actual day-to-day construction. They’re there to do their security things and it’s someone else’s problem.
Kieran Kunhya
The security industry also kind of has a very aggressive tone towards things. The language they use is extremely aggressive. They use very strong language like, “You will get popped.” So and to Joe Public, “get popped,” you know, means something quite bad. For them it means to get hacked. The way I would look at it personally is a little bit like the padlock on your home. Not everyone… The padlock on your home or, you know, the lock on your home is there to protect against the capabilities of what it’s there to protect. It’s not there to protect nuclear secrets. It’s not there to protect Fort Knox. And it could be looked at that they’re using AI at a level of scale to go and pick those locks and then say, “Hey, your lock’s not secure.”
The security industry also kind of has a very aggressive tone towards things. The language they use is extremely aggressive. They use very strong language like, “You will get popped.” So and to Joe Public, “get popped,” you know, means something quite bad. For them it means to get hacked. The way I would look at it personally is a little bit like the padlock on your home. Not everyone… The padlock on your home or, you know, the lock on your home is there to protect against the capabilities of what it’s there to protect. It’s not there to protect nuclear secrets. It’s not there to protect Fort Knox. And it could be looked at that they’re using AI at a level of scale to go and pick those locks and then say, “Hey, your lock’s not secure.”
Kieran Kunhya
You need to deal with this.” Whereas actually they’re the ones with resources to be able to fix this. But that seems to not be something either they’ll contribute to in terms of patches or in terms of financially. And the scale of AI is kind of the issue. The bug reports are very wordy. They’re very, very—it’s almost a denial of service by AI-generated bug reports on very niche codecs. And the other issue the security community has is everything is marked high priority. You’re going to, you know, “This is the most important thing in the world, and you need to deal with this. High, high, high, vulnerable, scary, scary, scary,” on a game codec used on one disk in 1993.
You need to deal with this.” Whereas actually they’re the ones with resources to be able to fix this. But that seems to not be something either they’ll contribute to in terms of patches or in terms of financially. And the scale of AI is kind of the issue. The bug reports are very wordy. They’re very, very—it’s almost a denial of service by AI-generated bug reports on very niche codecs. And the other issue the security community has is everything is marked high priority. You’re going to, you know, “This is the most important thing in the world, and you need to deal with this. High, high, high, vulnerable, scary, scary, scary,” on a game codec used on one disk in 1993.
Kieran Kunhya
And that’s where the dichotomy lies. Going around telling everyone that their padlock’s not safe—well, that’s a hobby project of somebody. The safety of that codec is consummate to what that person thinks. It’s their hobby. It’s good that they’re security analyzing it, but it doesn’t need a big scary warning, “This is a critical vulnerability.” You may also recently see that there was another quote-unquote vulnerability. It wasn’t Google in this case, but a filter could overflow and have an integer overflow, and one of your pixels could be the wrong color. And this was marked high, 7.5 severity in red.
And that’s where the dichotomy lies. Going around telling everyone that their padlock’s not safe—well, that’s a hobby project of somebody. The safety of that codec is consummate to what that person thinks. It’s their hobby. It’s good that they’re security analyzing it, but it doesn’t need a big scary warning, “This is a critical vulnerability.” You may also recently see that there was another quote-unquote vulnerability. It wasn’t Google in this case, but a filter could overflow and have an integer overflow, and one of your pixels could be the wrong color. And this was marked high, 7.5 severity in red.
Kieran Kunhya
And at some point, the security industry needs to realize you can’t keep crying wolf like this because this just leads to people, you know, the equivalent thereof of putting password stickers on their PC. You know, you can’t just keep crying wolf every day. And I appreciate, you know, that’s their modus operandi is to create as much scare and fear. But from the Google standpoint, at the end of the day, they need to contribute either financially or with patches. Google uses FFmpeg at a scale probably you or I couldn’t even contemplate, millions of CPU cores. And yes, they contribute in areas mostly regarding their own products, so VP9, AV1. But in a wider sense, there’s a disproportionate level of contribution. Yes, they fund students. Yes, they fund Summer of Code.
And at some point, the security industry needs to realize you can’t keep crying wolf like this because this just leads to people, you know, the equivalent thereof of putting password stickers on their PC. You know, you can’t just keep crying wolf every day. And I appreciate, you know, that’s their modus operandi is to create as much scare and fear. But from the Google standpoint, at the end of the day, they need to contribute either financially or with patches. Google uses FFmpeg at a scale probably you or I couldn’t even contemplate, millions of CPU cores. And yes, they contribute in areas mostly regarding their own products, so VP9, AV1. But in a wider sense, there’s a disproportionate level of contribution. Yes, they fund students. Yes, they fund Summer of Code.
Kieran Kunhya
And I think… so Alex Strange is a former FFmpeg developer, I think posting in a personal capacity.
And I think… so Alex Strange is a former FFmpeg developer, I think posting in a personal capacity.
Lex Fridman
So he posted about security engineers on Hacker News. His post reads, “The problem with security reports in general is security people are rampant self-promoters”—in parentheses, Linus once called them something worse. “Imagine you’re a humble volunteer open source developer. If a security researcher finds a bug in your code, they’re going to make up a cute name for it, start a website with a logo. Google is going to give them a million-dollar bounty. They’re going to go to DEF CON and get a prize, and I assume go to some kind of secret security people orgy where everyone is dressed like they’re in The Matrix. Nobody is going to do any of this for you when you fix it.” Uh, basically commenting on the sort of the incentives for the different people involved and misaligned.
So he posted about security engineers on Hacker News. His post reads, “The problem with security reports in general is security people are rampant self-promoters”—in parentheses, Linus once called them something worse. “Imagine you’re a humble volunteer open source developer. If a security researcher finds a bug in your code, they’re going to make up a cute name for it, start a website with a logo. Google is going to give them a million-dollar bounty. They’re going to go to DEF CON and get a prize, and I assume go to some kind of secret security people orgy where everyone is dressed like they’re in The Matrix. Nobody is going to do any of this for you when you fix it.” Uh, basically commenting on the sort of the incentives for the different people involved and misaligned.
Jean-Baptiste Kempf
The problem here is the disproportion of means on discovery compared to patching it, right? And this is the biggest issue, right? And after that debacle, Google did some changes.
The problem here is the disproportion of means on discovery compared to patching it, right? And this is the biggest issue, right? And after that debacle, Google did some changes.
Kieran Kunhya
They are now starting to send patches, which is-
They are now starting to send patches, which is-
Jean-Baptiste Kempf
And they also now have reward tools for fixing issues. So it, it has changed a bit because of that debacle. So it’s good, right? But we’ve seen—and we talk about Google—but we have seen like some other large companies saying, “Oh, you need to fix this bug because it’s critical in our product.”
And they also now have reward tools for fixing issues. So it, it has changed a bit because of that debacle. So it’s good, right? But we’ve seen—and we talk about Google—but we have seen like some other large companies saying, “Oh, you need to fix this bug because it’s critical in our product.”
Lex Fridman
Can you explain the XZ fiasco? The FFmpeg tweet reads, “The XZ fiasco has shown how a dependence on unpaid volunteers can cause major problems. Trillion-dollar corporations expect free and urgent support from volunteers. Microsoft Teams posted on a bug tracker full of volunteers that their issue is high priority. After politely requesting a support contract from Microsoft for long-term maintenance, they offered a one-time payment of a few thousand dollars instead. This is unacceptable. We didn’t make it up. This is what Microsoft Teams actually did.” And then you give the image and the details and all that kind of stuff, showing that these trillion-dollar companies are not giving much money, not giving much support.
Can you explain the XZ fiasco? The FFmpeg tweet reads, “The XZ fiasco has shown how a dependence on unpaid volunteers can cause major problems. Trillion-dollar corporations expect free and urgent support from volunteers. Microsoft Teams posted on a bug tracker full of volunteers that their issue is high priority. After politely requesting a support contract from Microsoft for long-term maintenance, they offered a one-time payment of a few thousand dollars instead. This is unacceptable. We didn’t make it up. This is what Microsoft Teams actually did.” And then you give the image and the details and all that kind of stuff, showing that these trillion-dollar companies are not giving much money, not giving much support.
Kieran Kunhya
They think an open source project is a traditional vendor that they have an SLA. They think a public bug tracker is actually, you know, a third-party vendor’s Jira where you can do all of these things. It’s not. It is there to report bugs. I think the thing that made this particularly heinous was the name-dropping of Microsoft, the name-dropping that this is a visible product. If this was just a general bug report, I think that would have made it a lot better.
They think an open source project is a traditional vendor that they have an SLA. They think a public bug tracker is actually, you know, a third-party vendor’s Jira where you can do all of these things. It’s not. It is there to report bugs. I think the thing that made this particularly heinous was the name-dropping of Microsoft, the name-dropping that this is a visible product. If this was just a general bug report, I think that would have made it a lot better.
Lex Fridman
Yeah, so they literally said, like, “This is a big deal because a lot of people are using it in Microsoft.” I wonder what happens psychologically. So I think what happens in these companies, maybe you can correct me, is they… You’re right. They just think of FFmpeg as like a vendor that Microsoft surely is paying a huge amount of money to. They kind of assume that in their interaction, and nobody anywhere on the stack is going like, “Wait a minute. Shouldn’t we be giving like millions of dollars to FFmpeg?”
Yeah, so they literally said, like, “This is a big deal because a lot of people are using it in Microsoft.” I wonder what happens psychologically. So I think what happens in these companies, maybe you can correct me, is they… You’re right. They just think of FFmpeg as like a vendor that Microsoft surely is paying a huge amount of money to. They kind of assume that in their interaction, and nobody anywhere on the stack is going like, “Wait a minute. Shouldn’t we be giving like millions of dollars to FFmpeg?”
Jean-Baptiste Kempf
And this is a very big problem in large—like we’re talking about some companies, but it’s the same everywhere, right? A lot of those companies. Like the… when we talk to that person, right, he was just like a manager on one project in Microsoft Teams, right? He had, like, never really discussed with the open source community. He had no idea, right? It was like… but the problem is that usually there is what we call OSPOs, right? Open Source Program Offices in those type of companies, and they are the ones who are supposed to discuss with open source vendors.
And this is a very big problem in large—like we’re talking about some companies, but it’s the same everywhere, right? A lot of those companies. Like the… when we talk to that person, right, he was just like a manager on one project in Microsoft Teams, right? He had, like, never really discussed with the open source community. He had no idea, right? It was like… but the problem is that usually there is what we call OSPOs, right? Open Source Program Offices in those type of companies, and they are the ones who are supposed to discuss with open source vendors.
Jean-Baptiste Kempf
Or open source communities. But like they often don’t explain that correctly internally, right? And here it’s just like we are not your supplier. If you want me to be your supplier, I’m very happy, right? I will send you a contract and SLAs. Like I created five companies who are doing that around open source projects, so that’s okay.
Or open source communities. But like they often don’t explain that correctly internally, right? And here it’s just like we are not your supplier. If you want me to be your supplier, I’m very happy, right? I will send you a contract and SLAs. Like I created five companies who are doing that around open source projects, so that’s okay.
Lex Fridman
We should say that some, some of the spicy tweets that Kieran, you’re behind, and some of the debacle produced results.
We should say that some, some of the spicy tweets that Kieran, you’re behind, and some of the debacle produced results.
Jean-Baptiste Kempf
Yes.
Yes.
Lex Fridman
Positive results.
Positive results.
Kieran Kunhya
Donations have increased substantially. They’re still not enough to cover even a single full-time developer, but on both a, you know, awareness level and a technical level, there’s substantially more technical awareness and sort of awareness of the importance of FFmpeg as a result of X and what’s happened. I can say, you know, it solved its purpose. People realize the level of importance FFmpeg has.
Donations have increased substantially. They’re still not enough to cover even a single full-time developer, but on both a, you know, awareness level and a technical level, there’s substantially more technical awareness and sort of awareness of the importance of FFmpeg as a result of X and what’s happened. I can say, you know, it solved its purpose. People realize the level of importance FFmpeg has.
Jean-Baptiste Kempf
And on VideoLAN it’s the same, right? Like for example, a, a very simple example. For more than a year, we couldn’t update VLC on Android because of a bug on the Play Store, on Android Play Store, right? The only way we got someone to answer was to put a very spicy, as you say, tweet saying that we are going to stop distributing VLC for Android, right? And we have around 100 million people using that. And now then someone from Android actually came and discussed to us, right? We had the same issue with, with Microsoft or, or like saying that we were going to stop distributing VLC on the Windows Store.
And on VideoLAN it’s the same, right? Like for example, a, a very simple example. For more than a year, we couldn’t update VLC on Android because of a bug on the Play Store, on Android Play Store, right? The only way we got someone to answer was to put a very spicy, as you say, tweet saying that we are going to stop distributing VLC for Android, right? And we have around 100 million people using that. And now then someone from Android actually came and discussed to us, right? We had the same issue with, with Microsoft or, or like saying that we were going to stop distributing VLC on the Windows Store.
Jean-Baptiste Kempf
And unfortunately, we are so small that the only very strong power we have to solve those issues is blaming on social networks because it snowballs and now they listen to us. But so as large companies often have difficulty talking to us. Like for example, VLC, right, is probably one of the top 10 software used on Windows. I am not part of Microsoft ISV programs, right? I don’t have a point of contact at Microsoft, right? While I’m sure any other software, Adobe, Spotify, has a point of contact. I don’t have that, right? So raising awareness works. It’s sometimes very spicy, lot of drama. Well, X and Twitter are okay for that, but it’s efficient.
And unfortunately, we are so small that the only very strong power we have to solve those issues is blaming on social networks because it snowballs and now they listen to us. But so as large companies often have difficulty talking to us. Like for example, VLC, right, is probably one of the top 10 software used on Windows. I am not part of Microsoft ISV programs, right? I don’t have a point of contact at Microsoft, right? While I’m sure any other software, Adobe, Spotify, has a point of contact. I don’t have that, right? So raising awareness works. It’s sometimes very spicy, lot of drama. Well, X and Twitter are okay for that, but it’s efficient.
Lex Fridman
So everybody listening to this should go follow FFmpeg on Twitter, on X, follow VideoLAN on Twitter, on X. Go donate. Donate-
So everybody listening to this should go follow FFmpeg on Twitter, on X, follow VideoLAN on Twitter, on X. Go donate. Donate-
Kieran Kunhya
Thank you
Thank you
Lex Fridman
… to FFmpeg.
… to FFmpeg.
Kieran Kunhya
And thank you, Lex. Over the years, several years you’ve been a supporter of, you know, FFmpeg and VideoLAN on X. You know, giving us shout-outs, appreciating, you know, what we do.
And thank you, Lex. Over the years, several years you’ve been a supporter of, you know, FFmpeg and VideoLAN on X. You know, giving us shout-outs, appreciating, you know, what we do.
Lex Fridman
FFmpeg for life.
FFmpeg for life.
Jean-Baptiste Kempf
And for example, like Tim Sweeney, Carmack, and a few others, like very high-level people have raised also the awareness on, on our X accounts, and that helped a lot also.
And for example, like Tim Sweeney, Carmack, and a few others, like very high-level people have raised also the awareness on, on our X accounts, and that helped a lot also.
Kieran Kunhya
Karpathy as well.
Karpathy as well.
Jean-Baptiste Kempf
Karpathy, yes.
Karpathy, yes.
Kieran Kunhya
Karpathy as well, yeah.
Karpathy as well, yeah.
Lex Fridman
Yeah. I mean, also, you know, outside of the fact that so many people use it, it’s so impactful on the world, it’s also a great representation of a great open source project. Like the value of assembly and C and making sure that like you take programming seriously for real world systems.
Yeah. I mean, also, you know, outside of the fact that so many people use it, it’s so impactful on the world, it’s also a great representation of a great open source project. Like the value of assembly and C and making sure that like you take programming seriously for real world systems.
Kieran Kunhya
It’s not just that. We’ll talk about assembly later I’m sure, ’cause that’s its whole topic in itself, but it’s also celebrating people like Andreas Rheinhardt who do maintenance. It is, I believe unpaid, as a volunteer. He’s doing massive refactorings. Andreas Rheinhardt and Anton Khirnov rewriting ffmpeg.c with threading. Celebrating those guys, celebrating the untold labor that’s gone into this that actually doesn’t change anything from the user standpoint. The files are exactly the same, but wow, the airplane has been rebuilt whilst it’s in the air.
It’s not just that. We’ll talk about assembly later I’m sure, ’cause that’s its whole topic in itself, but it’s also celebrating people like Andreas Rheinhardt who do maintenance. It is, I believe unpaid, as a volunteer. He’s doing massive refactorings. Andreas Rheinhardt and Anton Khirnov rewriting ffmpeg.c with threading. Celebrating those guys, celebrating the untold labor that’s gone into this that actually doesn’t change anything from the user standpoint. The files are exactly the same, but wow, the airplane has been rebuilt whilst it’s in the air.
Lex Fridman
Christian Garcia said, “As a teenager running this account,” referring to the FFmpeg a- … account, and you responded, “Teenagers have written more assembly in FFmpeg than Google engineers.” But also just pointing out that there’s a lot of incredible contributors who are teenagers.
Christian Garcia said, “As a teenager running this account,” referring to the FFmpeg a- … account, and you responded, “Teenagers have written more assembly in FFmpeg than Google engineers.” But also just pointing out that there’s a lot of incredible contributors who are teenagers.
Kieran Kunhya
Like JB said, we don’t care who you are, where you’re from, what you do. Teenagers have written thousands of lines of assembly over the years. Give a shout-out back in the days to Daniel Kang. So also highlighting the work of people like Ruikai Peng. This is a 16-year-old, some of his first contributions to FFmpeg, actually doing and putting some of these quote-unquote security researchers to shame by actually finding issues and fixing them and being 16. There’s no barriers. There’s no barriers to you have to study at college under this person and understand these. You can learn C, and let’s be honest, it’s from the K&R book. Learn C. You can learn assembly. We’ll talk about that maybe a bit later. You can contribute to world-class technologies.
Like JB said, we don’t care who you are, where you’re from, what you do. Teenagers have written thousands of lines of assembly over the years. Give a shout-out back in the days to Daniel Kang. So also highlighting the work of people like Ruikai Peng. This is a 16-year-old, some of his first contributions to FFmpeg, actually doing and putting some of these quote-unquote security researchers to shame by actually finding issues and fixing them and being 16. There’s no barriers. There’s no barriers to you have to study at college under this person and understand these. You can learn C, and let’s be honest, it’s from the K&R book. Learn C. You can learn assembly. We’ll talk about that maybe a bit later. You can contribute to world-class technologies.
Jean-Baptiste Kempf
In VLC one of the oldest contributors called Felix, he’s the one doing everything on Mac and iOS. He’s starting working on VLC. He was 16. We had a guy called Edward Wang, who used to be a Google Summer of Code student who stayed for three years around VideoLAN. He was 14, right? And part of Google Summer of Code and Google Code-in, which were programs where basically we have students or high schoolers, we wrote a ton of assembly for x264 and for VLC and for FFmpeg, right? So everyone can contribute.
In VLC one of the oldest contributors called Felix, he’s the one doing everything on Mac and iOS. He’s starting working on VLC. He was 16. We had a guy called Edward Wang, who used to be a Google Summer of Code student who stayed for three years around VideoLAN. He was 14, right? And part of Google Summer of Code and Google Code-in, which were programs where basically we have students or high schoolers, we wrote a ton of assembly for x264 and for VLC and for FFmpeg, right? So everyone can contribute.
Kieran Kunhya
And he also did a good job because he didn’t play the alarmist CVE heist, create a CVE (which is like a public exposure of security), and do these big scary red 7.5 high priority. He just fixed an issue in Git after three days and just fixed it. He didn’t need to go and play a big security drama about it. And I think I posted, you know, “the kids are all right.” Whereas there is a portion—I’m not saying all security people do this, but there is a portion of the security community, as Alex said, that likes to hype themselves up by creating drama. They would have happily raised, “This is a high priority CVE 8.0” or whatever on an issue that actually was in Git. It wasn’t even in a release, it was in development, and three days later was fixed.
And he also did a good job because he didn’t play the alarmist CVE heist, create a CVE (which is like a public exposure of security), and do these big scary red 7.5 high priority. He just fixed an issue in Git after three days and just fixed it. He didn’t need to go and play a big security drama about it. And I think I posted, you know, “the kids are all right.” Whereas there is a portion—I’m not saying all security people do this, but there is a portion of the security community, as Alex said, that likes to hype themselves up by creating drama. They would have happily raised, “This is a high priority CVE 8.0” or whatever on an issue that actually was in Git. It wasn’t even in a release, it was in development, and three days later was fixed.
Jean-Baptiste Kempf
Well, I just want to put a little bit of love out there, even to the bigger community. Much love and respect to Google engineers. Like you said, they’re some of the best software engineers in the world, and they do contribute a lot— … even on the security front. And also, you know, I’m a big fan of Theo. Much love to Theo. He was part of this debacle and drama a little bit. I think when you just zoom out on the grand arc of human history, the drama contributed positively to everybody involved. Donations went up. It brought more attention to the topic, allowed everybody to bicker in a way that ultimately got them to figure out what FFmpeg is all about.
Well, I just want to put a little bit of love out there, even to the bigger community. Much love and respect to Google engineers. Like you said, they’re some of the best software engineers in the world, and they do contribute a lot— … even on the security front. And also, you know, I’m a big fan of Theo. Much love to Theo. He was part of this debacle and drama a little bit. I think when you just zoom out on the grand arc of human history, the drama contributed positively to everybody involved. Donations went up. It brought more attention to the topic, allowed everybody to bicker in a way that ultimately got them to figure out what FFmpeg is all about.
Kieran Kunhya
So the way we looked at this is like it’s a rap battle at the end of the day, you know? No, but it is, it is.
So the way we looked at this is like it’s a rap battle at the end of the day, you know? No, but it is, it is.
Jean-Baptiste Kempf
It is. It is.
It is. It is.
Kieran Kunhya
We say stuff, we say stuff- … but we can, we can leave it on. X is a perfect place for, you know, international rap battle. You say stuff. I say stuff about your mama, but it doesn’t mean, you know, I have an actual personal issue with her. And that’s what it looks like. The Theo situation, you know, JB can maybe expand, went a little bit too far and there was a little… But, you know, it’s just a bit of fun. It’s just a bit of rap battle. It’s a bit—it’s WWE. You know, everyone’s having a bit of fun on X. It doesn’t need to be taken seriously. You know, the teenagers thing, you know, that… So that guy was a Google employee saying, “Hey, you know, there are other ways to run an open source business.” You know, and there’s like, oh, man, just have a bit of fun, you know?
We say stuff, we say stuff- … but we can, we can leave it on. X is a perfect place for, you know, international rap battle. You say stuff. I say stuff about your mama, but it doesn’t mean, you know, I have an actual personal issue with her. And that’s what it looks like. The Theo situation, you know, JB can maybe expand, went a little bit too far and there was a little… But, you know, it’s just a bit of fun. It’s just a bit of rap battle. It’s a bit—it’s WWE. You know, everyone’s having a bit of fun on X. It doesn’t need to be taken seriously. You know, the teenagers thing, you know, that… So that guy was a Google employee saying, “Hey, you know, there are other ways to run an open source business.” You know, and there’s like, oh, man, just have a bit of fun, you know?
Kieran Kunhya
That’s what the point of this account is. And furthermore, if you can teach people about the ways of open source projects, assembly, et cetera, by doing that, I think there’s a lot to be offered here. It’s not dunking on people for dunking’s sake. It’s showing actually the story that I think X learnt is these are not big corporate open source projects. This is not Kubernetes where there’s, you know, hundreds, maybe thousands of people- … paid to develop this stuff. These are just people in their basements in their spare time, and if you can address that topic in a fun and entertaining way- … I think that that’s the good thing and that’s, that’s the value of X and then the reach we have.
That’s what the point of this account is. And furthermore, if you can teach people about the ways of open source projects, assembly, et cetera, by doing that, I think there’s a lot to be offered here. It’s not dunking on people for dunking’s sake. It’s showing actually the story that I think X learnt is these are not big corporate open source projects. This is not Kubernetes where there’s, you know, hundreds, maybe thousands of people- … paid to develop this stuff. These are just people in their basements in their spare time, and if you can address that topic in a fun and entertaining way- … I think that that’s the good thing and that’s, that’s the value of X and then the reach we have.
Jean-Baptiste Kempf
And to be honest, right, like even at Google, Google is one entity, but so many different people, right? And there is a ton of Google engineers we work with all the time, and even like Google from YouTube to Chrome to Chrome Media to the rest of Google, those are very different type of entities. But what we do is efficient. And, for example for Theo, right? It went a bit too far. I had him… Like I calmed everyone down. I had him on the phone. We say, “Okay, like this goes too far,” and so on. But in the end yeah, it’s a rap battle, but it’s positive for the project. It—like the awareness we have on open source and, and I mean true open source from communities right now—is increased dramatically in the last two years, and this is useful.
And to be honest, right, like even at Google, Google is one entity, but so many different people, right? And there is a ton of Google engineers we work with all the time, and even like Google from YouTube to Chrome to Chrome Media to the rest of Google, those are very different type of entities. But what we do is efficient. And, for example for Theo, right? It went a bit too far. I had him… Like I calmed everyone down. I had him on the phone. We say, “Okay, like this goes too far,” and so on. But in the end yeah, it’s a rap battle, but it’s positive for the project. It—like the awareness we have on open source and, and I mean true open source from communities right now—is increased dramatically in the last two years, and this is useful.
FFmpeg developers
Kieran Kunhya
What do you think motivates all the incredible contributors that we’ve been talking about? Like, what’s the engine? It’s so interesting to see.
What do you think motivates all the incredible contributors that we’ve been talking about? Like, what’s the engine? It’s so interesting to see.
Jean-Baptiste Kempf
So-
So-
Kieran Kunhya
Like you said, they’re sitting in the basement. What’s the driver? What’s the engine there?
Like you said, they’re sitting in the basement. What’s the driver? What’s the engine there?
Jean-Baptiste Kempf
There are many drivers, but weirdly the main one is that what we do in multimedia plays videos, and video is cool, right? And, for example, we have so many people in the community who arrive because they loved watching anime, right? And this is like the advice when people ask me, “What should I work on in open source? How do I start?” And my answer is always the same: work on something you love.
There are many drivers, but weirdly the main one is that what we do in multimedia plays videos, and video is cool, right? And, for example, we have so many people in the community who arrive because they loved watching anime, right? And this is like the advice when people ask me, “What should I work on in open source? How do I start?” And my answer is always the same: work on something you love.
Jean-Baptiste Kempf
I am working on VLC because I love movies, right? And I love watching the same movies over and over, even if my wife hates me when I do that, right? But because it’s interesting, right? Because it’s a topic that you like, right? The first, that’s the first thing where people come to usually to VLC and FFmpeg. The second thing is that technically we, because we search for excellence, this is the best school ever, right? This is the best school ever of programming. If you’re good in C, in FFmpeg, if you know how to write assembly, I assure you you’re going to be one of the best programmers ever, even if you’re working on writing TypeScript, because this is the most amazing thing to do.
I am working on VLC because I love movies, right? And I love watching the same movies over and over, even if my wife hates me when I do that, right? But because it’s interesting, right? Because it’s a topic that you like, right? The first, that’s the first thing where people come to usually to VLC and FFmpeg. The second thing is that technically we, because we search for excellence, this is the best school ever, right? This is the best school ever of programming. If you’re good in C, in FFmpeg, if you know how to write assembly, I assure you you’re going to be one of the best programmers ever, even if you’re working on writing TypeScript, because this is the most amazing thing to do.
Jean-Baptiste Kempf
And you will, like, have to get reviews by some of the most seasoned programmers ever who are going to look at every part of your code and tell you why it’s not great. It’s like we are the best teachers that you’ve ever had in programming, right?
And you will, like, have to get reviews by some of the most seasoned programmers ever who are going to look at every part of your code and tell you why it’s not great. It’s like we are the best teachers that you’ve ever had in programming, right?
Kieran Kunhya
Andrew Kelley started Zig. He was an FFmpeg developer and started Zig after his FFmpeg school. I mean, it, it’s the place to learn so many aspects of programming in the real world, in a thing used by billions of people. You have nowhere to hide. You have to be open and honest about your flaws and, and how you can learn and be better.
Andrew Kelley started Zig. He was an FFmpeg developer and started Zig after his FFmpeg school. I mean, it, it’s the place to learn so many aspects of programming in the real world, in a thing used by billions of people. You have nowhere to hide. You have to be open and honest about your flaws and, and how you can learn and be better.
Jean-Baptiste Kempf
And what is also interesting in multimedia is that you have 16 milliseconds to display a frame. It’s not like a game engine where you can basically slow down and wait a frame. So it’s, you need to be good, right? There is no choice, else you don’t have your video. And because of how codecs, if you miss a frame, you’re going to destroy the look of the video, right? So you need to be good. You need to be perfect to have the right thing. But also is that it’s not just pure programming in the mathematical sense, right? A lot of people don’t understand, but in order to program correctly on the open source multimedia community, you need to understand how computers work. And when you write assembly, you need to understand about CPU pipelining, right?
And what is also interesting in multimedia is that you have 16 milliseconds to display a frame. It’s not like a game engine where you can basically slow down and wait a frame. So it’s, you need to be good, right? There is no choice, else you don’t have your video. And because of how codecs, if you miss a frame, you’re going to destroy the look of the video, right? So you need to be good. You need to be perfect to have the right thing. But also is that it’s not just pure programming in the mathematical sense, right? A lot of people don’t understand, but in order to program correctly on the open source multimedia community, you need to understand how computers work. And when you write assembly, you need to understand about CPU pipelining, right?
Jean-Baptiste Kempf
You need to understand how SIMD works, how the ALU works, right? You need to understand how I/O works, right? And this is what I think that is missing to a lot of engineers and software engineers today, is understanding what we call computer architecture. And, like, seriously, some of the debates is like, should we use this assembly call or this one? And people say, “Well, no, it’s going to be like three cycles on this type of CPU and this one,” and has massive impact on the output, right?
You need to understand how SIMD works, how the ALU works, right? You need to understand how I/O works, right? And this is what I think that is missing to a lot of engineers and software engineers today, is understanding what we call computer architecture. And, like, seriously, some of the debates is like, should we use this assembly call or this one? And people say, “Well, no, it’s going to be like three cycles on this type of CPU and this one,” and has massive impact on the output, right?
Kieran Kunhya
We should expand. FFmpeg is probably one of the biggest CPU users in the world. There’s proba- it’s probably running- … as we speak easily 100 mil- order of magnitude 100 million, maybe even a billion CPUs as we speak. So every instruction matters. There’s not… The impact, at least in terms of CPU, is massive for everything that we do.
We should expand. FFmpeg is probably one of the biggest CPU users in the world. There’s proba- it’s probably running- … as we speak easily 100 mil- order of magnitude 100 million, maybe even a billion CPUs as we speak. So every instruction matters. There’s not… The impact, at least in terms of CPU, is massive for everything that we do.
Jean-Baptiste Kempf
So first you come because it’s an interesting subject, then you stay because it’s excellent, and in the end you’re very proud of it because it’s on the end of everyone. Like so many people like, “Oh, I’m working for whatever consulting company and I’m doing some portal to download invoices for your PG&E.” Wow, great. Like, so many jobs are like that. You’re not going to tell that to your grandma. But if you go to see your grandma and say, “I do this so that you can play video on your laptop,” they understand. And this is very important, right? Because you’re working on VLC, FFmpeg, H.264. It’s in the end of hundreds of millions of people and you have an impact. And so you can be proud of yourself.
So first you come because it’s an interesting subject, then you stay because it’s excellent, and in the end you’re very proud of it because it’s on the end of everyone. Like so many people like, “Oh, I’m working for whatever consulting company and I’m doing some portal to download invoices for your PG&E.” Wow, great. Like, so many jobs are like that. You’re not going to tell that to your grandma. But if you go to see your grandma and say, “I do this so that you can play video on your laptop,” they understand. And this is very important, right? Because you’re working on VLC, FFmpeg, H.264. It’s in the end of hundreds of millions of people and you have an impact. And so you can be proud of yourself.
Jean-Baptiste Kempf
And so I think that in addition to doing a great resume, all those things are why people contribute.
And so I think that in addition to doing a great resume, all those things are why people contribute.
Kieran Kunhya
Yeah, those are side effects. My favorite quote on this topic is John Collison. He said, “The world is a museum of passion projects.” You know, everything out there is a passion project. And open source multimedia and open source in general, you can just do that so much faster. There’s such a faster network effect, you know?
Yeah, those are side effects. My favorite quote on this topic is John Collison. He said, “The world is a museum of passion projects.” You know, everything out there is a passion project. And open source multimedia and open source in general, you can just do that so much faster. There’s such a faster network effect, you know?
Kieran Kunhya
I can open a cafe and that can be my passion project, but I have to get building codes, I have to build a building, I have to find a location, I have to do all the, you know, all sorts of things. Well, in the software world, that passion project can move quickly, it can be amplified by the network effect, and that amplification can be more than the sum of the parts. You know, you can be, you can find people interested in extremely obscure things and have a network effect and make something that is truly amazing.
I can open a cafe and that can be my passion project, but I have to get building codes, I have to build a building, I have to find a location, I have to do all the, you know, all sorts of things. Well, in the software world, that passion project can move quickly, it can be amplified by the network effect, and that amplification can be more than the sum of the parts. You know, you can be, you can find people interested in extremely obscure things and have a network effect and make something that is truly amazing.
Lex Fridman
And on that topic of passion projects, Tim Sweeney actually said in a reply to a tweet that was complimenting JB. He said, quote, “Many things in the world only happen because an awesome person decides to do it. This is the case with VLC.” And that speaks to something interesting to me, that it does seem that a small number of people, sometimes one person, can create something incredible in the software world. Like you said this over and over and over. I think JavaScript is an incredible thing created by initially a single person. Some of the programming languages like Python and C and Java, like just one person has this vision, has this design, and brings it—sometimes over a weekend is the initial spark.
And on that topic of passion projects, Tim Sweeney actually said in a reply to a tweet that was complimenting JB. He said, quote, “Many things in the world only happen because an awesome person decides to do it. This is the case with VLC.” And that speaks to something interesting to me, that it does seem that a small number of people, sometimes one person, can create something incredible in the software world. Like you said this over and over and over. I think JavaScript is an incredible thing created by initially a single person. Some of the programming languages like Python and C and Java, like just one person has this vision, has this design, and brings it—sometimes over a weekend is the initial spark.
Jean-Baptiste Kempf
Yes, Linus built Git in two weeks. Wow.
Yes, Linus built Git in two weeks. Wow.
Lex Fridman
It changed the world, Git. I mean, it really changed the world.
It changed the world, Git. I mean, it really changed the world.
Kieran Kunhya
Linus’ passion project. “Hey, I’m uploading this tarball to an FTP, like deal with it.”
Linus’ passion project. “Hey, I’m uploading this tarball to an FTP, like deal with it.”
Jean-Baptiste Kempf
But for me, it’s not just in software, right? And I believe in individuals that are going to change the world, right? And it’s with a good, as you said, vision, right? I want to do that. It is useful, it will be useful. And whether it’s going to like build trains or cars or rockets or something like, I believe people who believe in themselves and have a vision can have a huge impact for humanity.
But for me, it’s not just in software, right? And I believe in individuals that are going to change the world, right? And it’s with a good, as you said, vision, right? I want to do that. It is useful, it will be useful. And whether it’s going to like build trains or cars or rockets or something like, I believe people who believe in themselves and have a vision can have a huge impact for humanity.
VLC and FFmpeg
Lex Fridman
Let’s actually zoom out before we zoom back in. We’ll just keep going up and down the stack. So you know, we’ve been talking back and forth VLC and FFmpeg. Kieran, you said that FFmpeg and VideoLAN, VLC coexist, and there’s no central point of importance. It’s a kind of what you call the binary star system. They succeed because of each other. Can you explain the difference, how they interact? What is the-
Let’s actually zoom out before we zoom back in. We’ll just keep going up and down the stack. So you know, we’ve been talking back and forth VLC and FFmpeg. Kieran, you said that FFmpeg and VideoLAN, VLC coexist, and there’s no central point of importance. It’s a kind of what you call the binary star system. They succeed because of each other. Can you explain the difference, how they interact? What is the-
Kieran Kunhya
Sure
Sure
Lex Fridman
… are they competitors?
… are they competitors?
Kieran Kunhya
I don’t, I don’t think they’re competitors. I think, I think the simple answer is, the short answer before I go into detail is: VLC is to FFmpeg as Android is to Linux. So they depend on each other, but they, they coexist because of each other. So they are a binary star system is the analogy I used.
I don’t, I don’t think they’re competitors. I think, I think the simple answer is, the short answer before I go into detail is: VLC is to FFmpeg as Android is to Linux. So they depend on each other, but they, they coexist because of each other. So they are a binary star system is the analogy I used.
Lex Fridman
By the way, I feel horrible that I just recently learned that Alpha Centauri, the closest star system to us, is a triple star system.
By the way, I feel horrible that I just recently learned that Alpha Centauri, the closest star system to us, is a triple star system.
Jean-Baptiste Kempf
And, and when you start doing the physics, it’s a nightmare, right? But like-
And, and when you start doing the physics, it’s a nightmare, right? But like-
Kieran Kunhya
Hence the three-body problem. But anyway. So a lot of FFmpeg pipelines involve the x264 project, which is a VideoLAN project. I would put a finger in the air and say 80-plus percent of those pipelines are dependent on a VideoLAN project. VLC, obviously, as we’ve discussed, a VideoLAN project, uses FFmpeg, gives it reach, exposure to weird files historically, used some donation money to fund FFmpeg development, and we’ll talk a bit maybe about some of the reverse engineering later. So it’s a binary star system. They work and feed off each other. Many of the developers are shared. There’s no central location. It’s a virtuous cycle working together.
Hence the three-body problem. But anyway. So a lot of FFmpeg pipelines involve the x264 project, which is a VideoLAN project. I would put a finger in the air and say 80-plus percent of those pipelines are dependent on a VideoLAN project. VLC, obviously, as we’ve discussed, a VideoLAN project, uses FFmpeg, gives it reach, exposure to weird files historically, used some donation money to fund FFmpeg development, and we’ll talk a bit maybe about some of the reverse engineering later. So it’s a binary star system. They work and feed off each other. Many of the developers are shared. There’s no central location. It’s a virtuous cycle working together.
Lex Fridman
And we should mention that x264 is the encoder for H.264 video standard. So H.264 is the standard. X264-
And we should mention that x264 is the encoder for H.264 video standard. So H.264 is the standard. X264-
Kieran Kunhya
Is the open source implementation of the standard
Is the open source implementation of the standard
Lex Fridman
… that’s used by basically everybody- … for everything. It’s, that is the main driver of this. When you think of an MP4 file that has H.264 codec in it-
… that’s used by basically everybody- … for everything. It’s, that is the main driver of this. When you think of an MP4 file that has H.264 codec in it-
Kieran Kunhya
If it came from a software environment, like a data center or somewhere, the chances are it was created with x264.
If it came from a software environment, like a data center or somewhere, the chances are it was created with x264.
Lex Fridman
And that’s under the flag of VideoLAN.
And that’s under the flag of VideoLAN.
Kieran Kunhya
That’s a VideoLAN project. So in the VideoLAN graphic, it sits in the VideoLAN world.
That’s a VideoLAN project. So in the VideoLAN graphic, it sits in the VideoLAN world.
Lex Fridman
And VideoLAN has a— says a bunch of stuff in it. Go to the VideoLAN website, there’s a bunch of icons.
And VideoLAN has a— says a bunch of stuff in it. Go to the VideoLAN website, there’s a bunch of icons.
Jean-Baptiste Kempf
Like if you look, there are so many libraries, right?
Like if you look, there are so many libraries, right?
Lex Fridman
libdvdcss- … libdvdnav, libdvbpsi, libVLC of course, vlc-unity, libbluray- Blu-ray.
libdvdcss- … libdvdnav, libdvbpsi, libVLC of course, vlc-unity, libbluray- Blu-ray.
Jean-Baptiste Kempf
Blu-ray.
Blu-ray.
Lex Fridman
Yeah, there’s many more.
Yeah, there’s many more.
Jean-Baptiste Kempf
And there is so many more, right? Lately, the dav1d project that we might talk about is the last project from VideoLAN. It’s everywhere, right? And we do, we have a libspatialaudio lately that we announced. We have a-
And there is so many more, right? Lately, the dav1d project that we might talk about is the last project from VideoLAN. It’s everywhere, right? And we do, we have a libspatialaudio lately that we announced. We have a-
Kieran Kunhya
checkasm.
checkasm.
Jean-Baptiste Kempf
checkasm-
checkasm-
Kieran Kunhya
We’ll talk about that later.
We’ll talk about that later.
Jean-Baptiste Kempf
… which is like an insane project- … but amazing. So, and x264 is one of those VideoLAN projects. And my opinion, for example, is that x264 was, is the most amazing encoder ever designed, and this helped the adoption of FFmpeg. A lot of people and large companies went through FFmpeg because they wanted to use x264, and x264 increased the popularity of FFmpeg. But also VLC had its popularity because it played so many files that were done by FFmpeg, right? So it’s many projects that are intertwined and work together.
… which is like an insane project- … but amazing. So, and x264 is one of those VideoLAN projects. And my opinion, for example, is that x264 was, is the most amazing encoder ever designed, and this helped the adoption of FFmpeg. A lot of people and large companies went through FFmpeg because they wanted to use x264, and x264 increased the popularity of FFmpeg. But also VLC had its popularity because it played so many files that were done by FFmpeg, right? So it’s many projects that are intertwined and work together.
Kieran Kunhya
Yeah. Unfortunately, there’s a thing on X where VLC is mentioned and there’s people, “A quick reminder that it’s FFmpeg inside doing the actual work.” And that’s like I said, it’s not, that’s not the case. We work together.
Yeah. Unfortunately, there’s a thing on X where VLC is mentioned and there’s people, “A quick reminder that it’s FFmpeg inside doing the actual work.” And that’s like I said, it’s not, that’s not the case. We work together.
Jean-Baptiste Kempf
And to give you an idea, right? When I compiled VLC for Windows, I compiled around 16 million lines of code, right? One million of those are inside the VLC repository, and FFmpeg in total is probably around two, right? But so it means that so many dependencies are outside. And if you also look at FFmpeg per se, FFmpeg also is integrating third-party libraries like x264, but libopus and so many others, right? So we all depend on each other.
And to give you an idea, right? When I compiled VLC for Windows, I compiled around 16 million lines of code, right? One million of those are inside the VLC repository, and FFmpeg in total is probably around two, right? But so it means that so many dependencies are outside. And if you also look at FFmpeg per se, FFmpeg also is integrating third-party libraries like x264, but libopus and so many others, right? So we all depend on each other.
History of FFmpeg
Lex Fridman
Yeah, that’s why I was hoping to do this episode as we are doing that just kind of joins FFmpeg and VLC- … because it’s really two of the same, like you said, binary star system and we’re all just orbiting it. Can we give a shout-out to some of the people along the way? We didn’t really quite talk about the history of FFmpeg, so maybe can you tell me about Fabrice? Can you tell me about Michael Niedermayer? Can you tell me about some of the key figures here?
Yeah, that’s why I was hoping to do this episode as we are doing that just kind of joins FFmpeg and VLC- … because it’s really two of the same, like you said, binary star system and we’re all just orbiting it. Can we give a shout-out to some of the people along the way? We didn’t really quite talk about the history of FFmpeg, so maybe can you tell me about Fabrice? Can you tell me about Michael Niedermayer? Can you tell me about some of the key figures here?
Kieran Kunhya
Let’s just talk about the eras of FFmpeg, because there’s key eras and key people that made this possible. Fabrice Bellard, as you mentioned, creating the concept, and then probably in the 2000 era— I would call the era, Eras Tour of FFmpeg— is the 2000 era was Michael Niedermayer. So key things he got done was exhaustive support for DivX and Xvid at the time, and all sorts of weird variants of what’s known as MPEG-4 Part 2. So this predates the MPEG-4 Part 10 that we’re used to. So this was 2000 era video codecs where there were flavor after flavor of weird, weird decoders.
Let’s just talk about the eras of FFmpeg, because there’s key eras and key people that made this possible. Fabrice Bellard, as you mentioned, creating the concept, and then probably in the 2000 era— I would call the era, Eras Tour of FFmpeg— is the 2000 era was Michael Niedermayer. So key things he got done was exhaustive support for DivX and Xvid at the time, and all sorts of weird variants of what’s known as MPEG-4 Part 2. So this predates the MPEG-4 Part 10 that we’re used to. So this was 2000 era video codecs where there were flavor after flavor of weird, weird decoders.
Kieran Kunhya
At the time in the 2000s, you needed a new player to play every different type of file format. So there was Windows Media Player to play Windows Media formats. There was RealPlayer to play RealMedia formats. And those were the other key thing in FFmpeg at the time were native decoders for those. I actually do remember being a teenager, I must have been, figuring out there was this one player that could play, could decode these files without having separate bloated players. Because at the time when you downloaded RealPlayer, there was a ton of other stuff in there, a ton of ads, a ton of other things, and just having a simple library that was fast led to that.
At the time in the 2000s, you needed a new player to play every different type of file format. So there was Windows Media Player to play Windows Media formats. There was RealPlayer to play RealMedia formats. And those were the other key thing in FFmpeg at the time were native decoders for those. I actually do remember being a teenager, I must have been, figuring out there was this one player that could play, could decode these files without having separate bloated players. Because at the time when you downloaded RealPlayer, there was a ton of other stuff in there, a ton of ads, a ton of other things, and just having a simple library that was fast led to that.
Kieran Kunhya
And then I think 2008 was a big change because that’s when H.264 got its maturity and I think something hopefully we’ll talk about a bit more. This was the beginning of high definition video. So H.264 was the key decoder of that. So I’d call that the late 2000s and 2010s, and that’s when the big reverse engineers came along and really did astonishing work. The beginning was a single player that could play Xvid, DivX, Windows Media, and RealPlayer was already a massive achievement in itself without codec packs, without weird stuff you had to download that had weird ads and weird spyware.
And then I think 2008 was a big change because that’s when H.264 got its maturity and I think something hopefully we’ll talk about a bit more. This was the beginning of high definition video. So H.264 was the key decoder of that. So I’d call that the late 2000s and 2010s, and that’s when the big reverse engineers came along and really did astonishing work. The beginning was a single player that could play Xvid, DivX, Windows Media, and RealPlayer was already a massive achievement in itself without codec packs, without weird stuff you had to download that had weird ads and weird spyware.
Jean-Baptiste Kempf
VLC 1.0 was out in those times, 2000, 2009, 2010. And this is like where it exploded.
VLC 1.0 was out in those times, 2000, 2009, 2010. And this is like where it exploded.
Lex Fridman
Yeah, without codec packs, it just works- … across all these different-
Yeah, without codec packs, it just works- … across all these different-
Jean-Baptiste Kempf
It, de facto, it’s just like all the codec packs are FFmpeg inside VLC, plus we have other modules for all the types of codecs.
It, de facto, it’s just like all the codec packs are FFmpeg inside VLC, plus we have other modules for all the types of codecs.
Kieran Kunhya
But back at the time that wasn’t… is there were weird, in the 2000s, there were weird codec packs with DLLs coming from this place, DLLs coming from that-
But back at the time that wasn’t… is there were weird, in the 2000s, there were weird codec packs with DLLs coming from this place, DLLs coming from that-
Jean-Baptiste Kempf
With a lot of spyware.
With a lot of spyware.
Kieran Kunhya
… with spyware, with you know what. It wasn’t reliable, you didn’t know, and having a single player that was open source or single playback module/player that could do this that was open source. But I think the thing to emphasize is this task in the 2000s that Michael did was Sisyphean. It was really, the number of edge cases are poor beyond comprehension in terms of you could have a Chinese CCTV system that did one weird variant of MPEG-4 Part 2, what’s known as MPEG-4 ASP, and that was a weird variant, and you had to fix that without breaking everybody else- … times a million.
… with spyware, with you know what. It wasn’t reliable, you didn’t know, and having a single player that was open source or single playback module/player that could do this that was open source. But I think the thing to emphasize is this task in the 2000s that Michael did was Sisyphean. It was really, the number of edge cases are poor beyond comprehension in terms of you could have a Chinese CCTV system that did one weird variant of MPEG-4 Part 2, what’s known as MPEG-4 ASP, and that was a weird variant, and you had to fix that without breaking everybody else- … times a million.
Reverse engineering codecs
Lex Fridman
So that’s where a lot of the reverse engineering was happening.
So that’s where a lot of the reverse engineering was happening.
Kieran Kunhya
It started in the 2000s with the Windows Media stuff because that was- … proprietary. It started with the RealMedia, so with Benjamin Larsson.
It started in the 2000s with the Windows Media stuff because that was- … proprietary. It started with the RealMedia, so with Benjamin Larsson.
Jean-Baptiste Kempf
Kostya Shishkov.
Kostya Shishkov.
Kieran Kunhya
Kostya Shishkov, that era. Those were the key, that was the key groundwork. And then in the 2010s was kind of the Paul Mahol, Kostya era building, doing some of the most difficult codecs. JB maybe can talk about GoToMeeting 4 and GoToMeeting 5, and-
Kostya Shishkov, that era. Those were the key, that was the key groundwork. And then in the 2010s was kind of the Paul Mahol, Kostya era building, doing some of the most difficult codecs. JB maybe can talk about GoToMeeting 4 and GoToMeeting 5, and-
Lex Fridman
What’s the GoToMeeting?
What’s the GoToMeeting?
Jean-Baptiste Kempf
So, like, let’s talk about this amazing Ukrainian guy called Kostya, who was at that time living in Germany, and who was in love with Sweden, right? He— And the guy was the most… He’s like a lot of the people in the community are very clever. He’s one of those who are, like, borderline geniuses, right? He was able to reverse engineer extremely complex codecs and he does that, and we do a bit of engineering with Kieran, but clearly not at this level.
So, like, let’s talk about this amazing Ukrainian guy called Kostya, who was at that time living in Germany, and who was in love with Sweden, right? He— And the guy was the most… He’s like a lot of the people in the community are very clever. He’s one of those who are, like, borderline geniuses, right? He was able to reverse engineer extremely complex codecs and he does that, and we do a bit of engineering with Kieran, but clearly not at this level.
Kieran Kunhya
No, no, yeah.
No, no, yeah.
Jean-Baptiste Kempf
He reverse engineered binary blobs, which are 20 megabytes?
He reverse engineered binary blobs, which are 20 megabytes?
Kieran Kunhya
Yeah, so just for reference, one megabyte binary blob to reverse engineer is probably order of magnitude a month of work, and this guy is doing 20, 30 megabyte blobs. Maybe we’ll talk about that in a minute, about the subtleties of how you do that. But this guy is doing it for very difficult and very obscure codecs.
Yeah, so just for reference, one megabyte binary blob to reverse engineer is probably order of magnitude a month of work, and this guy is doing 20, 30 megabyte blobs. Maybe we’ll talk about that in a minute, about the subtleties of how you do that. But this guy is doing it for very difficult and very obscure codecs.
Jean-Baptiste Kempf
And did that for fun, right? And so GoToMeeting was a big problem with VLC because that was like the number one feature request for a long time, so I put a bounty. And the guy at some point said, “Okay, JB, I’m going to do it.” And in a matter of two months, and then he explained how he did it. He was just like, “Oh, I looked at the code, like this looked like a DCT that I used to see on WMV and so on.” He did that, and the funniest part is that the code he’s written is a ton of jokes. And there is a ton of JB, right, my name, and Kempf and Kempf and Kostya jokes inside the code. The code is beautiful, right?
And did that for fun, right? And so GoToMeeting was a big problem with VLC because that was like the number one feature request for a long time, so I put a bounty. And the guy at some point said, “Okay, JB, I’m going to do it.” And in a matter of two months, and then he explained how he did it. He was just like, “Oh, I looked at the code, like this looked like a DCT that I used to see on WMV and so on.” He did that, and the funniest part is that the code he’s written is a ton of jokes. And there is a ton of JB, right, my name, and Kempf and Kempf and Kostya jokes inside the code. The code is beautiful, right?
Lex Fridman
So one of the things I wanna comment is I’ve gotten a chance to speak to some of the developers, some of the assembly language level people, and they all always make everything sound like it’s kinda easy. There’s a kind of humility because, maybe just the level of what’s required to do this stuff is so high that everything else seems easy, I guess is the lesson to take away from that.
So one of the things I wanna comment is I’ve gotten a chance to speak to some of the developers, some of the assembly language level people, and they all always make everything sound like it’s kinda easy. There’s a kind of humility because, maybe just the level of what’s required to do this stuff is so high that everything else seems easy, I guess is the lesson to take away from that.
Jean-Baptiste Kempf
So in the community, like some of the most impressive people are the ones doing reverse engineering- … and the other ones doing the assembly folds, right? And both of those type of people are amazing. x264, for example, became amazing because of a guy called Loren Merritt- … who is, was from University of Washington, I think.
So in the community, like some of the most impressive people are the ones doing reverse engineering- … and the other ones doing the assembly folds, right? And both of those type of people are amazing. x264, for example, became amazing because of a guy called Loren Merritt- … who is, was from University of Washington, I think.
Kieran Kunhya
At the time, yeah.
At the time, yeah.
Jean-Baptiste Kempf
And who was, like, who made everything great and fast doing a ton of assembly. So this is like the golden era, I guess, where so many things got done.
And who was, like, who made everything great and fast doing a ton of assembly. So this is like the golden era, I guess, where so many things got done.
Kieran Kunhya
So, yeah, if you look at Kostya, for example, he looked at the world as a binary specification. He didn’t need documentation or anything. It’s, “I have a binary and I can figure all of this out.” And he regularly used the phrase “binary specification.” Ah, you know, it’s not a problem. And he would go away, and he would come back, and he would do interesting stuff.
So, yeah, if you look at Kostya, for example, he looked at the world as a binary specification. He didn’t need documentation or anything. It’s, “I have a binary and I can figure all of this out.” And he regularly used the phrase “binary specification.” Ah, you know, it’s not a problem. And he would go away, and he would come back, and he would do interesting stuff.
Lex Fridman
Can you actually speak to the details or add color and texture to what it takes to reverse engineer a blob?
Can you actually speak to the details or add color and texture to what it takes to reverse engineer a blob?
Kieran Kunhya
Yeah. So let’s look at GoToMeeting, for example, is a good one because I record a meeting on GoToMeeting, for example. How do I play it back without needing this GoToMeeting player? There may not even be a player. I may need to send a recording of a meeting to someone that doesn’t have a player or whatever. So first of all, there’s a ton of other stuff there. There’s an actual video conferencing client. You need to go and find—it may be easy, it may not be easy to find—the actual module doing the decompression. You need a way to actually dump the YUV data from the module. So often it involves opening in a disassembler, trying to guess where the hooks are to incorporate that module and run that module natively to decode a sample file.
Yeah. So let’s look at GoToMeeting, for example, is a good one because I record a meeting on GoToMeeting, for example. How do I play it back without needing this GoToMeeting player? There may not even be a player. I may need to send a recording of a meeting to someone that doesn’t have a player or whatever. So first of all, there’s a ton of other stuff there. There’s an actual video conferencing client. You need to go and find—it may be easy, it may not be easy to find—the actual module doing the decompression. You need a way to actually dump the YUV data from the module. So often it involves opening in a disassembler, trying to guess where the hooks are to incorporate that module and run that module natively to decode a sample file.
Kieran Kunhya
So figure out where this module is doing the decoding process and find a way to hook in and output the raw YUV data, ’cause you will need that- … as a point of comparison for when you actually do the reverse engineering, ’cause you’ll need to be bit exact or in some cases close to bit exact. And then you open up your disassembler, use a lot of intuition to go and figure out, you know, where the DCT is, where’s entropy coding. There is a kind of, not a rule book, but there’s always a pattern of some sort. For example, GoToMeeting, you know it will be a lot of screen codec tools. There’s also different variants, so often I think there’s, what, GoToMeeting 4, 5-
So figure out where this module is doing the decoding process and find a way to hook in and output the raw YUV data, ’cause you will need that- … as a point of comparison for when you actually do the reverse engineering, ’cause you’ll need to be bit exact or in some cases close to bit exact. And then you open up your disassembler, use a lot of intuition to go and figure out, you know, where the DCT is, where’s entropy coding. There is a kind of, not a rule book, but there’s always a pattern of some sort. For example, GoToMeeting, you know it will be a lot of screen codec tools. There’s also different variants, so often I think there’s, what, GoToMeeting 4, 5-
Jean-Baptiste Kempf
Well, 2 or 3, 4, I think.
Well, 2 or 3, 4, I think.
Kieran Kunhya
2, 3, 4.
2, 3, 4.
Lex Fridman
So as you mentioned here, going to Perplexity, GoToMeeting uses its own proprietary codec for older s- recorded sessions historically stored in WMV files that require a special decoder to play properly on Windows. Without this decoder installed, Windows Media Player and some editors cannot decode the video track, so you may only hear audio or see a black screen. Boy, do I remember that. But this is reverse engineering that.
So as you mentioned here, going to Perplexity, GoToMeeting uses its own proprietary codec for older s- recorded sessions historically stored in WMV files that require a special decoder to play properly on Windows. Without this decoder installed, Windows Media Player and some editors cannot decode the video track, so you may only hear audio or see a black screen. Boy, do I remember that. But this is reverse engineering that.
Jean-Baptiste Kempf
This is key, right? Because the GoToMeeting is something that not many people know anymore, right? Well, you know about Zoom and Teams and so on. But like, now let’s fast-forward 10 years, 15 years, and like this is a Gotomeeting.exe for Windows 32 bits, right? Which is like, oh yeah, but I’m on Android, I’m on an iPad, I’m somewhere else, right? How are you going to do that? I’m going to be on RISC-V, on Arm. Those are blocked, but there are tons of files we need to support for the future. And this is why those type of work are exceptionally useful for humanity.
This is key, right? Because the GoToMeeting is something that not many people know anymore, right? Well, you know about Zoom and Teams and so on. But like, now let’s fast-forward 10 years, 15 years, and like this is a Gotomeeting.exe for Windows 32 bits, right? Which is like, oh yeah, but I’m on Android, I’m on an iPad, I’m somewhere else, right? How are you going to do that? I’m going to be on RISC-V, on Arm. Those are blocked, but there are tons of files we need to support for the future. And this is why those type of work are exceptionally useful for humanity.
Lex Fridman
I just have to say, though, that reverse engineering process is mind-blowing. It’s crazy. It’s like, it’s a kinda like, you know, I’ve been reading a lot and interview archeologists. I mean, you just have so little signal. Yes, yes, you know over time you get so much experience, you understand the structure of the original code, so you can kinda start inferring basics. But you’re like archaeologists with a little brush trying to reconstruct the entire human civilization.
I just have to say, though, that reverse engineering process is mind-blowing. It’s crazy. It’s like, it’s a kinda like, you know, I’ve been reading a lot and interview archeologists. I mean, you just have so little signal. Yes, yes, you know over time you get so much experience, you understand the structure of the original code, so you can kinda start inferring basics. But you’re like archaeologists with a little brush trying to reconstruct the entire human civilization.
Kieran Kunhya
Kieran is too humble, but Kieran has done some reverse engineering also.
Kieran is too humble, but Kieran has done some reverse engineering also.
Lex Fridman
Of CineForm, yeah, at the time-
Of CineForm, yeah, at the time-
Kieran Kunhya
CineForm, nice.
CineForm, nice.
Lex Fridman
… yeah, at the time before actually led to the open sourcing of that work. So in parallel to doing the binary side, you obviously have samples. In many cases, you don’t have many samples so you have to figure out what all the different flavors are, and you may have a So CineForm, for example, is actually a collection of different approaches and toolkits within that codec ’cause often it grows naturally. And the hard part is finding a sample that gets you kind of somewhere to start without having to implement 10 different other things. So start there. I think thankfully at the time I found a sample by pure chance that had a lot of flat blocks.
… yeah, at the time before actually led to the open sourcing of that work. So in parallel to doing the binary side, you obviously have samples. In many cases, you don’t have many samples so you have to figure out what all the different flavors are, and you may have a So CineForm, for example, is actually a collection of different approaches and toolkits within that codec ’cause often it grows naturally. And the hard part is finding a sample that gets you kind of somewhere to start without having to implement 10 different other things. So start there. I think thankfully at the time I found a sample by pure chance that had a lot of flat blocks.
Lex Fridman
It was animation, so that really helped a lot because it wasn’t using particularly complex coding tools, et cetera, and you could kind of get somewhere and then, and then build up and build up until you figure, “Hey, here’s a few bits here. I missed this. I missed this, this if branch that it does,” and go, “Oh.” So when we say samples, you mean sample videos- … and then, and then you’re tracking, trying to infer, like, what is this codec doing- … by observing the sample and then looking at what, at the machine lo-
It was animation, so that really helped a lot because it wasn’t using particularly complex coding tools, et cetera, and you could kind of get somewhere and then, and then build up and build up until you figure, “Hey, here’s a few bits here. I missed this. I missed this, this if branch that it does,” and go, “Oh.” So when we say samples, you mean sample videos- … and then, and then you’re tracking, trying to infer, like, what is this codec doing- … by observing the sample and then looking at what, at the machine lo-
Kieran Kunhya
The machine code saying-
The machine code saying-
Lex Fridman
At the machine code.
At the machine code.
Kieran Kunhya
… “Ah, I have byte, this byte is six. Take this branch.” And in a different sample, oh, it’s-
… “Ah, I have byte, this byte is six. Take this branch.” And in a different sample, oh, it’s-
Lex Fridman
That’s nuts, man.
That’s nuts, man.
Kieran Kunhya
And, and, and-
And, and, and-
Lex Fridman
That is nuts.
That is nuts.
Kieran Kunhya
… so you see, this is nuts. Then you go to things like GoToMeeting. It’s like-
… so you see, this is nuts. Then you go to things like GoToMeeting. It’s like-
Lex Fridman
Mine was easy, right?
Mine was easy, right?
Kieran Kunhya
… ima- imagine-
… ima- imagine-
Lex Fridman
Yeah, right.
Yeah, right.
Kieran Kunhya
… two orders of magnitude more complexity. A guy alone somewhere in Germany doing that. And for a long time, you work, you’re in a black box because a decoder, for a long time, because there are so many steps from the entropy decoding, the intra prediction, the motion prediction, the IDCT, and so on. For a long time, you don’t see anything, right? So you’re debugging purely in memory.
… two orders of magnitude more complexity. A guy alone somewhere in Germany doing that. And for a long time, you work, you’re in a black box because a decoder, for a long time, because there are so many steps from the entropy decoding, the intra prediction, the motion prediction, the IDCT, and so on. For a long time, you don’t see anything, right? So you’re debugging purely in memory.
Lex Fridman
Debugging guesswork.
Debugging guesswork.
Kieran Kunhya
And you may have the buffer that the coefficients are stored in completely wrong, and so you may be going down a complete rabbit hole thinking it’s this and then, oh damn, that’s not, that’s, that’s something else, and-
And you may have the buffer that the coefficients are stored in completely wrong, and so you may be going down a complete rabbit hole thinking it’s this and then, oh damn, that’s not, that’s, that’s something else, and-
Lex Fridman
And you’re doing that on binaries that are tens of megabytes, millions of instructions, right?
And you’re doing that on binaries that are tens of megabytes, millions of instructions, right?
Kieran Kunhya
So you’re, you’re stepping through the debugger, like one by one, you know, instruction by instruction going, “Hey, this instruction changes this. This does this.”
So you’re, you’re stepping through the debugger, like one by one, you know, instruction by instruction going, “Hey, this instruction changes this. This does this.”
Lex Fridman
Pausing the program on the CPU level. Like it’s-
Pausing the program on the CPU level. Like it’s-
Kieran Kunhya
Pausing it, yeah, on the CPU level, watching what’s going on, trying to figure out-
Pausing it, yeah, on the CPU level, watching what’s going on, trying to figure out-
Lex Fridman
Sometimes you need to, like, be in a VM, so that you can pause the VM.
Sometimes you need to, like, be in a VM, so that you can pause the VM.
Kieran Kunhya
Yeah, pause the VM, dump the memory, ’cause there could, some of the codecs could have encryption. There could be like a DRM on there. So you need to dump the memory from a virtual machine.
Yeah, pause the VM, dump the memory, ’cause there could, some of the codecs could have encryption. There could be like a DRM on there. So you need to dump the memory from a virtual machine.
Lex Fridman
Like when I joined École Centrale Paris in 2003, Jon Lech Johansen basically broke the DVD specification and created DeCSS, showed us how he was breaking a DRM, which was MP4 FairPlay from Apple. What he did on his laptop, and I was young, I was 21, was just like mind-blowing because he was basically debugging Windows inside a type of VM with ex- Like, wow. It’s incredible. It’s mind-blowing and inspiring. Does it get, like from your experience and from what you’ve seen in the community, does it get discouraging? Does it get-
Like when I joined École Centrale Paris in 2003, Jon Lech Johansen basically broke the DVD specification and created DeCSS, showed us how he was breaking a DRM, which was MP4 FairPlay from Apple. What he did on his laptop, and I was young, I was 21, was just like mind-blowing because he was basically debugging Windows inside a type of VM with ex- Like, wow. It’s incredible. It’s mind-blowing and inspiring. Does it get, like from your experience and from what you’ve seen in the community, does it get discouraging? Does it get-
Kieran Kunhya
People help you. People send you samples. People are keen. Sometimes you don’t have access to an encoder, so this is even more difficult because you just, you just ask and you have to ask for samples. I remember VideoLAN used to tweet for samples at one stage. “Hey, I need this obscure sample,” and-
People help you. People send you samples. People are keen. Sometimes you don’t have access to an encoder, so this is even more difficult because you just, you just ask and you have to ask for samples. I remember VideoLAN used to tweet for samples at one stage. “Hey, I need this obscure sample,” and-
Lex Fridman
For a long time I was, “Oh, I need this codec, and I need this codec.”
For a long time I was, “Oh, I need this codec, and I need this codec.”
Kieran Kunhya
And if you were really lucky, you would find like… If you were unlucky, you’d get nothing or you’d get one or two, and then they would… Sometimes you’d find a goldmine. It’s like, “Yeah, my company has 100,000 of these files ’cause we’re dependent on it for some reason.” And so those are the, those are kind of the best because then they can test bit exactness across the huge range of coding tools.
And if you were really lucky, you would find like… If you were unlucky, you’d get nothing or you’d get one or two, and then they would… Sometimes you’d find a goldmine. It’s like, “Yeah, my company has 100,000 of these files ’cause we’re dependent on it for some reason.” And so those are the, those are kind of the best because then they can test bit exactness across the huge range of coding tools.
Lex Fridman
Can you explain bit exactness?
Can you explain bit exactness?
Kieran Kunhya
Bit exactness, so most but not all video codecs, certainly from about the 2000s onwards, have a bit exact definition, so every implementation must produce exactly the same bits, bit for bit, in exactly the same data that comes out of a decoder.
Bit exactness, so most but not all video codecs, certainly from about the 2000s onwards, have a bit exact definition, so every implementation must produce exactly the same bits, bit for bit, in exactly the same data that comes out of a decoder.
Lex Fridman
For like a large number of samples?
For like a large number of samples?
Kieran Kunhya
For a given sample. So Lex’s implementation, JB’s implementation, and my implementation of H.264 must match bit exactly. That wasn’t the case in the ’90s of MPEG-2, probably fair to say one of the biggest mistakes the video industry made, and I think people who were in the room in ’92—I don’t think most or both of us were in diapers, I suspect—but have acknowledged… I would give a shout-out to Yuriy Reznik. He’s acknowledged that was one of the big mistakes of the era.
For a given sample. So Lex’s implementation, JB’s implementation, and my implementation of H.264 must match bit exactly. That wasn’t the case in the ’90s of MPEG-2, probably fair to say one of the biggest mistakes the video industry made, and I think people who were in the room in ’92—I don’t think most or both of us were in diapers, I suspect—but have acknowledged… I would give a shout-out to Yuriy Reznik. He’s acknowledged that was one of the big mistakes of the era.
Lex Fridman
And you’re saying the encoders needed to be able to run tests and then the, the, the bit exactness—I mean, that’s a nice thing to guarantee. Like there’s a parallel sort of development here on the way the, the web browser works, which is a, you know, takes HTML and displays it, and there’s no bit exactness there across the different engines.
And you’re saying the encoders needed to be able to run tests and then the, the, the bit exactness—I mean, that’s a nice thing to guarantee. Like there’s a parallel sort of development here on the way the, the web browser works, which is a, you know, takes HTML and displays it, and there’s no bit exactness there across the different engines.
Kieran Kunhya
I would, I would point out actually FFmpeg is unique in the sense that it’s, it has been a winner-takes-all scenario. You have… Browsers is a good analogy because it has to parse a lot of different content and render it in a particular way, like a decoder. But there still are multiple browser engines. There’s Firefox’s one, there’s Chrome’s one, there’s a few Japanese ones that are pretty decent. That’s not been the case in multimedia in general across a wide range of codecs. FFmpeg has kind of won it all, I suppose, in a sense because of, because of the fact that you can get—every new codec added is actually worth more than the value of that codec itself because it makes the whole thing better.
I would, I would point out actually FFmpeg is unique in the sense that it’s, it has been a winner-takes-all scenario. You have… Browsers is a good analogy because it has to parse a lot of different content and render it in a particular way, like a decoder. But there still are multiple browser engines. There’s Firefox’s one, there’s Chrome’s one, there’s a few Japanese ones that are pretty decent. That’s not been the case in multimedia in general across a wide range of codecs. FFmpeg has kind of won it all, I suppose, in a sense because of, because of the fact that you can get—every new codec added is actually worth more than the value of that codec itself because it makes the whole thing better.
Lex Fridman
Man, this is really cool. Going to Perplexity. Yuriy Reznik is a multimedia and signal processing researcher, got his PhD in computer science from Kyiv University with over 150 papers and more than 80 granted US patents, contributor to major multimedia standards including H.264, MPEG-4, AVC-H.265 MPEG-4 ALS, G.718, and-
Man, this is really cool. Going to Perplexity. Yuriy Reznik is a multimedia and signal processing researcher, got his PhD in computer science from Kyiv University with over 150 papers and more than 80 granted US patents, contributor to major multimedia standards including H.264, MPEG-4, AVC-H.265 MPEG-4 ALS, G.718, and-
Kieran Kunhya
G.71 is telco stuff. Telco.
G.71 is telco stuff. Telco.
Lex Fridman
Oh. And so he was more connected to companies.
Oh. And so he was more connected to companies.
Kieran Kunhya
RealAudio, RealVideo, right? That was- … very important at that time
RealAudio, RealVideo, right? That was- … very important at that time
Lex Fridman
… Zencoder, Brightcove, Contex. This, man, I need to hang out with Yuriy. He’s legit. And he’s like one of the nicest person-
… Zencoder, Brightcove, Contex. This, man, I need to hang out with Yuriy. He’s legit. And he’s like one of the nicest person-
Kieran Kunhya
Slack guy, yeah
Slack guy, yeah
Lex Fridman
… ever, right? Like for example, for my startup that I’m doing right now called Kyber, right? I met Yuriy because I met him every year at the Mile-High Video Conference, which is in Denver. And he gave me like so many good ideas and good things. He’s like really amazing person.
… ever, right? Like for example, for my startup that I’m doing right now called Kyber, right? I met Yuriy because I met him every year at the Mile-High Video Conference, which is in Denver. And he gave me like so many good ideas and good things. He’s like really amazing person.
Kieran Kunhya
He tells us how great it is to be, you know, even know us. And then we just like, you know, you look at that and it’s—I think it’s the other way around, Yuriy.
He tells us how great it is to be, you know, even know us. And then we just like, you know, you look at that and it’s—I think it’s the other way around, Yuriy.
FFmpeg testing
Lex Fridman
That reminds me of a thing that you mentioned to me about FATE testing and, like, the insanely rigorous process that’s used to test everything that’s incorporated into FFmpeg. Can you take me through the testing process?
That reminds me of a thing that you mentioned to me about FATE testing and, like, the insanely rigorous process that’s used to test everything that’s incorporated into FFmpeg. Can you take me through the testing process?
Kieran Kunhya
Yeah. So FFmpeg has a system called FATE, FFmpeg Automated Testing Environment. Because FFmpeg runs on so many different OSs and can be compiled with so many different compilers, there’s been a crazy number of configurations. So you can see the absurd combination of compiler variants, operating system variants, instruction sets. You can see at the top macOS has tons of different variants because it has iOS, it has tvOS.
Yeah. So FFmpeg has a system called FATE, FFmpeg Automated Testing Environment. Because FFmpeg runs on so many different OSs and can be compiled with so many different compilers, there’s been a crazy number of configurations. So you can see the absurd combination of compiler variants, operating system variants, instruction sets. You can see at the top macOS has tons of different variants because it has iOS, it has tvOS.
Lex Fridman
Well, I’m looking at a page fate.ffmpeg.org 81 minutes ago, 76 minutes ago, looking at the different architectures, the operating systems, the different compilers, Apple Clang version-
Well, I’m looking at a page fate.ffmpeg.org 81 minutes ago, 76 minutes ago, looking at the different architectures, the operating systems, the different compilers, Apple Clang version-
Kieran Kunhya
Combinations are crazy
Combinations are crazy
Lex Fridman
… the combination is insane. RISC-
… the combination is insane. RISC-
Kieran Kunhya
So these are all run by volunteers, so these are all volunteer systems. The ones at the top, for example, the Macs I host in my office, for example, host all sorts of different stuff. Other people host other things. So it’s really there to make sure… because FFmpeg does quite complex C code, for example, you do have miscompilations. So the compiler will sometimes compile C code incorrectly. For example, this happens once in a while.
So these are all run by volunteers, so these are all volunteer systems. The ones at the top, for example, the Macs I host in my office, for example, host all sorts of different stuff. Other people host other things. So it’s really there to make sure… because FFmpeg does quite complex C code, for example, you do have miscompilations. So the compiler will sometimes compile C code incorrectly. For example, this happens once in a while.
Lex Fridman
Oh, there’s like, there’s a log of all the compilations.
Oh, there’s like, there’s a log of all the compilations.
Kieran Kunhya
Yeah, log of all the compilations, all the tests. I think one of the other ones will show all the tests passing.
Yeah, log of all the compilations, all the tests. I think one of the other ones will show all the tests passing.
Jean-Baptiste Kempf
If you click, you can see all the tests- … back. All tests successful.
If you click, you can see all the tests- … back. All tests successful.
Kieran Kunhya
In logs test, yeah. So you see all those tests are passing of all the different codecs, all the different filter transformations, all the— the level of scale is quite crazy.
In logs test, yeah. So you see all those tests are passing of all the different codecs, all the different filter transformations, all the— the level of scale is quite crazy.
Lex Fridman
Oh, that’s nuts.
Oh, that’s nuts.
Kieran Kunhya
On all the combinations. It’s not just a matrix at this point. It’s like a pivot table of different combinations.
On all the combinations. It’s not just a matrix at this point. It’s like a pivot table of different combinations.
Lex Fridman
That’s nuts.
That’s nuts.
Kieran Kunhya
And it’s a key part of what we do because you may be able to test something locally, you make a change, but actually that breaks GCC version 11 on Mac or something like that, and you’re able to then fix that. We also have miscompilation, so the C code, sometimes the compiler can have a bug in it where it creates the wrong output, and that can have quite a big effect sometimes on a video because of the way frames have dependencies. Even a small change in the output can cascade to actually quite big glitches.
And it’s a key part of what we do because you may be able to test something locally, you make a change, but actually that breaks GCC version 11 on Mac or something like that, and you’re able to then fix that. We also have miscompilation, so the C code, sometimes the compiler can have a bug in it where it creates the wrong output, and that can have quite a big effect sometimes on a video because of the way frames have dependencies. Even a small change in the output can cascade to actually quite big glitches.
Jean-Baptiste Kempf
You see PowerPC, you see RISC, you see ARM.
You see PowerPC, you see RISC, you see ARM.
Kieran Kunhya
There was PowerPC, there was RISC, there was weird stuff in the past like DEC Alpha. There was-
There was PowerPC, there was RISC, there was weird stuff in the past like DEC Alpha. There was-
Jean-Baptiste Kempf
You see Visual Studio, different versions of Clang or GCC.
You see Visual Studio, different versions of Clang or GCC.
Kieran Kunhya
Visual Studio, Intel compiler, Apple Clang, you name it.
Visual Studio, Intel compiler, Apple Clang, you name it.
Lex Fridman
What are some of the pain points? Like maybe do you have emotional triggers, maybe nightmares, about a particular operating system, a particular container, codec combination of—
What are some of the pain points? Like maybe do you have emotional triggers, maybe nightmares, about a particular operating system, a particular container, codec combination of—
Kieran Kunhya
I mean, for me, it’s really easy because I have a day job. My company builds… The company I started builds equipment for broadcasting sports matches between TV stadiums and studios, for example. We have to work with 10-bit video, and 10-bit video has a set of challenges that you can’t process 10-bit data natively on a CPU. So that means you have to stick it in 16 bits. So that means you have six wasted bits. So there’s different packing formats to actually pack the data more efficiently because when you send that over a network, you need to save that 40%. For example, on PCI Express, you may only have bus bandwidth to do that. And so I think internally we have about…
I mean, for me, it’s really easy because I have a day job. My company builds… The company I started builds equipment for broadcasting sports matches between TV stadiums and studios, for example. We have to work with 10-bit video, and 10-bit video has a set of challenges that you can’t process 10-bit data natively on a CPU. So that means you have to stick it in 16 bits. So that means you have six wasted bits. So there’s different packing formats to actually pack the data more efficiently because when you send that over a network, you need to save that 40%. For example, on PCI Express, you may only have bus bandwidth to do that. And so I think internally we have about…
Kieran Kunhya
Some are industry ones and some are internal to our own hardware that we build. We have a, I think, a 5 by 5 or 6 by 6 matrix of every single format to every single other format conversion. In fact, one of them I sent you, and they’re all written in handwritten assembly, and they all support different CPU generations. So this is really traumatic, handling all these different combinations times a million.
Some are industry ones and some are internal to our own hardware that we build. We have a, I think, a 5 by 5 or 6 by 6 matrix of every single format to every single other format conversion. In fact, one of them I sent you, and they’re all written in handwritten assembly, and they all support different CPU generations. So this is really traumatic, handling all these different combinations times a million.
Jean-Baptiste Kempf
By the way, the company you’re talking about is Open Broadcast Systems.
By the way, the company you’re talking about is Open Broadcast Systems.
Assembly code (handwritten)
Kieran Kunhya
Yeah, so no, no relation to the free OBS streaming service. But JB and I have started companies broadly speaking around the FFmpeg VLC ethos, so that’s really low-level work. So in most companies, this wouldn’t be written in assembly. It would be accepted that C is fast. As you can see from that, C is not fast.
Yeah, so no, no relation to the free OBS streaming service. But JB and I have started companies broadly speaking around the FFmpeg VLC ethos, so that’s really low-level work. So in most companies, this wouldn’t be written in assembly. It would be accepted that C is fast. As you can see from that, C is not fast.
Lex Fridman
So here it says 62 times faster than C.
So here it says 62 times faster than C.
Kieran Kunhya
Yeah. So it’s taking the ethos of doing low-level programming, real-time programming, and using that for commercial applications, and JB and I have started companies around that, in many cases hiring developers from the open source community to use that ethos. And so that’s a great example of some of the things we’re doing. In most companies, it would be, say, “Oh, I’ll write this in C and it’s fast and we’re done,” but actually you can get a lot better.
Yeah. So it’s taking the ethos of doing low-level programming, real-time programming, and using that for commercial applications, and JB and I have started companies around that, in many cases hiring developers from the open source community to use that ethos. And so that’s a great example of some of the things we’re doing. In most companies, it would be, say, “Oh, I’ll write this in C and it’s fast and we’re done,” but actually you can get a lot better.
Jean-Baptiste Kempf
For me, some of the headaches we have is around some OS that are difficult to support, right? Because if you look at VLC and thanks to FATE and FFmpeg, we run on… The last version of VLC runs on Windows XP and still run there and runs on Windows 11. We work on macOS 10.7 to the latest macOS, whatever it is, right, 26. We work on iOS since iOS 9; well, we are actually iOS 26, right? We support many types of Linuxes, BSD, Solaris. The last version still runs on OS/2, right? Like there is maybe 10 users of OS/2 in the world, and one of them is maintaining VLC.
For me, some of the headaches we have is around some OS that are difficult to support, right? Because if you look at VLC and thanks to FATE and FFmpeg, we run on… The last version of VLC runs on Windows XP and still run there and runs on Windows 11. We work on macOS 10.7 to the latest macOS, whatever it is, right, 26. We work on iOS since iOS 9; well, we are actually iOS 26, right? We support many types of Linuxes, BSD, Solaris. The last version still runs on OS/2, right? Like there is maybe 10 users of OS/2 in the world, and one of them is maintaining VLC.
Jean-Baptiste Kempf
Then you realize that this very small team around VLC and using FFmpeg codecs and all the other ones support more OSs than Microsoft or Google or Apple, and they have infinite amount of power and resources. But for example, the worst is iOS. In order to build on iOS 9, we need to do some very clever mixing of several version of the Xcode IDE and SDK from Apple, and do a type of Frankenstein version of that so that we can still support iOS 9, which is not supported at all by the compiler of Apple in order to still run on Arm32 on iOS 9. And you’ve seen on FATE that it was still supporting iOS 9, right? So my headaches are mostly related to the support of so many OSs.
Then you realize that this very small team around VLC and using FFmpeg codecs and all the other ones support more OSs than Microsoft or Google or Apple, and they have infinite amount of power and resources. But for example, the worst is iOS. In order to build on iOS 9, we need to do some very clever mixing of several version of the Xcode IDE and SDK from Apple, and do a type of Frankenstein version of that so that we can still support iOS 9, which is not supported at all by the compiler of Apple in order to still run on Arm32 on iOS 9. And you’ve seen on FATE that it was still supporting iOS 9, right? So my headaches are mostly related to the support of so many OSs.
Jean-Baptiste Kempf
And it’s important because, like, we receive so many people saying, “Hey, thank you. I still have my iPad 2 to watch movies,” and it still works on iOS 9, right? And it’s also an impact of, like, not forcing people to buy new hardware when it works fine if you optimize it correctly. Which brings us to what we were saying about assembly. It’s also fighting, like, the fact that you need to buy something new nonstop while you could optimize more, which is a lost art.
And it’s important because, like, we receive so many people saying, “Hey, thank you. I still have my iPad 2 to watch movies,” and it still works on iOS 9, right? And it’s also an impact of, like, not forcing people to buy new hardware when it works fine if you optimize it correctly. Which brings us to what we were saying about assembly. It’s also fighting, like, the fact that you need to buy something new nonstop while you could optimize more, which is a lost art.
Lex Fridman
You gotta tell me about this lost art or this, uh- … the carriers of the flame of assembly. What is Assembly? Why is it beautiful? Why is it challenging? How does it work?
You gotta tell me about this lost art or this, uh- … the carriers of the flame of assembly. What is Assembly? Why is it beautiful? Why is it challenging? How does it work?
Kieran Kunhya
So when you write assembly code, you write this using the instructions the actual processor is using directly. So most of the time you would write in a language, let’s take C as a good example. The compiler would use that to create assembly language and machine code instructions for you based off your C code. And there’s a specific flavor of assembly that we use in FFmpeg that’s called SIMD, single instruction, multiple data. So this means, for example, say I want to add five to a number in scalar assembly, so this is what’s known as you work on an individual element. So I want to have a number of– I have the number ten and I want to add five. I use the add instruction, and I add five to ten, and I get 15.
So when you write assembly code, you write this using the instructions the actual processor is using directly. So most of the time you would write in a language, let’s take C as a good example. The compiler would use that to create assembly language and machine code instructions for you based off your C code. And there’s a specific flavor of assembly that we use in FFmpeg that’s called SIMD, single instruction, multiple data. So this means, for example, say I want to add five to a number in scalar assembly, so this is what’s known as you work on an individual element. So I want to have a number of– I have the number ten and I want to add five. I use the add instruction, and I add five to ten, and I get 15.
Kieran Kunhya
With SIMD, I can have a whole vector of 16 different numbers. They could all be different. If I want to add five to that, I can run one instruction, and that one instruction sums all 16 elements. And that, as you can imagine, lends itself very well to video. Video is, you know, pixel grid, so I can perform operations on multiple pixels at the same time. The key thing that we do differently in FFmpeg is we don’t use any abstractions or any major abstractions on top of that. So there’s a part of the world that uses what’s known as intrinsics. So these are C functions that behave very similarly but not quite the same to writing assembly by hand. So the registers that data is stored in on the CPU, the compiler allocates those for you.
With SIMD, I can have a whole vector of 16 different numbers. They could all be different. If I want to add five to that, I can run one instruction, and that one instruction sums all 16 elements. And that, as you can imagine, lends itself very well to video. Video is, you know, pixel grid, so I can perform operations on multiple pixels at the same time. The key thing that we do differently in FFmpeg is we don’t use any abstractions or any major abstractions on top of that. So there’s a part of the world that uses what’s known as intrinsics. So these are C functions that behave very similarly but not quite the same to writing assembly by hand. So the registers that data is stored in on the CPU, the compiler allocates those for you.
Kieran Kunhya
And so the key thing to understand when we write SIMD is we have a 10x, and not percentage, 10x to 50x speed improvement. That function is 62x—
And so the key thing to understand when we write SIMD is we have a 10x, and not percentage, 10x to 50x speed improvement. That function is 62x—
Lex Fridman
That’s nuts.
That’s nuts.
Kieran Kunhya
… on the FFmpeg account, as you know, posts and tweets a lot about that to try and say, “Hey, we are doing this stuff.”
… on the FFmpeg account, as you know, posts and tweets a lot about that to try and say, “Hey, we are doing this stuff.”
Lex Fridman
You are a person who sees the beauty in assembly, but it’s also extremely useful for these kinds of application to actually- … significantly outperform even C, which is crazy.
You are a person who sees the beauty in assembly, but it’s also extremely useful for these kinds of application to actually- … significantly outperform even C, which is crazy.
Jean-Baptiste Kempf
It is necessary. Right? Because, like, one of the projects that we need to talk about is called dav1d, right? So dav1d is a decoder for the format that was done by Alliance for Open Media, which is a video decoder called AV1.
It is necessary. Right? Because, like, one of the projects that we need to talk about is called dav1d, right? So dav1d is a decoder for the format that was done by Alliance for Open Media, which is a video decoder called AV1.
Lex Fridman
So if, for people who don’t know, we’ve been talking about H.264. AV1 is another hugely popular standard and codec that is increasingly taking over the internet.
So if, for people who don’t know, we’ve been talking about H.264. AV1 is another hugely popular standard and codec that is increasingly taking over the internet.
Jean-Baptiste Kempf
And when this format was launched many people said, especially even from the Alliance for Open Media, right, which is Google, Netflix, Amazon, Mozilla, say, “Well, this format is so complex, it must be done in hardware to do decoding,” right? And well, I arrived with a few other people, mostly Ronald, Henrik, and Martin, and we said, “We need to have an extremely good software decoder because it’s going to take time to have hardware.” And so we wrote this project, which is beyond insane. We are talking about 30,000 lines of C, but 240,000 lines of handwritten assembly, right?
And when this format was launched many people said, especially even from the Alliance for Open Media, right, which is Google, Netflix, Amazon, Mozilla, say, “Well, this format is so complex, it must be done in hardware to do decoding,” right? And well, I arrived with a few other people, mostly Ronald, Henrik, and Martin, and we said, “We need to have an extremely good software decoder because it’s going to take time to have hardware.” And so we wrote this project, which is beyond insane. We are talking about 30,000 lines of C, but 240,000 lines of handwritten assembly, right?
Lex Fridman
Handwritten assembly, 240,000 lines. That’s incredible. That means—I mean, some of the stuff we’re talking about is probably the biggest assembly code bases.
Handwritten assembly, 240,000 lines. That’s incredible. That means—I mean, some of the stuff we’re talking about is probably the biggest assembly code bases.
Jean-Baptiste Kempf
To give you an idea, and Kieran can correct me, but I think the FFmpeg has 100,000 lines of assembly for all the codecs.
To give you an idea, and Kieran can correct me, but I think the FFmpeg has 100,000 lines of assembly for all the codecs.
Kieran Kunhya
For all codecs. Mm-hmm.
For all codecs. Mm-hmm.
Jean-Baptiste Kempf
And just this one has 240,000. It’s a VideoLAN project, of course. And it is optimized at the maximum because the motto when we’re starting the project is every cycle matters, right? Every cycle matters because dav1d is used in VLC and in some software AV1 playback stacks. We are talking about probably 3 billion devices which are going to decode video nonstop because, for example, 30% of the video from Netflix are now in AV1, 50% of YouTube, right? So, and you often don’t have a hardware decoder because not many devices have a hardware decoder. And with dav1d, we realized that with one or two cores you were able to decode 720p correctly. So it is, like, literally-
And just this one has 240,000. It’s a VideoLAN project, of course. And it is optimized at the maximum because the motto when we’re starting the project is every cycle matters, right? Every cycle matters because dav1d is used in VLC and in some software AV1 playback stacks. We are talking about probably 3 billion devices which are going to decode video nonstop because, for example, 30% of the video from Netflix are now in AV1, 50% of YouTube, right? So, and you often don’t have a hardware decoder because not many devices have a hardware decoder. And with dav1d, we realized that with one or two cores you were able to decode 720p correctly. So it is, like, literally-
Kieran Kunhya
Yeah, that’s dav1d.
Yeah, that’s dav1d.
Jean-Baptiste Kempf
… incredible, right?
… incredible, right?
Kieran Kunhya
That’s dav1d. Look at that, Lex.
That’s dav1d. Look at that, Lex.
Lex Fridman
Yeah, so this is another spicy tweet from you. This is what peak video codec should look like, 79.9% assembly-
Yeah, so this is another spicy tweet from you. This is what peak video codec should look like, 79.9% assembly-
Kieran Kunhya
That’s almost
That’s almost
Lex Fridman
… 19.6% C and 0.5% other.
… 19.6% C and 0.5% other.
Jean-Baptiste Kempf
And what’s incredible is with those tweets, which are factual, people get crazy. They are unhappy, right? They say-
And what’s incredible is with those tweets, which are factual, people get crazy. They are unhappy, right? They say-
Kieran Kunhya
For a year, for the last two years they go crazy, “No, intrinsics is fine. The compiler is…” Oh, they go, “I have never-“
For a year, for the last two years they go crazy, “No, intrinsics is fine. The compiler is…” Oh, they go, “I have never-“
Jean-Baptiste Kempf
“You can optimize your compiler, auto-vectorization, it’s your fault, you don’t understand.” And we’ve tried that forever, right?
“You can optimize your compiler, auto-vectorization, it’s your fault, you don’t understand.” And we’ve tried that forever, right?
Kieran Kunhya
For two years, and two years later, showing hundreds of examples of handwritten assembly. “No, no, no, you’re doing it wrong. The compiler can do this.”
For two years, and two years later, showing hundreds of examples of handwritten assembly. “No, no, no, you’re doing it wrong. The compiler can do this.”
Lex Fridman
So we should actually just articulate a little clearer. So the intuition there from the software engineering folks, when you have code like… Okay, let’s just take an example, C++. There’s a compiler that’s doing a lot of the optimization. And the presumption is if you have a good enough compiler, if you continue to improve the compiler, you’re going to generate code- … that can perform like optimal performance. You cannot possibly beat it. And you’re consistently challenging that thought that if you do-
So we should actually just articulate a little clearer. So the intuition there from the software engineering folks, when you have code like… Okay, let’s just take an example, C++. There’s a compiler that’s doing a lot of the optimization. And the presumption is if you have a good enough compiler, if you continue to improve the compiler, you’re going to generate code- … that can perform like optimal performance. You cannot possibly beat it. And you’re consistently challenging that thought that if you do-
Kieran Kunhya
By orders of magnitude.
By orders of magnitude.
Lex Fridman
… by orders of magnitude- … handcrafted assembly can outperform C.
… by orders of magnitude- … handcrafted assembly can outperform C.
Jean-Baptiste Kempf
The two things that they tell us is, yeah, but modern compilers have auto-vectorization, right? Because SIMD that we’re doing is vectorization. And like it’s not even close, right? It’s not even close, right? It’s not like 5%, 10% slower. It’s multiple times slower.
The two things that they tell us is, yeah, but modern compilers have auto-vectorization, right? Because SIMD that we’re doing is vectorization. And like it’s not even close, right? It’s not even close, right? It’s not like 5%, 10% slower. It’s multiple times slower.
Lex Fridman
So can we… I don’t know if you can say something philosophically, because there’s a lot of, there’s a lot of great software engineers, great engineers, great machine learning people. Karpathy will listen to this and say, “What’s the intuition he’s supposed to get from this? What are we supposed to…”
So can we… I don’t know if you can say something philosophically, because there’s a lot of, there’s a lot of great software engineers, great engineers, great machine learning people. Karpathy will listen to this and say, “What’s the intuition he’s supposed to get from this? What are we supposed to…”
Kieran Kunhya
Karpathy learnt assembly because of the tweets, by the way. I just… He start- He went, he’s like, “Oh, I think this is a movement.”
Karpathy learnt assembly because of the tweets, by the way. I just… He start- He went, he’s like, “Oh, I think this is a movement.”
Lex Fridman
He’s like, “Let me figure out what’s happening here.”
He’s like, “Let me figure out what’s happening here.”
Kieran Kunhya
No, no, he… and you know the way he documents his work and so on.
No, no, he… and you know the way he documents his work and so on.
Jean-Baptiste Kempf
Philosophically, what’s important to realize is that we passed the time where hardware was going so much faster, right? We are at the end of Moore’s law. We have limitation for AI, for memory. You need to go down in the stack and optimize more to get more power from what you have, because our request for power, CPU power, GPU power are exploding while the hardware is not exploding in speed, right? So what people do is that they add more cores, right? But that’s basically like at some point you can add 250 cores, right? So what we do is to take every inch of the machine.
Philosophically, what’s important to realize is that we passed the time where hardware was going so much faster, right? We are at the end of Moore’s law. We have limitation for AI, for memory. You need to go down in the stack and optimize more to get more power from what you have, because our request for power, CPU power, GPU power are exploding while the hardware is not exploding in speed, right? So what people do is that they add more cores, right? But that’s basically like at some point you can add 250 cores, right? So what we do is to take every inch of the machine.
Kieran Kunhya
Not just that, not just that. We abuse the machine. We go and use the machine in ways that the creator didn’t expect. Sometimes we use an instruction that’s completely unrelated to what we do. We use a cryptography instruction in video processing to do nothing related.
Not just that, not just that. We abuse the machine. We go and use the machine in ways that the creator didn’t expect. Sometimes we use an instruction that’s completely unrelated to what we do. We use a cryptography instruction in video processing to do nothing related.
Jean-Baptiste Kempf
And one of other things that we do, for example in dav1d, which is a bit crazy, is that we don’t use the function calling convention from the operating system.
And one of other things that we do, for example in dav1d, which is a bit crazy, is that we don’t use the function calling convention from the operating system.
Kieran Kunhya
We should explain that.
We should explain that.
Jean-Baptiste Kempf
That is extremely- … complex. But basically, usually when you do move from one function in code to another, there is a way to save the registry, the state of the CPU to enter another function. And this is like standard.
That is extremely- … complex. But basically, usually when you do move from one function in code to another, there is a way to save the registry, the state of the CPU to enter another function. And this is like standard.
Kieran Kunhya
It’s a bit complex. I would simplify this a bit. So, dav1d does things to abuse the calling convention. You could define the calling convention as I’ve written a function and I want to call another function. How is the data shared between the functions? Because there’s a convention, what’s known as a calling convention, and what dav1d does for optimal reasons is create its own calling convention sometimes. So if I wanna call Lex Fridman’s library, we’ve got to agree on a convention so that I can share data with you in the assembly language space. And one of the challenges in assembly is every operating s- well, not every operating system, but there are, well, at least four that I can think of on x86, Linux 32-bit, Windows 32-bit, Windows 64, Linux 64.
It’s a bit complex. I would simplify this a bit. So, dav1d does things to abuse the calling convention. You could define the calling convention as I’ve written a function and I want to call another function. How is the data shared between the functions? Because there’s a convention, what’s known as a calling convention, and what dav1d does for optimal reasons is create its own calling convention sometimes. So if I wanna call Lex Fridman’s library, we’ve got to agree on a convention so that I can share data with you in the assembly language space. And one of the challenges in assembly is every operating s- well, not every operating system, but there are, well, at least four that I can think of on x86, Linux 32-bit, Windows 32-bit, Windows 64, Linux 64.
Kieran Kunhya
They all have their own calling conventions. And so one of the amazing things Loren Merritt did, who we talked about before, was create a very lightweight abstraction layer, so you could write your assembly code once and it handled all the calling convention stuff for you, which was always a problem because you had to manage four different variants. But dav1d takes this even further, for speed reasons it does its own calling convention within itself to bypass the kind of rules, the rules of functions and say, “Okay, actually I’m gonna call a function this way because I know it’s within my library.”
They all have their own calling conventions. And so one of the amazing things Loren Merritt did, who we talked about before, was create a very lightweight abstraction layer, so you could write your assembly code once and it handled all the calling convention stuff for you, which was always a problem because you had to manage four different variants. But dav1d takes this even further, for speed reasons it does its own calling convention within itself to bypass the kind of rules, the rules of functions and say, “Okay, actually I’m gonna call a function this way because I know it’s within my library.”
Lex Fridman
Does it have to be special to every single operating system?
Does it have to be special to every single operating system?
Kieran Kunhya
Well, if it’s custom, no. But the challenge is in general, yes, and in terms of each instruction set. So the thing to also emphasize is we do this on every instruction set. So every instruction set has its own handwritten assembly, which is even more crazy. And that matrix has got bigger in recent years because of RISC-V, because of ARM64, because of the new SVE. There’s SME. x86 has AVX-512, AVX. So we do runtime processor detection. We see what the machine FFmpeg is running on or dav1d’s running on is capable of, because you could be on a laptop from 2008 where this isn’t there. Runtime detection, we set function pointers accordingly. And then from then on, off you go.
Well, if it’s custom, no. But the challenge is in general, yes, and in terms of each instruction set. So the thing to also emphasize is we do this on every instruction set. So every instruction set has its own handwritten assembly, which is even more crazy. And that matrix has got bigger in recent years because of RISC-V, because of ARM64, because of the new SVE. There’s SME. x86 has AVX-512, AVX. So we do runtime processor detection. We see what the machine FFmpeg is running on or dav1d’s running on is capable of, because you could be on a laptop from 2008 where this isn’t there. Runtime detection, we set function pointers accordingly. And then from then on, off you go.
Lex Fridman
Or you could be on a machine with RISC-V.
Or you could be on a machine with RISC-V.
Jean-Baptiste Kempf
Yes. And in all that, we don’t even respect the calling convention of the operating system in order to be faster, because we know that we are going to be called from within our binary, so we can share data without saving all the registry in the common way, because that can lead to loading and saving registry on the L1 and L2 CPU and gets us faster. So that’s why I said that understanding CPU architecture, computer architecture is key. And this is also why it’s handwritten. I don’t know anyone, I’ve never heard any other project than Dav1d doing that. This is why Kieran calls it an art, right? It is an art.
Yes. And in all that, we don’t even respect the calling convention of the operating system in order to be faster, because we know that we are going to be called from within our binary, so we can share data without saving all the registry in the common way, because that can lead to loading and saving registry on the L1 and L2 CPU and gets us faster. So that’s why I said that understanding CPU architecture, computer architecture is key. And this is also why it’s handwritten. I don’t know anyone, I’ve never heard any other project than Dav1d doing that. This is why Kieran calls it an art, right? It is an art.
Kieran Kunhya
I think in a mass world, there isn’t something on billions of devices. I know there are some specialist industries. I know in high-frequency trading, they take this really seriously, where they’re receiving feeds from a market, and they need to react within X number of microseconds, and so the instructions matter. But that’s not a mass-produced thing that’s on a billion devices. That’s hyper-specialized, running on hyper-specialized hardware. We’re running on all hardware from-
I think in a mass world, there isn’t something on billions of devices. I know there are some specialist industries. I know in high-frequency trading, they take this really seriously, where they’re receiving feeds from a market, and they need to react within X number of microseconds, and so the instructions matter. But that’s not a mass-produced thing that’s on a billion devices. That’s hyper-specialized, running on hyper-specialized hardware. We’re running on all hardware from-
Lex Fridman
Sorry to linger on it, but, like, that’s a really counterintuitive, almost, like, revolutionary idea here, that there’s a huge amount of value to assembly. Like, what are we supposed to take away from that? Like, what… You know, there’s a bunch of people listening to this, they’re basically like, sorry, for myself included, you know, I programmed for many, many years in C/C++, going up the standards of C++, fell in love with C++, even meta programming and so on, and then transitioned more and more because of machine learning about 15 years ago to Python.
Sorry to linger on it, but, like, that’s a really counterintuitive, almost, like, revolutionary idea here, that there’s a huge amount of value to assembly. Like, what are we supposed to take away from that? Like, what… You know, there’s a bunch of people listening to this, they’re basically like, sorry, for myself included, you know, I programmed for many, many years in C/C++, going up the standards of C++, fell in love with C++, even meta programming and so on, and then transitioned more and more because of machine learning about 15 years ago to Python.
Lex Fridman
And so, like, for me in this Python world, JavaScript world, now vibe coding, where I’m just using natural language, sitting in my jacuzzi, drinking a drink… and just talking to the computer, like record stops. Why is the value to go back all the way down to the low level? Like, what’s the intuition?
And so, like, for me in this Python world, JavaScript world, now vibe coding, where I’m just using natural language, sitting in my jacuzzi, drinking a drink… and just talking to the computer, like record stops. Why is the value to go back all the way down to the low level? Like, what’s the intuition?
Jean-Baptiste Kempf
Because you can get more power per dollar invested, right? And sometimes it’s going to be a problem that is limited by your hardware. A good analogy is what you see in quantization in LLMs, right? And people are doing, “Oh, I’m going to do that in FP8 or FP4 or some crazy things like Microsoft Phi, who did it in 1.5,” because you’re constrained by memory, because you’re constrained by the machine you can run. Because at some point we are doing real time, and I believe this is going to happen on AI inference also, is that at some point you need to get faster, and you cannot always get more powerful hardware, right? So you need to analyze code and see where, like, where is the mission critical, where are the things that are called nonstops.
Because you can get more power per dollar invested, right? And sometimes it’s going to be a problem that is limited by your hardware. A good analogy is what you see in quantization in LLMs, right? And people are doing, “Oh, I’m going to do that in FP8 or FP4 or some crazy things like Microsoft Phi, who did it in 1.5,” because you’re constrained by memory, because you’re constrained by the machine you can run. Because at some point we are doing real time, and I believe this is going to happen on AI inference also, is that at some point you need to get faster, and you cannot always get more powerful hardware, right? So you need to analyze code and see where, like, where is the mission critical, where are the things that are called nonstops.
Jean-Baptiste Kempf
And for example, dav1d is a good example. It’s going to be run billions of hours per day. That makes sense. It doesn’t make sense to be on the glue of FFmpeg- … CLI. It makes sense over there.
And for example, dav1d is a good example. It’s going to be run billions of hours per day. That makes sense. It doesn’t make sense to be on the glue of FFmpeg- … CLI. It makes sense over there.
Lex Fridman
Yeah, and this has to do, also we’ll talk about it more, but your new effort, your new company, Kyber, is doing that kind of thing for ultra-low latency, so the slogan being, “Every millisecond counts.” And when you actually extremely highly constrained in some dimension-
Yeah, and this has to do, also we’ll talk about it more, but your new effort, your new company, Kyber, is doing that kind of thing for ultra-low latency, so the slogan being, “Every millisecond counts.” And when you actually extremely highly constrained in some dimension-
Jean-Baptiste Kempf
We are also arriving at a point where we’ve done so many great things, but the hardware is getting back to us, right? Because cost is increasing, because we need more power, and so you’re limited by either your CPU, your RAM, or your networking, and you need to optimize, and this is where value is going to be. Especially because, like, doing AI is going to help do the programming of, like, business, right? And so the core thing that you will not be able to vibe code are optimization for the hardware to be as fast as is possible.
We are also arriving at a point where we’ve done so many great things, but the hardware is getting back to us, right? Because cost is increasing, because we need more power, and so you’re limited by either your CPU, your RAM, or your networking, and you need to optimize, and this is where value is going to be. Especially because, like, doing AI is going to help do the programming of, like, business, right? And so the core thing that you will not be able to vibe code are optimization for the hardware to be as fast as is possible.
Lex Fridman
I’d love to talk to you about who and how should learn assembly, but first, I think we need a bathroom break. Quick ten-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. And now back to the episode. All right, and we’re back. There’s this nice repo with the assembly lessons. First of all, do you think developers should learn how to program in assembly, and how would you go about learning it? What is this asm-lessons?
I’d love to talk to you about who and how should learn assembly, but first, I think we need a bathroom break. Quick ten-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. And now back to the episode. All right, and we’re back. There’s this nice repo with the assembly lessons. First of all, do you think developers should learn how to program in assembly, and how would you go about learning it? What is this asm-lessons?
Kieran Kunhya
So I personally wasn’t happy with the way assembly is taught in books and online, ’cause it’s very grammar-focused, and you don’t, in general, learn a language from learning the grammar and the structure. You learn a language by asking someone what their name is, and you start from there, and you go and solve real problems that you have when you want to communicate. You don’t learn sentence structure, and this is the interrogative and the adverb, and all the assembly books seem to be doing like that, going through every instruction, even ones that aren’t really relevant, explaining what they all do and how they… It actually doesn’t really change much.
So I personally wasn’t happy with the way assembly is taught in books and online, ’cause it’s very grammar-focused, and you don’t, in general, learn a language from learning the grammar and the structure. You learn a language by asking someone what their name is, and you start from there, and you go and solve real problems that you have when you want to communicate. You don’t learn sentence structure, and this is the interrogative and the adverb, and all the assembly books seem to be doing like that, going through every instruction, even ones that aren’t really relevant, explaining what they all do and how they… It actually doesn’t really change much.
Kieran Kunhya
So, and the other problem that we have in our community is assembly is taught sort of hand to hand, like person to person, like blacksmithing one by one. That’s the only logical sort of analogy, and that doesn’t really scale online. It doesn’t do other things. So I’ve started a set of assembly lessons in the way it’s done in FFmpeg, which is a little bit different to the way assembly in general for… I don’t know. I’m trying to think the other good big use case of assembly is in embedded devices, in really low power, cheap devices, and that’s completely different to what we’re doing here. I think it would be good if you could highlight the requirements, which are quite simple. It’s high school mathematics and C.
So, and the other problem that we have in our community is assembly is taught sort of hand to hand, like person to person, like blacksmithing one by one. That’s the only logical sort of analogy, and that doesn’t really scale online. It doesn’t do other things. So I’ve started a set of assembly lessons in the way it’s done in FFmpeg, which is a little bit different to the way assembly in general for… I don’t know. I’m trying to think the other good big use case of assembly is in embedded devices, in really low power, cheap devices, and that’s completely different to what we’re doing here. I think it would be good if you could highlight the requirements, which are quite simple. It’s high school mathematics and C.
Kieran Kunhya
And actually not even C, really, really it’s pointers. To emphasize, yes, we’ve talked about how brilliant this stuff is, but high schoolers like Daniel Kang have written assembly in FFmpeg. I think there’s been contributions because of these lessons. So it’s really about trying to get this dying art to continue, because we’ve shown it’s possible with dav1d to produce something amazing. There’s still a lot of codecs in FFmpeg that are only maybe partially assembly optimized. And so it really starts with basics and continues, explains a lot of the jargon, a lot of the syntax. It doesn’t really try and explain to you, you know, interrupt handlers and interrupt instructions and all of these different jump targets; actually makes this really vector focused.
And actually not even C, really, really it’s pointers. To emphasize, yes, we’ve talked about how brilliant this stuff is, but high schoolers like Daniel Kang have written assembly in FFmpeg. I think there’s been contributions because of these lessons. So it’s really about trying to get this dying art to continue, because we’ve shown it’s possible with dav1d to produce something amazing. There’s still a lot of codecs in FFmpeg that are only maybe partially assembly optimized. And so it really starts with basics and continues, explains a lot of the jargon, a lot of the syntax. It doesn’t really try and explain to you, you know, interrupt handlers and interrupt instructions and all of these different jump targets; actually makes this really vector focused.
Lex Fridman
And describes all kinds of registers, general purpose registers, vector registers; really nice examples. Oh, this is cool.
And describes all kinds of registers, general purpose registers, vector registers; really nice examples. Oh, this is cool.
Kieran Kunhya
It’s a classic, yeah, it’s a classic example of FFmpeg. But some of this assembly language is really beautiful, and I think it’s beautiful because it’s kind of like flying a Spitfire. It’s really aviation at its purest, but also pushing the aircraft beyond what the designer thought was possible. So we’re abusing, for example, sometimes cryptography instructions to do certain things, and there’s a level of beauty and art where it’s really you and the processor.
It’s a classic, yeah, it’s a classic example of FFmpeg. But some of this assembly language is really beautiful, and I think it’s beautiful because it’s kind of like flying a Spitfire. It’s really aviation at its purest, but also pushing the aircraft beyond what the designer thought was possible. So we’re abusing, for example, sometimes cryptography instructions to do certain things, and there’s a level of beauty and art where it’s really you and the processor.
Kieran Kunhya
There’s nothing in between. It’s you and the joystick of the cockpit, and you move that joystick, and it’s physically connected to the ailerons, and you can push that plane beyond what it can normally do, and there’s a level of, yeah, beauty and amazingness to go that. But I don’t think the sort of person-by-person assembly that is… someone taught me, and I’ve taught multiple people, is gonna work long run just because of the particular flavor and the way that we do it.
There’s nothing in between. It’s you and the joystick of the cockpit, and you move that joystick, and it’s physically connected to the ailerons, and you can push that plane beyond what it can normally do, and there’s a level of, yeah, beauty and amazingness to go that. But I don’t think the sort of person-by-person assembly that is… someone taught me, and I’ve taught multiple people, is gonna work long run just because of the particular flavor and the way that we do it.
Lex Fridman
It’s literally no, I should… I was gonna say wizards handing it down. I realize I look like a wizard- … wearing this hat. But you’re basically just like the sages, the wise sages handing- … down the craft. Can I ask you about LLMs? Like- … can they help?
It’s literally no, I should… I was gonna say wizards handing it down. I realize I look like a wizard- … wearing this hat. But you’re basically just like the sages, the wise sages handing- … down the craft. Can I ask you about LLMs? Like- … can they help?
Kieran Kunhya
They had more of an understanding than I expected, but they are still… I’ve asked it questions, and it still goes and starts not hallucinating, but making modifications, and then I go, “Is it bit exact?” “No.” “Fix it.” And then it just goes and does the same thing, and it’s going, it… There isn’t the corpus of information like Stack Overflow to work on.
They had more of an understanding than I expected, but they are still… I’ve asked it questions, and it still goes and starts not hallucinating, but making modifications, and then I go, “Is it bit exact?” “No.” “Fix it.” And then it just goes and does the same thing, and it’s going, it… There isn’t the corpus of information like Stack Overflow to work on.
Jean-Baptiste Kempf
There is not enough data to train on. And this is the biggest issue. I started my career actually doing some assembly for Itanium, right? So the Itanium is a dead processor type, which was done by Intel and HP a long time ago when they wanted to do 64 bits. Well, they lost, and then we got AMD, who did it, AMD64, which became x86-64. But Itanium was extremely interesting in the sense that those were processors who had a ton of computing power to do floats, FMAs, which is similar to what we need now for LLMs, right? And you could pack three operations per line that could be loaded. So basically, you had an output of basically six billion operations per second, but the memory bus only allowed 1.5, right?
There is not enough data to train on. And this is the biggest issue. I started my career actually doing some assembly for Itanium, right? So the Itanium is a dead processor type, which was done by Intel and HP a long time ago when they wanted to do 64 bits. Well, they lost, and then we got AMD, who did it, AMD64, which became x86-64. But Itanium was extremely interesting in the sense that those were processors who had a ton of computing power to do floats, FMAs, which is similar to what we need now for LLMs, right? And you could pack three operations per line that could be loaded. So basically, you had an output of basically six billion operations per second, but the memory bus only allowed 1.5, right?
Jean-Baptiste Kempf
So your CPU was four times faster, so you had to do crazy things to pack things in memory or reuse the registers, and those type of semantics, no language could do that, right? So like I have the Itanium programming book because Intel did amazing books, but that’s exactly what Kieran says. If you don’t know what you’re going to do, it’s impossible to read, right? It’s a ton of jargon and so on. While those lessons are amazing because they are targeted to a real problem, and you can do it yourself.
So your CPU was four times faster, so you had to do crazy things to pack things in memory or reuse the registers, and those type of semantics, no language could do that, right? So like I have the Itanium programming book because Intel did amazing books, but that’s exactly what Kieran says. If you don’t know what you’re going to do, it’s impossible to read, right? It’s a ton of jargon and so on. While those lessons are amazing because they are targeted to a real problem, and you can do it yourself.
Kieran Kunhya
And people have. People have. There are patches, and they said, “Oh, I studied your lessons, and here’s my first changes.”
And people have. People have. There are patches, and they said, “Oh, I studied your lessons, and here’s my first changes.”
Lex Fridman
That’s amazing.
That’s amazing.
Jean-Baptiste Kempf
And part of that in the lessons is a framework called x86inc, written by Loren when he was working on x264, and it allows you to do more things about that to create a type of like not caring too much about different calling conventions. And we had a lot of students who gave code to x264 using that a long time ago, right? So it’s really doable, and I believe it’s necessary to understand assembly language, even if you don’t do it much, to understand what’s going on inside your computer, and that will make you a better programmer. And I assure you that because doing that, you will understand some of the architecture of the memory inside your computer, right?
And part of that in the lessons is a framework called x86inc, written by Loren when he was working on x264, and it allows you to do more things about that to create a type of like not caring too much about different calling conventions. And we had a lot of students who gave code to x264 using that a long time ago, right? So it’s really doable, and I believe it’s necessary to understand assembly language, even if you don’t do it much, to understand what’s going on inside your computer, and that will make you a better programmer. And I assure you that because doing that, you will understand some of the architecture of the memory inside your computer, right?
Jean-Baptiste Kempf
Understanding registers, L1, L2, L3, RAM, SSDs, disk, and so on, which are very important because then you have a good programming culture that will make you a better programmer.
Understanding registers, L1, L2, L3, RAM, SSDs, disk, and so on, which are very important because then you have a good programming culture that will make you a better programmer.
Rust programming language
Lex Fridman
What do you think about the Rust programming language? ‘Cause that’s a bit of a meme.
What do you think about the Rust programming language? ‘Cause that’s a bit of a meme.
Jean-Baptiste Kempf
We have very different opinions with Kieran.
We have very different opinions with Kieran.
Kieran Kunhya
I think it’s valuable what they’re doing in terms of memory safety as a concept.
I think it’s valuable what they’re doing in terms of memory safety as a concept.
Lex Fridman
Can it achieve some of the speed up that assembly achieves?
Can it achieve some of the speed up that assembly achieves?
Kieran Kunhya
Oh, not assembly by hand, no. I think that that’s a given. C potentially, but I see it very… It has a very big Esperanto vibe about it. It’s like we’re gonna solve this, and we’re doing this in a particular way.
Oh, not assembly by hand, no. I think that that’s a given. C potentially, but I see it very… It has a very big Esperanto vibe about it. It’s like we’re gonna solve this, and we’re doing this in a particular way.
Lex Fridman
Meaning it’s a bit too utopian?
Meaning it’s a bit too utopian?
Kieran Kunhya
There’s a lot of focus on the self-importance rather than solving real-world problems. It reminds me of the Sinclair C5. Sir Clive Sinclair of Sinclair Computers built a car, and he said, “Oh, everyone will be traveling around in one of these electric cars.” And it was… Rust reminds me of that, where I think the community doesn’t quite understand that in order to get people to move, you have to build something that’s as good as, if not better than what you have now. Yes, people are doing Rust rewrites, but if they only do 85, 90% of the feature set of what we need, like things like coreutils, that last 1% takes 99% of the time.
There’s a lot of focus on the self-importance rather than solving real-world problems. It reminds me of the Sinclair C5. Sir Clive Sinclair of Sinclair Computers built a car, and he said, “Oh, everyone will be traveling around in one of these electric cars.” And it was… Rust reminds me of that, where I think the community doesn’t quite understand that in order to get people to move, you have to build something that’s as good as, if not better than what you have now. Yes, people are doing Rust rewrites, but if they only do 85, 90% of the feature set of what we need, like things like coreutils, that last 1% takes 99% of the time.
Kieran Kunhya
To use Elon’s famous quote, “Prototypes are easy.” Like this kind of stuff is easy. But this, to get a real electric car, you have to make a car as good as, if not better than what we have now, and Rust isn’t in that stage yet. I think we’d– I don’t think anyone would object to seeing Rust code in FFmpeg, but it needs to work as well and support the same unit testing as everything else. It needs to be flawless. It can’t just randomly break. They can’t just randomly break ABI when they want to. It needs to have, I think, more– I think it still has only one compiler implementation. So it’s got to be as good as, if not better, and saying, “Hey, here’s my utopia of memory safety,” isn’t enough, even though we probably all agree that that’s the goal.
To use Elon’s famous quote, “Prototypes are easy.” Like this kind of stuff is easy. But this, to get a real electric car, you have to make a car as good as, if not better than what we have now, and Rust isn’t in that stage yet. I think we’d– I don’t think anyone would object to seeing Rust code in FFmpeg, but it needs to work as well and support the same unit testing as everything else. It needs to be flawless. It can’t just randomly break. They can’t just randomly break ABI when they want to. It needs to have, I think, more– I think it still has only one compiler implementation. So it’s got to be as good as, if not better, and saying, “Hey, here’s my utopia of memory safety,” isn’t enough, even though we probably all agree that that’s the goal.
Jean-Baptiste Kempf
So I’ve done a ton of Rust, and the two major topics I had was adding Rust modules inside VLC. One of the reasons VLC got popular and which was one of the main architectural decision, is that VLC is a very small core and a ton of modules, right? And so you can write modules in C, in C++, in Objective-C, and anything that is basically interoperable with C. And so we did some Rust modules, and so I have experience on that, and I wrote some of it. And also, like, my new startup called Kyber, is an open source project mainly done in Rust. What Rust is extremely good in, in the sense that it’s a better C++ that cares about memory and allows you to do things about memory ownership that no one else can do so far.
So I’ve done a ton of Rust, and the two major topics I had was adding Rust modules inside VLC. One of the reasons VLC got popular and which was one of the main architectural decision, is that VLC is a very small core and a ton of modules, right? And so you can write modules in C, in C++, in Objective-C, and anything that is basically interoperable with C. And so we did some Rust modules, and so I have experience on that, and I wrote some of it. And also, like, my new startup called Kyber, is an open source project mainly done in Rust. What Rust is extremely good in, in the sense that it’s a better C++ that cares about memory and allows you to do things about memory ownership that no one else can do so far.
Jean-Baptiste Kempf
However, it’s great when you start a new project from scratch, and you do everything in Rust. But it’s very not good when you interop with existing part. And some part of the Rust community believes that they need to rewrite everything, and everything will be better with Rust. And the answer is like, no. Like, I’m almost always, in all my years of being engineer, manager, CTO of startup and so on, don’t rewrite, right?
However, it’s great when you start a new project from scratch, and you do everything in Rust. But it’s very not good when you interop with existing part. And some part of the Rust community believes that they need to rewrite everything, and everything will be better with Rust. And the answer is like, no. Like, I’m almost always, in all my years of being engineer, manager, CTO of startup and so on, don’t rewrite, right?
Lex Fridman
Is that– That’s the, that’s the initial instinct for a lot of people when they show up to a code base, probably before LLMs, is like… probably because they don’t understand the, the, the wisdom of the way things have been done in the past. They say, “Well, we need to rewrite it.” Hence why there’s a thousand JavaScript frameworks.
Is that– That’s the, that’s the initial instinct for a lot of people when they show up to a code base, probably before LLMs, is like… probably because they don’t understand the, the, the wisdom of the way things have been done in the past. They say, “Well, we need to rewrite it.” Hence why there’s a thousand JavaScript frameworks.
Jean-Baptiste Kempf
But the reason is the following, and this is very important to understand. It is an order of magnitude easier to write code than read code. And you see that also with LLM. They can write code, but analyzing is a lot- … more difficult. And so when you arrive to a very complex piece of code, right? You don’t understand it, right? Because it’s so much more effort to understand the code from someone else because you don’t have the thought process. And often I joke about some languages, mostly Perl for example, which has very complex syntax. And imagine I am at my maximum intellectual efficiency in programming, right? And I write the best code ever. I will not be able to understand it myself six months later, right?
But the reason is the following, and this is very important to understand. It is an order of magnitude easier to write code than read code. And you see that also with LLM. They can write code, but analyzing is a lot- … more difficult. And so when you arrive to a very complex piece of code, right? You don’t understand it, right? Because it’s so much more effort to understand the code from someone else because you don’t have the thought process. And often I joke about some languages, mostly Perl for example, which has very complex syntax. And imagine I am at my maximum intellectual efficiency in programming, right? And I write the best code ever. I will not be able to understand it myself six months later, right?
Jean-Baptiste Kempf
Because reading code is more difficult. So very often you arrive, you don’t understand all the wisdom, all the business logic, the reasons that were done that is maybe not documented. And you say, “Well, I’m going to write it.” And the thing is, no, you don’t, right? Because that’s, as Kieran said, right? I’m going to rewrite coreutils in Rust. And then, of course, you arrive very quickly at eighty percent then ninety percent, takes a bit more time, and then you got the last ones, right? On the other side, right? So for new projects, it’s great. Everything related to parsing files networking because of the memory checker, boundary checker, it’s amazing, and there is nothing else.
Because reading code is more difficult. So very often you arrive, you don’t understand all the wisdom, all the business logic, the reasons that were done that is maybe not documented. And you say, “Well, I’m going to write it.” And the thing is, no, you don’t, right? Because that’s, as Kieran said, right? I’m going to rewrite coreutils in Rust. And then, of course, you arrive very quickly at eighty percent then ninety percent, takes a bit more time, and then you got the last ones, right? On the other side, right? So for new projects, it’s great. Everything related to parsing files networking because of the memory checker, boundary checker, it’s amazing, and there is nothing else.
Jean-Baptiste Kempf
To answer a bit differently for us, imagine I take a piece of software like dav1d or x264, right? Which has a ton of runtime in assembly, right? I rewrite the C part in Rust, right? So it’s more secure. Yes. But then you arrive into the assembly, and you can jump anywhere in the memory because we are doing handwritten assembly. So even if I rewrite the C part in Rust, for security reason, you break all the security when you write handwritten assembly because we can jump anywhere. So in my opinion, we need to do something that is secure assembly, right?
To answer a bit differently for us, imagine I take a piece of software like dav1d or x264, right? Which has a ton of runtime in assembly, right? I rewrite the C part in Rust, right? So it’s more secure. Yes. But then you arrive into the assembly, and you can jump anywhere in the memory because we are doing handwritten assembly. So even if I rewrite the C part in Rust, for security reason, you break all the security when you write handwritten assembly because we can jump anywhere. So in my opinion, we need to do something that is secure assembly, right?
Jean-Baptiste Kempf
So which is compile time, check the assembly, which is similar to the checkasm projects that we’re doing on dav1d and x264 with VideoLAN, is to start instrumenting your assembly at compile time to check that it’s not jumping anywhere in the memory. Because else you might rewrite a part of C in Rust, but if you want to have the same performances, you’re going to have inline assembly, and so you destroy your whole security model. So that’s a bit what I think about Rust.
So which is compile time, check the assembly, which is similar to the checkasm projects that we’re doing on dav1d and x264 with VideoLAN, is to start instrumenting your assembly at compile time to check that it’s not jumping anywhere in the memory. Because else you might rewrite a part of C in Rust, but if you want to have the same performances, you’re going to have inline assembly, and so you destroy your whole security model. So that’s a bit what I think about Rust.
Kieran Kunhya
No, I just wanna… I would say on a personal level, I’m so in awe about assembly. I actually— Once in a… It never gets old, the speed improvements to show sixty-two x. So there are months, on a personal level, I run our internal test suite at work and just see I’m still in awe at the gains we have.
No, I just wanna… I would say on a personal level, I’m so in awe about assembly. I actually— Once in a… It never gets old, the speed improvements to show sixty-two x. So there are months, on a personal level, I run our internal test suite at work and just see I’m still in awe at the gains we have.
Lex Fridman
Well, there’s a source of joy and happiness with programming for different reasons. But I think one of the greatest happiness is in the optimization of code. And it sounds like you’re, like, at the cutting edge of that.
Well, there’s a source of joy and happiness with programming for different reasons. But I think one of the greatest happiness is in the optimization of code. And it sounds like you’re, like, at the cutting edge of that.
Kieran Kunhya
I was like, “Whoa, that was cool.”
I was like, “Whoa, that was cool.”
Jean-Baptiste Kempf
And in the community, I want to speak about two people who are wizards of assembly, right? The two of them are actually working living in north of Europe, Sweden and Finland. And Henrik Gramner knows so much about Intel x86 assembly that when we ask questions at Intel about things, they tell, like, “Why are you asking us, Intel? You have Henrik. Henrik knows better.” He knows all the cycles of almost all the SIMD instruction by all the CPU generation. “Oh, yes, this is a P4, this is a Nehalem, this is a Core 2,” et cetera. That person is, like, the best person on assembly in the world.
And in the community, I want to speak about two people who are wizards of assembly, right? The two of them are actually working living in north of Europe, Sweden and Finland. And Henrik Gramner knows so much about Intel x86 assembly that when we ask questions at Intel about things, they tell, like, “Why are you asking us, Intel? You have Henrik. Henrik knows better.” He knows all the cycles of almost all the SIMD instruction by all the CPU generation. “Oh, yes, this is a P4, this is a Nehalem, this is a Core 2,” et cetera. That person is, like, the best person on assembly in the world.
Jean-Baptiste Kempf
And he’s the nicest person that you’ve seen, like, very g- He arrives, you don’t see he’s amazing. And the other one is called Martin, Martin Storsjö, and he’s– they’re doing mostly the same on Arm, right? So Neon, right? And iPhones and Androids and so on. And he codes in assembly on his phone, editing it with the crappy keyboard, like virtual keyboard you have while watching his kids play in the playground, right? Like, this is just like wizard level. So those two people are like-
And he’s the nicest person that you’ve seen, like, very g- He arrives, you don’t see he’s amazing. And the other one is called Martin, Martin Storsjö, and he’s– they’re doing mostly the same on Arm, right? So Neon, right? And iPhones and Androids and so on. And he codes in assembly on his phone, editing it with the crappy keyboard, like virtual keyboard you have while watching his kids play in the playground, right? Like, this is just like wizard level. So those two people are like-
Lex Fridman
Yes. So a part, when you’re programming assembly at that high level, a part of that is knowing the architecture that you’re programming on. So x86 and-
Yes. So a part, when you’re programming assembly at that high level, a part of that is knowing the architecture that you’re programming on. So x86 and-
Jean-Baptiste Kempf
On Arm in particular, yes.
On Arm in particular, yes.
Lex Fridman
… Arm in particular. But x86, I mean, these are complicated architectures, right?
… Arm in particular. But x86, I mean, these are complicated architectures, right?
Kieran Kunhya
Yeah. But Arm in some ways is more com… x86 with out of order execution is not so bad. Arm, you really need to understand all the different generations of Arm processor because they’re all different. There’s A72, … … Et cetera, et cetera. And there’s the Apple variant, there’s this variant, there’s that, and you need to write code that works efficiently on all of them. x86, well, broadly speaking, you have Intel, AMD, and you have sub-variants, but generally speaking, there’s… Something fast is gonna remain fast on all of the variants, whereas in Arm it’s a completely much more complicated ballgame.
Yeah. But Arm in some ways is more com… x86 with out of order execution is not so bad. Arm, you really need to understand all the different generations of Arm processor because they’re all different. There’s A72, … … Et cetera, et cetera. And there’s the Apple variant, there’s this variant, there’s that, and you need to write code that works efficiently on all of them. x86, well, broadly speaking, you have Intel, AMD, and you have sub-variants, but generally speaking, there’s… Something fast is gonna remain fast on all of the variants, whereas in Arm it’s a completely much more complicated ballgame.
FFmpeg and Libav fork
Lex Fridman
We’re taking a nonlinear journey through history here, but we’re talking about Michael Niedermayer. And I wanted to ask about this. For a time there was a split in FFmpeg and Libav.
We’re taking a nonlinear journey through history here, but we’re talking about Michael Niedermayer. And I wanted to ask about this. For a time there was a split in FFmpeg and Libav.
Jean-Baptiste Kempf
Yes. So in open source projects sometimes you disagree, right?
Yes. So in open source projects sometimes you disagree, right?
Lex Fridman
You have such a nice way of putting it, yeah.
You have such a nice way of putting it, yeah.
Jean-Baptiste Kempf
And the good thing is because of the license, you’re allowed to basically do your own, right? And this is normal, and this has happened all the time, right? At a point there was a GCC at the time of GCC 2 and EGCS which became then GCC 3, right? There is what we told KHTML with WebKit, with Blink. It is a same process. And also, like when I want to do a new feature today in VLC, I fork, I do my thing on my own, and then I merge back to the community. So there was a split in the open source community on FFmpeg, which become Libav and FFmpeg. And after a few years, well, the community merged back and people moved on. It’s a bit of drama that is normal in open source community, but forks are even… They’re important because they change the status quo of a community.
And the good thing is because of the license, you’re allowed to basically do your own, right? And this is normal, and this has happened all the time, right? At a point there was a GCC at the time of GCC 2 and EGCS which became then GCC 3, right? There is what we told KHTML with WebKit, with Blink. It is a same process. And also, like when I want to do a new feature today in VLC, I fork, I do my thing on my own, and then I merge back to the community. So there was a split in the open source community on FFmpeg, which become Libav and FFmpeg. And after a few years, well, the community merged back and people moved on. It’s a bit of drama that is normal in open source community, but forks are even… They’re important because they change the status quo of a community.
Jean-Baptiste Kempf
Not talking about FFmpeg and Libav here, but the GCC fork made GCC a ton better because some people wanted to change the architecture fundamentally to make it faster. And of course, it’s always question of people and so on, but in the end you realize that FFmpeg today is better than it was before the fork. And now, well, we’re back all together, right? And I spent a lot of time—and Kieran can say—in the community. It’s not often, to be honest, very well explained because a ton of the reasons are not very public. But I think that’s, that’s normal and that’s good.
Not talking about FFmpeg and Libav here, but the GCC fork made GCC a ton better because some people wanted to change the architecture fundamentally to make it faster. And of course, it’s always question of people and so on, but in the end you realize that FFmpeg today is better than it was before the fork. And now, well, we’re back all together, right? And I spent a lot of time—and Kieran can say—in the community. It’s not often, to be honest, very well explained because a ton of the reasons are not very public. But I think that’s, that’s normal and that’s good.
Lex Fridman
Yeah. I mean, you’re making it sound really nice, but there, there is battle, there’s pretty heated battles inside open source projects. I mean, it is a very passionate community and you’re kind of in a distributed way have to define the direction of things.
Yeah. I mean, you’re making it sound really nice, but there, there is battle, there’s pretty heated battles inside open source projects. I mean, it is a very passionate community and you’re kind of in a distributed way have to define the direction of things.
Lex Fridman
So here looking at Perplexity, “FFmpeg and Libav split in 2011 mainly over project governance, leadership style, and development processes, not because of a fundamental technical disagreement. FFmpeg effectively absorbed Libav’s work while Libav withered and most distributions and developers moved back to FFmpeg.” Yeah, that was a weird experience ’cause, you know, I’m a Linux user, so, you know, whether it’s Ubuntu and so on, all of a sudden, I think for a little bit, Ubuntu, I feel like, am I remembering correctly, switched to Libav and-
So here looking at Perplexity, “FFmpeg and Libav split in 2011 mainly over project governance, leadership style, and development processes, not because of a fundamental technical disagreement. FFmpeg effectively absorbed Libav’s work while Libav withered and most distributions and developers moved back to FFmpeg.” Yeah, that was a weird experience ’cause, you know, I’m a Linux user, so, you know, whether it’s Ubuntu and so on, all of a sudden, I think for a little bit, Ubuntu, I feel like, am I remembering correctly, switched to Libav and-
Jean-Baptiste Kempf
12, 14, something like that. Yes. Something like that.
12, 14, something like that. Yes. Something like that.
Lex Fridman
And then they switched back to FFmpeg.
And then they switched back to FFmpeg.
Jean-Baptiste Kempf
They switched back, yeah.
They switched back, yeah.
Lex Fridman
I was like, “What is happening?” So on the sort of… you get to feel the ripple effects of the different internal debates that are happening.
I was like, “What is happening?” So on the sort of… you get to feel the ripple effects of the different internal debates that are happening.
Kieran Kunhya
To be fair, on Apple, when you type GCC, you get Clang. Like they, they did something like that as well, so.
To be fair, on Apple, when you type GCC, you get Clang. Like they, they did something like that as well, so.
Jean-Baptiste Kempf
Yeah. So to me it’s like the fork was like heated drama, but most of the development from Libav was merged back into FFmpeg, right? So de facto FFmpeg got a superset around Libav, and so that gave the user, because in the end we work for the users, a larger set of features and a ton of things that were discussed. For example, the debate on reviews, on how we push, are something that now is completely settled in FFmpeg and is following what mostly what everyone in the community agrees, right? So de facto, everyone who was active on Libav came back and worked on FFmpeg because the disagreements were fixed, and in the end, FFmpeg is stronger than it was before, right? And- … I know people love drama, but-
Yeah. So to me it’s like the fork was like heated drama, but most of the development from Libav was merged back into FFmpeg, right? So de facto FFmpeg got a superset around Libav, and so that gave the user, because in the end we work for the users, a larger set of features and a ton of things that were discussed. For example, the debate on reviews, on how we push, are something that now is completely settled in FFmpeg and is following what mostly what everyone in the community agrees, right? So de facto, everyone who was active on Libav came back and worked on FFmpeg because the disagreements were fixed, and in the end, FFmpeg is stronger than it was before, right? And- … I know people love drama, but-
Lex Fridman
Well, my main concern, I understand, and I think looking at the long history, it’s all for the good. But I do… I am concerned because there’s so few humans that are critical to the success of open source projects that I have seen it be a psychological toll on folks and, you know, sometimes leads to burnout. So you have these incredible people that are at the core of open source projects. There is a moment that happens ’cause, like, what is the motivation of doing it? Ultimately, it’s because you’re passionate about it and it makes you happy. Then at a certain point, you wake up and it’s like, “This has been a bit too much heat from the drama.” So, like, at the project level, the project continues and often flourishes. But sometimes there’s these individual humans that are just like-
Well, my main concern, I understand, and I think looking at the long history, it’s all for the good. But I do… I am concerned because there’s so few humans that are critical to the success of open source projects that I have seen it be a psychological toll on folks and, you know, sometimes leads to burnout. So you have these incredible people that are at the core of open source projects. There is a moment that happens ’cause, like, what is the motivation of doing it? Ultimately, it’s because you’re passionate about it and it makes you happy. Then at a certain point, you wake up and it’s like, “This has been a bit too much heat from the drama.” So, like, at the project level, the project continues and often flourishes. But sometimes there’s these individual humans that are just like-
Jean-Baptiste Kempf
But-
But-
Lex Fridman
… I’ve had enough.
… I’ve had enough.
Jean-Baptiste Kempf
Yeah, but it’s not just about forks, right? So it’s a very- … what you are referring to is one of the most challenging and most interesting part of open source today is maintainers burnout, right? And AI is a problem because of that. And Daniel Stenberg, who is the maintainer of curl, who’s probably one of the best promoters of open source in the world—he’s, by the way, a member of the European Open Source Academy with me, so I’m very, like, humbled to be in the same community as him, right? He’s against what he calls AI slop, right? Because it gives a ton of fake reports or- … bad reports, bad patches, and then a lot of maintainers have a lot of burden to maintain the software. And this is straining the mind of open source developers much more than forks.
Yeah, but it’s not just about forks, right? So it’s a very- … what you are referring to is one of the most challenging and most interesting part of open source today is maintainers burnout, right? And AI is a problem because of that. And Daniel Stenberg, who is the maintainer of curl, who’s probably one of the best promoters of open source in the world—he’s, by the way, a member of the European Open Source Academy with me, so I’m very, like, humbled to be in the same community as him, right? He’s against what he calls AI slop, right? Because it gives a ton of fake reports or- … bad reports, bad patches, and then a lot of maintainers have a lot of burden to maintain the software. And this is straining the mind of open source developers much more than forks.
Jean-Baptiste Kempf
And for example, the XZ fiasco was because there was one guy maintaining it, and he got basically hammered by two attackers who were asking him questions nonstop at weird times at night to block him, and at some point he got fed up and said, “Okay, I can’t do that,” and gave the commit access to the attacker. So burnout in the open source community is something that exists, but mostly it’s about maintaining things, right?
And for example, the XZ fiasco was because there was one guy maintaining it, and he got basically hammered by two attackers who were asking him questions nonstop at weird times at night to block him, and at some point he got fed up and said, “Okay, I can’t do that,” and gave the commit access to the attacker. So burnout in the open source community is something that exists, but mostly it’s about maintaining things, right?
Lex Fridman
No, for sure. But I wonder how do we help that, ’cause those people are so important.
No, for sure. But I wonder how do we help that, ’cause those people are so important.
Jean-Baptiste Kempf
Mm
Mm
Lex Fridman
… the human beings are so important to the core of these pro- projects.
… the human beings are so important to the core of these pro- projects.
Jean-Baptiste Kempf
So, so for example, now I am maintaining a ton of multimedia and non-multimedia libraries- …as maintainer because the maintainers got fed up, right? Some on VideoLAN, some outside of VideoLAN, because it’s… sometimes you need tough skin, right? Because you get, like, it’s not really attacks, but “oh, this is not working, this is not working,” and you feel it personally. And this is also why resources or the Google fiasco was a problem, right? They don’t realize that in the end you have, you know, it’s like the same graph where you see, like, everything and it’s just like one random open source project that is maintaining the whole—
So, so for example, now I am maintaining a ton of multimedia and non-multimedia libraries- …as maintainer because the maintainers got fed up, right? Some on VideoLAN, some outside of VideoLAN, because it’s… sometimes you need tough skin, right? Because you get, like, it’s not really attacks, but “oh, this is not working, this is not working,” and you feel it personally. And this is also why resources or the Google fiasco was a problem, right? They don’t realize that in the end you have, you know, it’s like the same graph where you see, like, everything and it’s just like one random open source project that is maintaining the whole—
Lex Fridman
The Nebraska thing, yeah
The Nebraska thing, yeah
Jean-Baptiste Kempf
… internet. You see the one, right? The-
… internet. You see the one, right? The-
Lex Fridman
Yeah, this is the meme. I mean, it applies to, to a lot of open source projects.
Yeah, this is the meme. I mean, it applies to, to a lot of open source projects.
Jean-Baptiste Kempf
So many things, yeah.
So many things, yeah.
Lex Fridman
But this is the all modern digital multimedia infrastructure, and then that thing at the very bottom that everything relies on is FFmpeg. It’s true. And then there’s usually, you know, a handful of folks that are maintaining that.
But this is the all modern digital multimedia infrastructure, and then that thing at the very bottom that everything relies on is FFmpeg. It’s true. And then there’s usually, you know, a handful of folks that are maintaining that.
Jean-Baptiste Kempf
And FFmpeg or VLC, right, you have a community of 10, 15 core developers, are not the worst open source project. XZ, which is even in more installations, is one person, right? There is one guy-
And FFmpeg or VLC, right, you have a community of 10, 15 core developers, are not the worst open source project. XZ, which is even in more installations, is one person, right? There is one guy-
Lex Fridman
LibXML is-
LibXML is-
Jean-Baptiste Kempf
Yeah, LibXML, right? There was a big stop. No one is maintaining- … LibXML anymore, which is like the only library that is able to parse XML everywhere.
Yeah, LibXML, right? There was a big stop. No one is maintaining- … LibXML anymore, which is like the only library that is able to parse XML everywhere.
Lex Fridman
All the crazy edge cases of XML under ridiculous circumstances, and they get attacked by security researchers because there’s one other crazy edge case that they haven’t thought of, and it’s like, yeah, but the body of knowledge to actually resolve that is massive.
All the crazy edge cases of XML under ridiculous circumstances, and they get attacked by security researchers because there’s one other crazy edge case that they haven’t thought of, and it’s like, yeah, but the body of knowledge to actually resolve that is massive.
Jean-Baptiste Kempf
There is one guy maintaining all the time zones for everyone who is in the middle of, I think, was it Nebraska or-
There is one guy maintaining all the time zones for everyone who is in the middle of, I think, was it Nebraska or-
Open source burnout
Lex Fridman
Yeah, it could be, yeah.
Yeah, it could be, yeah.
Jean-Baptiste Kempf
… South Dakota? Like, the mental health of the open source maintainers is something that large corporations don’t care or don’t see, right? It’s just like, “Oh, yeah, I’m just doing an open source report,” and so on.
… South Dakota? Like, the mental health of the open source maintainers is something that large corporations don’t care or don’t see, right? It’s just like, “Oh, yeah, I’m just doing an open source report,” and so on.
Lex Fridman
Mm. Some of it is financial, but people should definitely support open source financially- … all, all across the board. But some of it is also, like, spiritual on a basic human level. There’s something that happens, like, with this image of F- FFmpeg and so much of the internet depending on it, where people almost, like, talk down to the folks who are carrying these projects forward and maintaining it.
Mm. Some of it is financial, but people should definitely support open source financially- … all, all across the board. But some of it is also, like, spiritual on a basic human level. There’s something that happens, like, with this image of F- FFmpeg and so much of the internet depending on it, where people almost, like, talk down to the folks who are carrying these projects forward and maintaining it.
Jean-Baptiste Kempf
In the security community, they certainly did. That was one of the things I think that argument came out is there was a portion of the security community who’s like, “No, these guys write crap code. They need to fix their crap code.” I’m like, “No, no, no, no. This is a guy’s hobby project. You have a security bot that’s gone and found some AI-generated stuff. That guy didn’t write crap code.” It’s just an edge case to the 99.99999 percentile he didn’t think about because it’s his hobby project decoding Star Wars games.”
In the security community, they certainly did. That was one of the things I think that argument came out is there was a portion of the security community who’s like, “No, these guys write crap code. They need to fix their crap code.” I’m like, “No, no, no, no. This is a guy’s hobby project. You have a security bot that’s gone and found some AI-generated stuff. That guy didn’t write crap code.” It’s just an edge case to the 99.99999 percentile he didn’t think about because it’s his hobby project decoding Star Wars games.”
Lex Fridman
Forget the hobby project aspect of it. It’s just hard work, and it’s beautiful, and it’s like the right approach there is to celebrate people- … for doing incredible, incredible work. It’s, it’s just incredible that humans step up- … not getting really paid at first or maybe ever, and then they’re doing it out of the love of it, and we need to, like, human civilization runs on people like that. We need to celebrate them.
Forget the hobby project aspect of it. It’s just hard work, and it’s beautiful, and it’s like the right approach there is to celebrate people- … for doing incredible, incredible work. It’s, it’s just incredible that humans step up- … not getting really paid at first or maybe ever, and then they’re doing it out of the love of it, and we need to, like, human civilization runs on people like that. We need to celebrate them.
Jean-Baptiste Kempf
To give you an idea, I received death threats on VideoLAN, right? And, um-
To give you an idea, I received death threats on VideoLAN, right? And, um-
Lex Fridman
You mentioned that to me. Like, what, what is, what is behind that?
You mentioned that to me. Like, what, what is, what is behind that?
Jean-Baptiste Kempf
So that must be, what, 2009, 2010, right? Um, Apple is moving from PowerPC to Core Duo, um, that probably in 2006, and by 2009 or 2010, I decide that we are not going to do new versions of VLC for PowerPC. At that time, like VLC, we were close to the number 1.0 release. We were four of us, right? Like, just like, “No, this is not possible.” So I receive a death threat with some powder in it, right? It– Remember there was some- … anthrax threats-
So that must be, what, 2009, 2010, right? Um, Apple is moving from PowerPC to Core Duo, um, that probably in 2006, and by 2009 or 2010, I decide that we are not going to do new versions of VLC for PowerPC. At that time, like VLC, we were close to the number 1.0 release. We were four of us, right? Like, just like, “No, this is not possible.” So I receive a death threat with some powder in it, right? It– Remember there was some- … anthrax threats-
Jean-Baptiste Kempf
… at that time, right? And it was because I had taken the decision to not maintain the PowerPC port anymore. And of course, it wasn’t anthrax, of course. It was some type of flour and so on. But I received that as a, with a letter of like, “You, you piece of shit, you should die, PowerPC forever,” and so on. And it was 2009 or 2010, right? I was young. I was just like, “Why? What did I do?”, right?
… at that time, right? And it was because I had taken the decision to not maintain the PowerPC port anymore. And of course, it wasn’t anthrax, of course. It was some type of flour and so on. But I received that as a, with a letter of like, “You, you piece of shit, you should die, PowerPC forever,” and so on. And it was 2009 or 2010, right? I was young. I was just like, “Why? What did I do?”, right?
Lex Fridman
Yeah, that can break your spirit. It’s like, why-
Yeah, that can break your spirit. It’s like, why-
Jean-Baptiste Kempf
My mother freaked out, right? We had to go to see the police and so on. And now, like, I’m going to say that I’m quite happy that this happened at that time. It forged me a lot, right? I am… I can see, I can take a lot of hate on me. I’m okay with it, right?
My mother freaked out, right? We had to go to see the police and so on. And now, like, I’m going to say that I’m quite happy that this happened at that time. It forged me a lot, right? I am… I can see, I can take a lot of hate on me. I’m okay with it, right?
Lex Fridman
It sucks that that’s part of reality, ’cause all the people that love VLC, all the people that love FFmpeg, like me, you know, I, I legitimately hundred- probably thousands of times in my life had a smile on my face because FFmpeg made me happy, period. And how many times did I get a chance to say that? Zero. Until I realized there’s a Twitter account, and every once in a while I’m, like, messaging it.
It sucks that that’s part of reality, ’cause all the people that love VLC, all the people that love FFmpeg, like me, you know, I, I legitimately hundred- probably thousands of times in my life had a smile on my face because FFmpeg made me happy, period. And how many times did I get a chance to say that? Zero. Until I realized there’s a Twitter account, and every once in a while I’m, like, messaging it.
Jean-Baptiste Kempf
One of the things I like on the Reddit meme about me, which I don’t like this meme for a lot of reasons, but… And someone says, “Oh, JB is on Reddit,” which I am, right? And I say, and say hello, right? And then I got so many people who say, “Oh, thank you for VLC.” And, like, I take pictures, and then I share that to the Signal, to IRC. Yes, we use IRC on different-
One of the things I like on the Reddit meme about me, which I don’t like this meme for a lot of reasons, but… And someone says, “Oh, JB is on Reddit,” which I am, right? And I say, and say hello, right? And then I got so many people who say, “Oh, thank you for VLC.” And, like, I take pictures, and then I share that to the Signal, to IRC. Yes, we use IRC on different-
Lex Fridman
I saw as a quick tangent, you mentioned IRC is like Slack for old people. So you still use IRC?
I saw as a quick tangent, you mentioned IRC is like Slack for old people. So you still use IRC?
Jean-Baptiste Kempf
Of course.
Of course.
Kieran Kunhya
Yeah. I have it on my phone as well.
Yeah. I have it on my phone as well.
Jean-Baptiste Kempf
Of course.
Of course.
Kieran Kunhya
Every day.
Every day.
Jean-Baptiste Kempf
Works fine.
Works fine.
Lex Fridman
Wow. It works fine, huh?
Wow. It works fine, huh?
Jean-Baptiste Kempf
Works fine, yes.
Works fine, yes.
Lex Fridman
You have to power with a crank, I guess.
You have to power with a crank, I guess.
Kieran Kunhya
No, but there’s no-
No, but there’s no-
Lex Fridman
There’s AOL. There’s AOL as your social media.
There’s AOL. There’s AOL as your social media.
Kieran Kunhya
There’s no ads, there’s no tracking, there’s nothing. Like, it’s-
There’s no ads, there’s no tracking, there’s nothing. Like, it’s-
Jean-Baptiste Kempf
The biggest issue, to be honest, compared to Slack is that it doesn’t have threads. That’s annoying. It doesn’t have emojis for reaction. Sometimes it would be nice.
The biggest issue, to be honest, compared to Slack is that it doesn’t have threads. That’s annoying. It doesn’t have emojis for reaction. Sometimes it would be nice.
Kieran Kunhya
I, IRCv3 has.
I, IRCv3 has.
Jean-Baptiste Kempf
Yes, V3, but no one does it, and you cannot edit your messages. Right? And the rest, it works perfectly fine forever.
Yes, V3, but no one does it, and you cannot edit your messages. Right? And the rest, it works perfectly fine forever.
Lex Fridman
But how do you communicate without emojis?
But how do you communicate without emojis?
Jean-Baptiste Kempf
Well, that’s why I said it’s for old people.
Well, that’s why I said it’s for old people.
Lex Fridman
Old people. All right.
Old people. All right.
Jean-Baptiste Kempf
And we do, we do emojis with like- … you know, the colons and dash and-
And we do, we do emojis with like- … you know, the colons and dash and-
Lex Fridman
Yeah, exactly.
Yeah, exactly.
Jean-Baptiste Kempf
… parentheses, right? So.
… parentheses, right? So.
Lex Fridman
Old school. So anyway, you communicate on IRC. What were you even talking about?
Old school. So anyway, you communicate on IRC. What were you even talking about?
Jean-Baptiste Kempf
Yeah, we are talking about death threats and, and-
Yeah, we are talking about death threats and, and-
Lex Fridman
Oh, damn.
Oh, damn.
Jean-Baptiste Kempf
… but having people thanking you, and sometimes- I get people who send me a message and, and, “Oh, thank you for VLC.” And I always answer because I want to validate the fact that you need to thank the open source community.
… but having people thanking you, and sometimes- I get people who send me a message and, and, “Oh, thank you for VLC.” And I always answer because I want to validate the fact that you need to thank the open source community.
Lex Fridman
Yeah, please, everybody listening to this, celebrate, celebrate FFmpeg, celebrate VLC, celebrate all the incredible open source projects, Linux, everything. There’s so many, there’s so many… And you know what? I mean, even outside of open source, just celebrate companies that create software that you use a lot and love.
Yeah, please, everybody listening to this, celebrate, celebrate FFmpeg, celebrate VLC, celebrate all the incredible open source projects, Linux, everything. There’s so many, there’s so many… And you know what? I mean, even outside of open source, just celebrate companies that create software that you use a lot and love.
Kieran Kunhya
Celebrate human endeavor. Celebrate the human effort to not just build something that’s okay- … build something that is damn good.
Celebrate human endeavor. Celebrate the human effort to not just build something that’s okay- … build something that is damn good.
Jean-Baptiste Kempf
Yes, this is important, right? Because as we said, right, we work for technol- we do something very complex for the normal people. Like, we want our excellence in tech to be useful for everyone. And this is why, like, this is why we work, right? This is why I wake up in the morning is because I want people to use our stuff- … Because it’s making everyone’s life easier.
Yes, this is important, right? Because as we said, right, we work for technol- we do something very complex for the normal people. Like, we want our excellence in tech to be useful for everyone. And this is why, like, this is why we work, right? This is why I wake up in the morning is because I want people to use our stuff- … Because it’s making everyone’s life easier.
Kieran Kunhya
Want to solve hard problems. Work on something interesting, work on some interesting technical challenges.
Want to solve hard problems. Work on something interesting, work on some interesting technical challenges.
Jean-Baptiste Kempf
As we are engineers, we love to build things, right? When I was young, like very early, I knew I wanted to build, to be an engineer. I wanted to do cars, right? Maybe at some point I will go back to cars, right? But this is like we want to build things that are cool and useful. And they need to be challenging, right? Because you want your brain to turn on.
As we are engineers, we love to build things, right? When I was young, like very early, I knew I wanted to build, to be an engineer. I wanted to do cars, right? Maybe at some point I will go back to cars, right? But this is like we want to build things that are cool and useful. And they need to be challenging, right? Because you want your brain to turn on.
Lex Fridman
When did the two of you first fall in love with programming, with building, with engineering?
When did the two of you first fall in love with programming, with building, with engineering?
Jean-Baptiste Kempf
When is the first time you programmed, Kieran?
When is the first time you programmed, Kieran?
Kieran Kunhya
Microsoft QBasic. As I was on Windows 3.1 and Windows 95, Microsoft QBasic.
Microsoft QBasic. As I was on Windows 3.1 and Windows 95, Microsoft QBasic.
Lex Fridman
Oh, wow. Wow. What’d you build?
Oh, wow. Wow. What’d you build?
Kieran Kunhya
Like a multiplication, just counting loops like 10, 20, 30, 40.
Like a multiplication, just counting loops like 10, 20, 30, 40.
Lex Fridman
Nice.
Nice.
Kieran Kunhya
Then I thought I could do everything after that. I wanted… I jumped from doing that to I want to create a soccer, no, a football, soccer video game. And I drew all the, I drew everything out. I was like, “I’m gonna do it.” And I didn’t quite grasp that actually it’s a massive piece of work to jump from BASIC and drawing some pictures to a video game, but there we go.
Then I thought I could do everything after that. I wanted… I jumped from doing that to I want to create a soccer, no, a football, soccer video game. And I drew all the, I drew everything out. I was like, “I’m gonna do it.” And I didn’t quite grasp that actually it’s a massive piece of work to jump from BASIC and drawing some pictures to a video game, but there we go.
Jean-Baptiste Kempf
Yeah. I think I did also Basics and then Turbo Pascal when I was, yeah, end of elementary school. But mostly the first time I actually did some serious programming was the, the first year of you call that middle school when you’re 11? I was living in Italy for a year in Florence and it was amazing year. And like the maths teacher told us to, to work in a programming language called Logo, where you had a turtle that was designing things- … On the screen, and you would turn left and right. And in the end, we used that to do a very complex programming because of course you could do things. And, and this changed, like, as I knew I was, I wanted to do things with computers and program.
Yeah. I think I did also Basics and then Turbo Pascal when I was, yeah, end of elementary school. But mostly the first time I actually did some serious programming was the, the first year of you call that middle school when you’re 11? I was living in Italy for a year in Florence and it was amazing year. And like the maths teacher told us to, to work in a programming language called Logo, where you had a turtle that was designing things- … On the screen, and you would turn left and right. And in the end, we used that to do a very complex programming because of course you could do things. And, and this changed, like, as I knew I was, I wanted to do things with computers and program.
x264 and internet video
Lex Fridman
I don’t think we quite talked about H.264 properly. We talked about David. Can we return-
I don’t think we quite talked about H.264 properly. We talked about David. Can we return-
Kieran Kunhya
Sure
Sure
Lex Fridman
… backtrack a little bit to H.264, this thing that powers basically all of the video on the internet? So can you tell me the story of H.264? And Kieran, you’re actually a contributor- … to H.264.
… backtrack a little bit to H.264, this thing that powers basically all of the video on the internet? So can you tell me the story of H.264? And Kieran, you’re actually a contributor- … to H.264.
Kieran Kunhya
So, so H.264 is a video encoder for the H.264 video standard. It dominates internet video, but also other areas such as Blu-ray discs. And Blu-ray discs are interesting because the people that make them really want the highest quality, and there’s some really cool high-end films that have been encoded broadcasting and all sorts of other areas. H.264 was a big step change ’cause it kinda happened at the right time as well. A lot of the development took place when HD video was coming out. Intel Core 2 and Nehalem CPUs were getting fast. You could do real-time video. But the most important thing was a key sort of focus on visual metrics.
So, so H.264 is a video encoder for the H.264 video standard. It dominates internet video, but also other areas such as Blu-ray discs. And Blu-ray discs are interesting because the people that make them really want the highest quality, and there’s some really cool high-end films that have been encoded broadcasting and all sorts of other areas. H.264 was a big step change ’cause it kinda happened at the right time as well. A lot of the development took place when HD video was coming out. Intel Core 2 and Nehalem CPUs were getting fast. You could do real-time video. But the most important thing was a key sort of focus on visual metrics.
Kieran Kunhya
So industry and academia for 20 years before, was obsessed with mathematical metrics or what’s known as peak signal-to-noise ratio. So mean squared error, logarithm of mean squared error, and that led to tons of issues because mean squared error leads to blurring because you actually want to minimize– You want to add a little bit of error to everything to reduce the mean squared error as opposed to having a big error, and that led to loads and loads of blurring. So hobbyists bucked that trend. It was for their own personal videos, mostly anime. So there were two things they did differently, and there was a big iterative feedback loop with the community. They did some stuff differently.
So industry and academia for 20 years before, was obsessed with mathematical metrics or what’s known as peak signal-to-noise ratio. So mean squared error, logarithm of mean squared error, and that led to tons of issues because mean squared error leads to blurring because you actually want to minimize– You want to add a little bit of error to everything to reduce the mean squared error as opposed to having a big error, and that led to loads and loads of blurring. So hobbyists bucked that trend. It was for their own personal videos, mostly anime. So there were two things they did differently, and there was a big iterative feedback loop with the community. They did some stuff differently.
Kieran Kunhya
Two big things: psychovisual rate distortion, so using block energy, trying to compensate for human perception when making decisions.
Two big things: psychovisual rate distortion, so using block energy, trying to compensate for human perception when making decisions.
Lex Fridman
So the psychovisual distortion, that’s the critical- … thing. That’s the thing. I mean, it’s kind of revolutionary, like, that we can, like, rethink. Don’t make it, like, this kind of theoretic thing- … of compression. Make it all about-
So the psychovisual distortion, that’s the critical- … thing. That’s the thing. I mean, it’s kind of revolutionary, like, that we can, like, rethink. Don’t make it, like, this kind of theoretic thing- … of compression. Make it all about-
Kieran Kunhya
Being pleasing visually to the eye.
Being pleasing visually to the eye.
Lex Fridman
Yeah, yeah. So compressing in a way that loses the least amount of information for the stuff that matters for us humans.
Yeah, yeah. So compressing in a way that loses the least amount of information for the stuff that matters for us humans.
Kieran Kunhya
Yes, exactly. As opposed to what industry– Some parts of industry are still obsessed by this, which is mathematical numbers that don’t look good in reality. And then adaptive quantization was the other big one where it was biasing bits against complex areas and redistributing them to less complex areas like grass. Grass has some high frequencies, but it’s kind of– it’s less complex overall compared to more complicated things. And this came around by ParkJoy. So ParkJoy was really the canonical sample that was… Is the running around in the park.
Yes, exactly. As opposed to what industry– Some parts of industry are still obsessed by this, which is mathematical numbers that don’t look good in reality. And then adaptive quantization was the other big one where it was biasing bits against complex areas and redistributing them to less complex areas like grass. Grass has some high frequencies, but it’s kind of– it’s less complex overall compared to more complicated things. And this came around by ParkJoy. So ParkJoy was really the canonical sample that was… Is the running around in the park.
Lex Fridman
This one.
This one.
Kieran Kunhya
Yeah. So this guy was really the- So this, this was created by Swedish television in the beginning of HD, and it was done on film, and it was no expense spared in terms of production quality, and it was given away for free. This was really– And this is the sample really that sorts the men from the boys in terms of it has so many challenges with the trees, with the water, with the grass, with the motion, with the… I don’t think there’s, there’s still been any, any public test sequence as good as that these days.
Yeah. So this guy was really the- So this, this was created by Swedish television in the beginning of HD, and it was done on film, and it was no expense spared in terms of production quality, and it was given away for free. This was really– And this is the sample really that sorts the men from the boys in terms of it has so many challenges with the trees, with the water, with the grass, with the motion, with the… I don’t think there’s, there’s still been any, any public test sequence as good as that these days.
Lex Fridman
So for people who are just listening, we’re looking at a bunch of humans running- … along a river, as you have the reflection, a lot of really high information textures everywhere, the leaves and the lighting playing with the leaves and all of this.
So for people who are just listening, we’re looking at a bunch of humans running- … along a river, as you have the reflection, a lot of really high information textures everywhere, the leaves and the lighting playing with the leaves and all of this.
Kieran Kunhya
You could show clearly that encoders with high PSNR-
You could show clearly that encoders with high PSNR-
Lex Fridman
Will blur everything
Will blur everything
Kieran Kunhya
… will blur everything, and you could see actually I could turn on psychovisual stuff, I could turn on adaptive quantization, and it would just look so much better. But your metrics– And these metrics at the time were considered so holy. These are the holy metrics that are untouchable. PSNR is the most important thing.
… will blur everything, and you could see actually I could turn on psychovisual stuff, I could turn on adaptive quantization, and it would just look so much better. But your metrics– And these metrics at the time were considered so holy. These are the holy metrics that are untouchable. PSNR is the most important thing.
Lex Fridman
Can you speak to how do you measure psychovisual stuff? Like, how do you turn how pleasing a compression is for a human eye- … into a number? Is that even possible?
Can you speak to how do you measure psychovisual stuff? Like, how do you turn how pleasing a compression is for a human eye- … into a number? Is that even possible?
Kieran Kunhya
That’s what, that’s what Netflix has been trying to do with VMAF. They said they’ve used a machine learning model.
That’s what, that’s what Netflix has been trying to do with VMAF. They said they’ve used a machine learning model.
Lex Fridman
That’s a more recent thing. But back when x264 was being developed, that’s by eye you’re basically-
That’s a more recent thing. But back when x264 was being developed, that’s by eye you’re basically-
Kieran Kunhya
It was by eye. It was developers on their laptops. So it’s not like even with big companies with professional screens or anything, it’s- And that was actually one of the goals, which was I don’t– The, the developers at the time, Loren Merritt in particular, it’s, “I don’t wanna test this on a thirty thousand dollar screen. It’s– I want this to look good on someone’s laptop at home.”
It was by eye. It was developers on their laptops. So it’s not like even with big companies with professional screens or anything, it’s- And that was actually one of the goals, which was I don’t– The, the developers at the time, Loren Merritt in particular, it’s, “I don’t wanna test this on a thirty thousand dollar screen. It’s– I want this to look good on someone’s laptop at home.”
Lex Fridman
Yeah. Brilliant.
Yeah. Brilliant.
Jean-Baptiste Kempf
There is another sample which is, um- … A sample that is Planet Earth’s killer sample that I absolutely love. And you are going to see why, right?
There is another sample which is, um- … A sample that is Planet Earth’s killer sample that I absolutely love. And you are going to see why, right?
Kieran Kunhya
Yeah, you’re going to love this.
Yeah, you’re going to love this.
Jean-Baptiste Kempf
The- it’s a ton of birds, right, flying, and the more it goes, the more there are birds, and at the end, right, it’s almost like you have millions of birds. It’s the most complex thing ever to encode, right? And well, you’re watching it on YouTube, and you see how bad the YouTube encoding is actually, right? And this is, like, phenomenal to optimize and get perfect quality in a constant bit rate. There was a lot of optimization, mostly by Loren also, um- on anime, right? For a long time, anime was very badly encoded because there was a ton of banding, right? And so you see those issues, and there was a ton of things. So x264 is, like– And today it’s still the reference to any encoder, new encoder, AV1, AV2, VVC, HEVC; everyone compares to x264.
The- it’s a ton of birds, right, flying, and the more it goes, the more there are birds, and at the end, right, it’s almost like you have millions of birds. It’s the most complex thing ever to encode, right? And well, you’re watching it on YouTube, and you see how bad the YouTube encoding is actually, right? And this is, like, phenomenal to optimize and get perfect quality in a constant bit rate. There was a lot of optimization, mostly by Loren also, um- on anime, right? For a long time, anime was very badly encoded because there was a ton of banding, right? And so you see those issues, and there was a ton of things. So x264 is, like– And today it’s still the reference to any encoder, new encoder, AV1, AV2, VVC, HEVC; everyone compares to x264.
Kieran Kunhya
One of my favorite films, Cinema Paradiso, I know the engineer who created the Blu-ray, and he showed me the comparisons of x264 versus others, and the… it’s completely different. And I think a bunch, a bunch of guys in the Blu-ray world started using x264. Um, I think the big one was Chris Henderson from Warner Bros. He did the whole Fringe box set with that. So quite, like, a thing a person on the street actually watches and wants to look good. And so they kind of took a risk in their jobs doing that because they’re in a big company. That big company can buy whatever they want.
One of my favorite films, Cinema Paradiso, I know the engineer who created the Blu-ray, and he showed me the comparisons of x264 versus others, and the… it’s completely different. And I think a bunch, a bunch of guys in the Blu-ray world started using x264. Um, I think the big one was Chris Henderson from Warner Bros. He did the whole Fringe box set with that. So quite, like, a thing a person on the street actually watches and wants to look good. And so they kind of took a risk in their jobs doing that because they’re in a big company. That big company can buy whatever they want.
Kieran Kunhya
And they said, “No, no, no, I want to use this free and open source thing so that things look good for my, my customers and build the best.” And to this day, I personally still try and avoid watching the most cinematic films on streaming services and buy the physical discs because they look, they look good without even having to buy an expensive TV. I think that’s the key thing.
And they said, “No, no, no, I want to use this free and open source thing so that things look good for my, my customers and build the best.” And to this day, I personally still try and avoid watching the most cinematic films on streaming services and buy the physical discs because they look, they look good without even having to buy an expensive TV. I think that’s the key thing.
Jean-Baptiste Kempf
And x264 is yet another example of open source project. It was started by Laurent Aimar when he was at École Centrale Paris, where VLC was born. And then you got a generation of people like Loren, like Jason, like Måns, like so many-
And x264 is yet another example of open source project. It was started by Laurent Aimar when he was at École Centrale Paris, where VLC was born. And then you got a generation of people like Loren, like Jason, like Måns, like so many-
Kieran Kunhya
Henrik from-
Henrik from-
Jean-Baptiste Kempf
Henrik,
Henrik,
Kieran Kunhya
Anton,
Anton,
Jean-Baptiste Kempf
and this is– Anton, and this is where the assembly thing that we use now on FFmpeg, David and so on, was born, right? So x264 is, like, an amazing project with people who were really all over the world, and I think most of them never met each other.
and this is– Anton, and this is where the assembly thing that we use now on FFmpeg, David and so on, was born, right? So x264 is, like, an amazing project with people who were really all over the world, and I think most of them never met each other.
Lex Fridman
But all of them, according to Kieran, or a large percentage, love anime. There’s several things I’ve never got into, and one of them is anime, and I need to
But all of them, according to Kieran, or a large percentage, love anime. There’s several things I’ve never got into, and one of them is anime, and I need to
Jean-Baptiste Kempf
I watch anime so much, especially at the time. Like, at the time, it was like a lot of anime content doesn’t exist commercially, right? We are before Crunchyroll, right? So what happens is usually people who love anime, who take some things, some DVDs in Japan and rip them because there is no commercial offering. And-
I watch anime so much, especially at the time. Like, at the time, it was like a lot of anime content doesn’t exist commercially, right? We are before Crunchyroll, right? So what happens is usually people who love anime, who take some things, some DVDs in Japan and rip them because there is no commercial offering. And-
Jean-Baptiste Kempf
some of the people who are, what we call fansubbers, are basically translating themselves to make subtitles, right? And at that time, you download completely illegally. It was the only way to do that, right? And so all of that was handcrafted, and it fits the open source community, right? Because they needed tools to encode, to do fansubbing, right? One of the most amazing open source projects for subtitles is called Aegisub, and it’s, it’s a subtitle… It’s done for anime, for, for, for South Asian and Japanese languages.
some of the people who are, what we call fansubbers, are basically translating themselves to make subtitles, right? And at that time, you download completely illegally. It was the only way to do that, right? And so all of that was handcrafted, and it fits the open source community, right? Because they needed tools to encode, to do fansubbing, right? One of the most amazing open source projects for subtitles is called Aegisub, and it’s, it’s a subtitle… It’s done for anime, for, for, for South Asian and Japanese languages.
Kieran Kunhya
There are weird textures in anime that I don’t think you get in real life content. I think that was a key one, which was optimizing these weird textures that you get- … because anime is not done in a normal fashion.
There are weird textures in anime that I don’t think you get in real life content. I think that was a key one, which was optimizing these weird textures that you get- … because anime is not done in a normal fashion.
Jean-Baptiste Kempf
Yeah. The way you produce it is not– You, you mostly produce it, like, on screens, right? Since a bit of time, and you have all those gradients, right, in colors? Because they are very easy to produce digitally, very complex to produce in real life. And, and the subtitles also are very complex because you need to have often the Japanese and then you need to have the diacritics, right? The what we call the ruby, right? Which is the hiragana and the katakana for the kanji. And then because of course you, so that you have the official subtitling, but you also need the English subtitles or the French subtitles because you want to learn that, right?
Yeah. The way you produce it is not– You, you mostly produce it, like, on screens, right? Since a bit of time, and you have all those gradients, right, in colors? Because they are very easy to produce digitally, very complex to produce in real life. And, and the subtitles also are very complex because you need to have often the Japanese and then you need to have the diacritics, right? The what we call the ruby, right? Which is the hiragana and the katakana for the kanji. And then because of course you, so that you have the official subtitling, but you also need the English subtitles or the French subtitles because you want to learn that, right?
Jean-Baptiste Kempf
And there is so many things crazy on, on subtitles and we had like crazy samples on, on subtitles that we’ve seen all around. So this is an important part of the, the culture, but also because there was no official offering. There was no way of doing that.
And there is so many things crazy on, on subtitles and we had like crazy samples on, on subtitles that we’ve seen all around. So this is an important part of the, the culture, but also because there was no official offering. There was no way of doing that.
Lex Fridman
Can you speak to the difference between H.264 and AV1 and then x264 and dav1d? This is this big step. Can you help people understand, are some of the streaming sites moving more towards that direction of AV1?
Can you speak to the difference between H.264 and AV1 and then x264 and dav1d? This is this big step. Can you help people understand, are some of the streaming sites moving more towards that direction of AV1?
Jean-Baptiste Kempf
Let’s be honest, all of those codecs since MPEG-2 video are the same concepts. The same concept about inverse transform, about intra prediction, motion compensation, entropy coding, all of them. However, each generation gives you a bump between twenty-five and fifty percent more compression for the same quality. And so you had the MPEG-2, you had the DivX era, you have H.264, which was, like, changing, right? H.264 improved so much. And then you had more, right? You had HEVC. You had VP9 at the same time of HEVC. VP9 is a bit similar to HEVC in terms of quality compression, but it’s royalty-free. Because in multimedia there is ton of patents and the licensing after H.264 became out of hand, right? And could cost hundreds of millions of dollars per year. So it made no sense.
Let’s be honest, all of those codecs since MPEG-2 video are the same concepts. The same concept about inverse transform, about intra prediction, motion compensation, entropy coding, all of them. However, each generation gives you a bump between twenty-five and fifty percent more compression for the same quality. And so you had the MPEG-2, you had the DivX era, you have H.264, which was, like, changing, right? H.264 improved so much. And then you had more, right? You had HEVC. You had VP9 at the same time of HEVC. VP9 is a bit similar to HEVC in terms of quality compression, but it’s royalty-free. Because in multimedia there is ton of patents and the licensing after H.264 became out of hand, right? And could cost hundreds of millions of dollars per year. So it made no sense.
Jean-Baptiste Kempf
So Google did this VP9 and the Alliance for Open Media did this new codec called AV1. So you can imagine that AV1 saves between forty and sixty percent less bandwidth than H.264- … for the same quality, visual quality.
So Google did this VP9 and the Alliance for Open Media did this new codec called AV1. So you can imagine that AV1 saves between forty and sixty percent less bandwidth than H.264- … for the same quality, visual quality.
Kieran Kunhya
At a given bitrate.
At a given bitrate.
Jean-Baptiste Kempf
At a given bitrate, right? So that’s really like you increase the quality– either you set the bitrate and you increase the quality, or you set the quality and you decrease your bitrate. But because now you move from SD to HD and HD to 4K and 4K to 4K HDR, like you increasing the size by like factor two, three, four, right? So you need to have better compression to keep it in terms of something that is manageable.
At a given bitrate, right? So that’s really like you increase the quality– either you set the bitrate and you increase the quality, or you set the quality and you decrease your bitrate. But because now you move from SD to HD and HD to 4K and 4K to 4K HDR, like you increasing the size by like factor two, three, four, right? So you need to have better compression to keep it in terms of something that is manageable.
Kieran Kunhya
It’s more coding tools, more bigger blocks, lots more sub-partitions in each block. It’s just exponentially more complex.
It’s more coding tools, more bigger blocks, lots more sub-partitions in each block. It’s just exponentially more complex.
Jean-Baptiste Kempf
It’s more complex because the encoder needs to search more possibilities, right? So you, for example one of the things that is easy to understand is to predict a block, a color block to another, you have directions, right? So you can go left, right, bottom, up, and then in terms of like the other quadrants, right? What I call north, northeast, northwest, and so on, right? But that’s eight directions. Then you can do more direction. You can do sixteen or sixty-nine or one hundred and twenty-eight, right? You can– And every time your encoder is going to spend more time to see, oh, well, this block is exactly this one and those type of tools that you can bring, and the encoder needs to check which of the tools are going to compress you better.
It’s more complex because the encoder needs to search more possibilities, right? So you, for example one of the things that is easy to understand is to predict a block, a color block to another, you have directions, right? So you can go left, right, bottom, up, and then in terms of like the other quadrants, right? What I call north, northeast, northwest, and so on, right? But that’s eight directions. Then you can do more direction. You can do sixteen or sixty-nine or one hundred and twenty-eight, right? You can– And every time your encoder is going to spend more time to see, oh, well, this block is exactly this one and those type of tools that you can bring, and the encoder needs to check which of the tools are going to compress you better.
Jean-Baptiste Kempf
And so I guess that AV1 encoding is two orders of magnitude more than H.264 in terms of CPU cycles, right? Order of magnitude, right?
And so I guess that AV1 encoding is two orders of magnitude more than H.264 in terms of CPU cycles, right? Order of magnitude, right?
Kieran Kunhya
Yeah. And as we discussed, CPUs are not getting faster. You’re just throwing more cores at the problem.
Yeah. And as we discussed, CPUs are not getting faster. You’re just throwing more cores at the problem.
Jean-Baptiste Kempf
But also it’s a fact that you encode once and you have hundreds of millions of users, right? So for example, YouTube, a very good example. YouTube encodes almost everything in H.264, but the popular video gets re-encoded in AV1 because it costs more, of course, to encode, but you encode once and you send that to millions, right? So it’s a trade-off between encoding time and complexity- … and CPU usage on the server side and on the client side. Because at the end, if you’re distributing a video to hundreds of thousands of people and the size is half of the other, then it’s better. It’s better for your battery, it’s better for your modem, et cetera, et cetera.
But also it’s a fact that you encode once and you have hundreds of millions of users, right? So for example, YouTube, a very good example. YouTube encodes almost everything in H.264, but the popular video gets re-encoded in AV1 because it costs more, of course, to encode, but you encode once and you send that to millions, right? So it’s a trade-off between encoding time and complexity- … and CPU usage on the server side and on the client side. Because at the end, if you’re distributing a video to hundreds of thousands of people and the size is half of the other, then it’s better. It’s better for your battery, it’s better for your modem, et cetera, et cetera.
Video compression basics
Lex Fridman
So we can lay out, let’s say, the top five codec-container combos would be H.264 inside MP4 containers, AV1 inside MP4/WebM containers, ProRes for nonlinear editing inside MOV containers. So for people who don’t know, I guess ProRes is
So we can lay out, let’s say, the top five codec-container combos would be H.264 inside MP4 containers, AV1 inside MP4/WebM containers, ProRes for nonlinear editing inside MOV containers. So for people who don’t know, I guess ProRes is
Kieran Kunhya
It’s Apple’s codec for editing, originally for Final Cut Pro, and it’s designed to be fast to decode, fast to seek, because an editor will need to move very quickly. So it’s a different use case to the distribution element.
It’s Apple’s codec for editing, originally for Final Cut Pro, and it’s designed to be fast to decode, fast to seek, because an editor will need to move very quickly. So it’s a different use case to the distribution element.
Lex Fridman
There’s no or very minimal temporal compression in the-
There’s no or very minimal temporal compression in the-
Kieran Kunhya
There’s none, yeah. There’s none in ProRes. So you can cut, so you can do cuts.
There’s none, yeah. There’s none in ProRes. So you can cut, so you can do cuts.
Jean-Baptiste Kempf
This is what we call intra-only codecs, right? So I’m going to explain quickly what is IPB frames.
This is what we call intra-only codecs, right? So I’m going to explain quickly what is IPB frames.
Lex Fridman
Yes, please.
Yes, please.
Jean-Baptiste Kempf
So I-frames, often key frames, are complete frames. It’s like an image. It’s a JPEG, right? You have… You can start, you see everything, right? And then the next image can be a P-frame, which is a predicted frame. So you take some part of the previous image saying, “Well, I need the block five and seven and forty-two,” and you replace it, and then you just give the extra information, right? But that means that in order to decode this P-frame, you need to have access to a previous I-frame, right? And then, of course, you have more complex ones, which are B-frames, which are bi-predicted frames, which can depend on different type of frames, some in the past, some in the future. And so ProRes is an intra-only codec. For the people who can see, this is-
So I-frames, often key frames, are complete frames. It’s like an image. It’s a JPEG, right? You have… You can start, you see everything, right? And then the next image can be a P-frame, which is a predicted frame. So you take some part of the previous image saying, “Well, I need the block five and seven and forty-two,” and you replace it, and then you just give the extra information, right? But that means that in order to decode this P-frame, you need to have access to a previous I-frame, right? And then, of course, you have more complex ones, which are B-frames, which are bi-predicted frames, which can depend on different type of frames, some in the past, some in the future. And so ProRes is an intra-only codec. For the people who can see, this is-
Lex Fridman
Yeah, that’s a good one
Yeah, that’s a good one
Jean-Baptiste Kempf
… a very good one, right? So I-frames are complete frames. Um, P-frames basically depend only on I-frames, and B-frames can depend on in front.
… a very good one, right? So I-frames are complete frames. Um, P-frames basically depend only on I-frames, and B-frames can depend on in front.
Lex Fridman
And this GOP, group of pictures—I think the default for actually FFmpeg for H.264 is like two hundred and fifty frames, something like this. And to me, it’s just, it’s like magic, like that you could predict that you could have a complete frame every-
And this GOP, group of pictures—I think the default for actually FFmpeg for H.264 is like two hundred and fifty frames, something like this. And to me, it’s just, it’s like magic, like that you could predict that you could have a complete frame every-
Jean-Baptiste Kempf
Several seconds, that means
Several seconds, that means
Lex Fridman
… several seconds, and then you could still, you could have this chain of predictions you make, and the fact that you can– The fact that somebody like me can use FFmpeg to compress something and not notice that the result still plays back smoothly is, is like magic.
… several seconds, and then you could still, you could have this chain of predictions you make, and the fact that you can– The fact that somebody like me can use FFmpeg to compress something and not notice that the result still plays back smoothly is, is like magic.
Jean-Baptiste Kempf
You can even have, and we use that in tons on Kyber, is what we call intra-refresh, where basically there is no I-frames present.
You can even have, and we use that in tons on Kyber, is what we call intra-refresh, where basically there is no I-frames present.
Kieran Kunhya
You have no I… You have one at the beginning. And you never send an I-frame. You get a-
You have no I… You have one at the beginning. And you never send an I-frame. You get a-
Lex Fridman
How does that work? What is it?
How does that work? What is it?
Kieran Kunhya
You build up an I-frame gradually across as the stream continues, so-
You build up an I-frame gradually across as the stream continues, so-
Lex Fridman
Ah. So you refresh certain parts- … of the image.
Ah. So you refresh certain parts- … of the image.
Jean-Baptiste Kempf
But so you never have an I-frame. Like this is intra-refresh that we use, right?
But so you never have an I-frame. Like this is intra-refresh that we use, right?
Lex Fridman
That’s even smarter.
That’s even smarter.
Jean-Baptiste Kempf
But for me, for me, the biggest mind-blown when I started was the B-frames. B-frames means bi-predicted frames can depend on frames that are coming in the future. That means that in order to decode this B frame, you need to wait for the next frame that is dependent- … buffer that, decode that one, so that you can decode the B frame, right? So the way you decode the frame, the decoding order is not the same as the display order. Right? That means the encoder needs to be very clever and decide that, “Well, you know, I’m going to depend on things like in the future.” So this is like-
But for me, for me, the biggest mind-blown when I started was the B-frames. B-frames means bi-predicted frames can depend on frames that are coming in the future. That means that in order to decode this B frame, you need to wait for the next frame that is dependent- … buffer that, decode that one, so that you can decode the B frame, right? So the way you decode the frame, the decoding order is not the same as the display order. Right? That means the encoder needs to be very clever and decide that, “Well, you know, I’m going to depend on things like in the future.” So this is like-
Lex Fridman
It’s incredible
It’s incredible
Jean-Baptiste Kempf
… mind-blowing.
… mind-blowing.
Kieran Kunhya
The fact it works so smoothly every day is kind of miraculous in some ways. It works so… You can have a stream that works across the world on their decoder versus one in the US versus one here of different manufacturers, and they produce bit for bit exactly the same material. That’s quite remarkable, and do quite complex things, and getting more and more complex and still be bit-exact. There’s a lot of work that goes into that.
The fact it works so smoothly every day is kind of miraculous in some ways. It works so… You can have a stream that works across the world on their decoder versus one in the US versus one here of different manufacturers, and they produce bit for bit exactly the same material. That’s quite remarkable, and do quite complex things, and getting more and more complex and still be bit-exact. There’s a lot of work that goes into that.
Lex Fridman
There’s a lot of knobs you can control in this whole process. There’s a lot of really fascinating parameters that I’ve gotten to know more and more over the years that FFmpeg gives you complete access to. Maybe you could speak to some of them. So first of all, like obviously, we can lower the resolution, we can lower the frame rate, we can use different kinds of codecs, as we mentioned, from H.264 to AV1. There’s ways to tune the trade-off between bitrate and quality, as we’ve kind of spoken to. You know, you could do constant bitrate, you can do constant quality, say CRF, QP. You can do the longer or shorter group of pictures, GOP, that we mentioned. I mean, all that kind of stuff. It’s crazy. Number of B-frames.
There’s a lot of knobs you can control in this whole process. There’s a lot of really fascinating parameters that I’ve gotten to know more and more over the years that FFmpeg gives you complete access to. Maybe you could speak to some of them. So first of all, like obviously, we can lower the resolution, we can lower the frame rate, we can use different kinds of codecs, as we mentioned, from H.264 to AV1. There’s ways to tune the trade-off between bitrate and quality, as we’ve kind of spoken to. You know, you could do constant bitrate, you can do constant quality, say CRF, QP. You can do the longer or shorter group of pictures, GOP, that we mentioned. I mean, all that kind of stuff. It’s crazy. Number of B-frames.
Jean-Baptiste Kempf
Yeah. What is crazy is that a ton of people’s job is to optimize those parameters, right? A ton of people that you see at YouTube, at Netflix, at Meta, and so on, they’re not writing codecs. They’re just like finding the right parameters for the file they have, for the format they have, right? Because like something that is for a movie or something that is user-generated content from your phone or a screen recording or something that you’re going to video edit, you don’t want the same things. And there are thousands of people whose job is just to optimize all that.
Yeah. What is crazy is that a ton of people’s job is to optimize those parameters, right? A ton of people that you see at YouTube, at Netflix, at Meta, and so on, they’re not writing codecs. They’re just like finding the right parameters for the file they have, for the format they have, right? Because like something that is for a movie or something that is user-generated content from your phone or a screen recording or something that you’re going to video edit, you don’t want the same things. And there are thousands of people whose job is just to optimize all that.
Lex Fridman
Yeah. They’re wizards. Hats off to them. YouTube, like, to deliver—all the streaming sites actually, to deliver at scale. And like YouTube is really magical because it’s not just doing like what Netflix does, which is one-way broadcasting type thing. It also has to upload videos from all the places. So they’re also doing encoding at scale- … for videos that are gonna be watched by like five people. And it still has to deliver them on a moment’s notice. No delay, nothing. I mean, very minimal latency. And also serve it in all different resolutions. Like YouTube is basically the web version of VLC.
Yeah. They’re wizards. Hats off to them. YouTube, like, to deliver—all the streaming sites actually, to deliver at scale. And like YouTube is really magical because it’s not just doing like what Netflix does, which is one-way broadcasting type thing. It also has to upload videos from all the places. So they’re also doing encoding at scale- … for videos that are gonna be watched by like five people. And it still has to deliver them on a moment’s notice. No delay, nothing. I mean, very minimal latency. And also serve it in all different resolutions. Like YouTube is basically the web version of VLC.
Jean-Baptiste Kempf
Yeah. Well, actually, it’s funny because, like Google Video, which was something they did before they acquired YouTube, was actually using the VLC plugin so that you could run VLC inside the web browser using the ActiveX plugin. And so it worked in Internet Explorer, and you were actually running VLC inside your browser. Which is funny because today we have the opposite, where we have VLC WebAssembly, where we compile all VLC and FFmpeg to decode, to run VLC inside the JavaScript virtual machine with WebAssembly.
Yeah. Well, actually, it’s funny because, like Google Video, which was something they did before they acquired YouTube, was actually using the VLC plugin so that you could run VLC inside the web browser using the ActiveX plugin. And so it worked in Internet Explorer, and you were actually running VLC inside your browser. Which is funny because today we have the opposite, where we have VLC WebAssembly, where we compile all VLC and FFmpeg to decode, to run VLC inside the JavaScript virtual machine with WebAssembly.
CIA and fake VLC
Lex Fridman
Okay, there’s this legendary story that you pointed me to that it was discovered via WikiLeaks release of Vault 7 documents. The CIA was using a modified version of VLC to basically try and trick people, what? To steal their data?
Okay, there’s this legendary story that you pointed me to that it was discovered via WikiLeaks release of Vault 7 documents. The CIA was using a modified version of VLC to basically try and trick people, what? To steal their data?
Jean-Baptiste Kempf
Yes, exactly.
Yes, exactly.
Lex Fridman
So can you explain what the heck happened? What…
So can you explain what the heck happened? What…
Jean-Baptiste Kempf
So, so this was a surprise, right? Because at some point, WikiLeaks mentioned some documents. There were a few ones with something related to Blu-rays and VLC, but the most interesting one was the CIA Vault 7, which, if I understand correctly, was the CIA had, like, a custom version of VLC where they had a specific plugin. Yeah, exactly. This is— Like, we had to write a press release on that.
So, so this was a surprise, right? Because at some point, WikiLeaks mentioned some documents. There were a few ones with something related to Blu-rays and VLC, but the most interesting one was the CIA Vault 7, which, if I understand correctly, was the CIA had, like, a custom version of VLC where they had a specific plugin. Yeah, exactly. This is— Like, we had to write a press release on that.
Lex Fridman
VideoLAN wrote a press release saying the only safe source for getting VLC media player is the official VideoLAN website. I mean, I suppose that’s a security vulnerability for basically any piece of open source software. Somebody can trick you.
VideoLAN wrote a press release saying the only safe source for getting VLC media player is the official VideoLAN website. I mean, I suppose that’s a security vulnerability for basically any piece of open source software. Somebody can trick you.
Jean-Baptiste Kempf
To download in a fake website—
To download in a fake website—
Lex Fridman
Yeah
Yeah
Jean-Baptiste Kempf
… or targeted advertisement, right? That was a targeted advertisement, to watch a specific file you need to watch with this custom version of VLC. And it was the normal binaries of VLC, except they added one DLL, I think it was psapi.dll-
… or targeted advertisement, right? That was a targeted advertisement, to watch a specific file you need to watch with this custom version of VLC. And it was the normal binaries of VLC, except they added one DLL, I think it was psapi.dll-
Lex Fridman
Yeah
Yeah
Jean-Baptiste Kempf
… which was basically reading your document folder, encrypting that, and sending that. And the thing is, this is very clever, to be honest, because once you’re watching a movie, right, you’re going to do that for two hours, and you’re not going to touch your computer. And sometimes it’s normal because it’s HD that your fans are going up and say, “Vroom,” and there is a ton of CPU usage because you’re using VLC, right? That’s normal. But the thing is, what you don’t see is that’s actually a powered version of VLC that is used by the CIA. We had exactly the same problem with Chinese hackers that were targeting Indian people, and that got VLC banned from India until I had to fight in courts in India, the Indian government, to unban VLC. They didn’t use VLC.
… which was basically reading your document folder, encrypting that, and sending that. And the thing is, this is very clever, to be honest, because once you’re watching a movie, right, you’re going to do that for two hours, and you’re not going to touch your computer. And sometimes it’s normal because it’s HD that your fans are going up and say, “Vroom,” and there is a ton of CPU usage because you’re using VLC, right? That’s normal. But the thing is, what you don’t see is that’s actually a powered version of VLC that is used by the CIA. We had exactly the same problem with Chinese hackers that were targeting Indian people, and that got VLC banned from India until I had to fight in courts in India, the Indian government, to unban VLC. They didn’t use VLC.
Jean-Baptiste Kempf
They took just one DLL, because we signed the DLL correctly and they used that DLL to do another program. So you had the vlc.exe and was calling libVLC, but it was calling it into a fake one. And they used that to, to target. There is not much we can do actually to block those type of hacks.
They took just one DLL, because we signed the DLL correctly and they used that DLL to do another program. So you had the vlc.exe and was calling libVLC, but it was calling it into a fake one. And they used that to, to target. There is not much we can do actually to block those type of hacks.
Lex Fridman
Yeah, and I think people should, for all open source software, for all software in general, people should pay attention where they download the thing.
Yeah, and I think people should, for all open source software, for all software in general, people should pay attention where they download the thing.
Jean-Baptiste Kempf
Yes, because that means that they were not downloading it from our website.
Yes, because that means that they were not downloading it from our website.
Lex Fridman
Do the search engines help you?
Do the search engines help you?
Jean-Baptiste Kempf
No, they don’t.
No, they don’t.
Lex Fridman
Just to clarify, ’cause you can, you know, to prevent threats from people manipulating SEO to get up there on the links and try to-
Just to clarify, ’cause you can, you know, to prevent threats from people manipulating SEO to get up there on the links and try to-
Jean-Baptiste Kempf
Absolutely not, right? We have a big issue for, like, more than ten years, is that there is a fake version of VLC in Germany that was reported for now for 12 years, and Google basically decides to not— They know what’s in it, but the binary is too big for their virus analyzer to analyze it. And so while if you’re in Germany, you can go to a website that is a fake version of VLC with a custom installer, and it’s very popular in Germany because their website is in German, and Google mentioned that before VideoLAN. And the weirdest thing is that it doesn’t do anything on your machine for three weeks.
Absolutely not, right? We have a big issue for, like, more than ten years, is that there is a fake version of VLC in Germany that was reported for now for 12 years, and Google basically decides to not— They know what’s in it, but the binary is too big for their virus analyzer to analyze it. And so while if you’re in Germany, you can go to a website that is a fake version of VLC with a custom installer, and it’s very popular in Germany because their website is in German, and Google mentioned that before VideoLAN. And the weirdest thing is that it doesn’t do anything on your machine for three weeks.
Jean-Baptiste Kempf
Because that’s how they do the detection. And after three weeks, there is a small program that is a service that installed at the same time that wakes up after three weeks, and it starts downloading spyware and adware. And Google knows about it. They’ve decided not to do anything. The guys used dark SEO in Germany to do that at some point. And this is very damaging, right? Because one of the things that they are downloading is actually something that is replacing your ads inside your machine, right?
Because that’s how they do the detection. And after three weeks, there is a small program that is a service that installed at the same time that wakes up after three weeks, and it starts downloading spyware and adware. And Google knows about it. They’ve decided not to do anything. The guys used dark SEO in Germany to do that at some point. And this is very damaging, right? Because one of the things that they are downloading is actually something that is replacing your ads inside your machine, right?
Lex Fridman
It’s actually quite surprisingly effective. Whoever is doing it with Twitter and X. With X, I’ll get emails about, “Your X account has been hacked.” And however they phrase it, it gets me to, like, at least click on the email, not to follow the thing, and then you’re like, “Man, whatever they’re doing with the psychology to try to trick you, they’re quite good.”
It’s actually quite surprisingly effective. Whoever is doing it with Twitter and X. With X, I’ll get emails about, “Your X account has been hacked.” And however they phrase it, it gets me to, like, at least click on the email, not to follow the thing, and then you’re like, “Man, whatever they’re doing with the psychology to try to trick you, they’re quite good.”
Jean-Baptiste Kempf
There is a security version of VLC, right? You received an email saying, “Hey, there is a security version update on VLC. Think about updating right now because—
There is a security version of VLC, right? You received an email saying, “Hey, there is a security version update on VLC. Think about updating right now because—
Lex Fridman
Right
Right
Jean-Baptiste Kempf
… it can hack your computer.” You come. It’s a website that looks decent, and—
… it can hack your computer.” You come. It’s a website that looks decent, and—
Lex Fridman
Yeah
Yeah
Jean-Baptiste Kempf
… and you download. It’s a new version of VLC. Great. You don’t know. A month later, you’re hacked. You have no idea. You’re part of a botnet.
… and you download. It’s a new version of VLC. Great. You don’t know. A month later, you’re hacked. You have no idea. You’re part of a botnet.
Lex Fridman
Yeah. So make sure wherever you’re downloading stuff, it’s legitimate. I’m part of the botnet. Speaking of which, so you’ve mentioned that VLC sandboxing is something you’re working on, and it’s actually something quite challenging. Why is it important? Why is it hard?
Yeah. So make sure wherever you’re downloading stuff, it’s legitimate. I’m part of the botnet. Speaking of which, so you’ve mentioned that VLC sandboxing is something you’re working on, and it’s actually something quite challenging. Why is it important? Why is it hard?
Jean-Baptiste Kempf
So VLC is a core with around 500 plugins, right? One of them is FFmpeg, but we have, we support so many other formats. We support new protocols, we support new filters, we support weird architectures. And in this release of VLC, you have modules that are going to call your drivers, right?
So VLC is a core with around 500 plugins, right? One of them is FFmpeg, but we have, we support so many other formats. We support new protocols, we support new filters, we support weird architectures. And in this release of VLC, you have modules that are going to call your drivers, right?
Jean-Baptiste Kempf
Mostly the hardware decoders, which are going to call your Intel, your NVIDIA, your AMD driver. And all calling FFmpeg, right? And there might be a security issue. There might be a security issue in the shader, there might be a security issue in VLC, in FFmpeg that is going to basically crash. The issue is that you’re running VLC like every, every other program, like Adobe, right? You’re running it on your machine, and it has access to all your documents, right? So the idea is to be sure that you do a sandbox so that we can protect from ourselves, because inside the VLC process is running some code that is not even ours. Either it’s open source for other projects that we integrate in VLC, or it’s your GPU driver or something that is provided by someone else inside.
Mostly the hardware decoders, which are going to call your Intel, your NVIDIA, your AMD driver. And all calling FFmpeg, right? And there might be a security issue. There might be a security issue in the shader, there might be a security issue in VLC, in FFmpeg that is going to basically crash. The issue is that you’re running VLC like every, every other program, like Adobe, right? You’re running it on your machine, and it has access to all your documents, right? So the idea is to be sure that you do a sandbox so that we can protect from ourselves, because inside the VLC process is running some code that is not even ours. Either it’s open source for other projects that we integrate in VLC, or it’s your GPU driver or something that is provided by someone else inside.
Jean-Baptiste Kempf
And so when we crash, we want to not allow people to do bad things, right? Because one of the common ways of hacking people is to crash a program, very often done with a web browser, very often done with PDF files, less often with multimedia, but that could happen. And when you crash, you launch something on the, on the machine of the person. Could be a ransomware, could be a botnet, right? So security of desktop applications is important. On mobile, it’s a bit different because most of the mobile applications are running inside their own sandbox. But for VLC, we could run it inside one sandbox, but the problem is that we need access to so many things that it’s basically we would have all the permissions, right?
And so when we crash, we want to not allow people to do bad things, right? Because one of the common ways of hacking people is to crash a program, very often done with a web browser, very often done with PDF files, less often with multimedia, but that could happen. And when you crash, you launch something on the, on the machine of the person. Could be a ransomware, could be a botnet, right? So security of desktop applications is important. On mobile, it’s a bit different because most of the mobile applications are running inside their own sandbox. But for VLC, we could run it inside one sandbox, but the problem is that we need access to so many things that it’s basically we would have all the permissions, right?
Jean-Baptiste Kempf
And so if you have a sandbox and you put some holes everywhere, it defeats the purpose, right? So what we are trying to do, and we’re actually doing, is splitting VLC into several processes. One is decoding, one is demuxing, one is filters, and all of them run into their own sandbox so that if a part of VLC crashes, like Chrome crashes on some, on some tab, right? It crashes, but it does not crash the whole program. And this is what we’re trying to do. And it’s difficult because it’s a sandbox that needs to sustain gigabits per second-
And so if you have a sandbox and you put some holes everywhere, it defeats the purpose, right? So what we are trying to do, and we’re actually doing, is splitting VLC into several processes. One is decoding, one is demuxing, one is filters, and all of them run into their own sandbox so that if a part of VLC crashes, like Chrome crashes on some, on some tab, right? It crashes, but it does not crash the whole program. And this is what we’re trying to do. And it’s difficult because it’s a sandbox that needs to sustain gigabits per second-
Jean-Baptiste Kempf
… of mem copies. Now, it’s not a website which is five megabytes or 10 megabytes. We’re talking about hundreds of megabits per second. So this is why it is quite challenging. And this is a research topic that we are working on in order to have a multimedia player that is secure.
… of mem copies. Now, it’s not a website which is five megabytes or 10 megabytes. We’re talking about hundreds of megabits per second. So this is why it is quite challenging. And this is a research topic that we are working on in order to have a multimedia player that is secure.
Lex Fridman
This is all the kind of stuff you have to think about when millions of people are using. You, you’ve mentioned something somewhere where, like all the different features of VLC, when you have that many people using it, somebody will use every single feature, and they will tell you about it.
This is all the kind of stuff you have to think about when millions of people are using. You, you’ve mentioned something somewhere where, like all the different features of VLC, when you have that many people using it, somebody will use every single feature, and they will tell you about it.
Jean-Baptiste Kempf
Best feature in VLC is called the puzzle filter, right? So you click the puzzle filter, and it transforms your video into a jigsaw puzzle, right?
Best feature in VLC is called the puzzle filter, right? So you click the puzzle filter, and it transforms your video into a jigsaw puzzle, right?
Lex Fridman
Nice.
Nice.
Jean-Baptiste Kempf
And you can click and move the pieces, right? It’s very, very useful when you’re watching a French movie, right? You’re bored- … because it’s like very long things or a love triangle, right? We’ve seen that so many times, right? But you need to watch it because someone, your wife or- … told you to do that-
And you can click and move the pieces, right? It’s very, very useful when you’re watching a French movie, right? You’re bored- … because it’s like very long things or a love triangle, right? We’ve seen that so many times, right? But you need to watch it because someone, your wife or- … told you to do that-
Lex Fridman
To catch up
To catch up
Jean-Baptiste Kempf
… or your boyfriend told you to do that. So you’re doing that, right? And you can click and move the pieces around. It’s absolutely useless, right? Like, who cares about that? First, it was done by a math teacher in high school in south of France to teach his students about Bézier curves, which is something that-
… or your boyfriend told you to do that. So you’re doing that, right? And you can click and move the pieces around. It’s absolutely useless, right? Like, who cares about that? First, it was done by a math teacher in high school in south of France to teach his students about Bézier curves, which is something that-
Jean-Baptiste Kempf
… everyone should know about, right? It’s very useful. But the code was clean, so it got in VLC. It was merged in 2010. Five years later, I receive an email saying, “Hello, JB. I have a problem with VLC. The puzzle is too simple.” And I was just like, “What?” And yes, the puzzle was in the UI maximums by 16 by 16, right? Only 256 pieces. And he says, “I’m sorry, but in a movie I love puzzles, this is too simple,” right? So there is a comment of me, you can check it online, which is JB changing that the dimensions are 256 by 256.
… everyone should know about, right? It’s very useful. But the code was clean, so it got in VLC. It was merged in 2010. Five years later, I receive an email saying, “Hello, JB. I have a problem with VLC. The puzzle is too simple.” And I was just like, “What?” And yes, the puzzle was in the UI maximums by 16 by 16, right? Only 256 pieces. And he says, “I’m sorry, but in a movie I love puzzles, this is too simple,” right? So there is a comment of me, you can check it online, which is JB changing that the dimensions are 256 by 256.
Lex Fridman
Right.
Right.
Jean-Baptiste Kempf
But my point is, so many useless features are used by a few people, right? There is a way to watch VLC movies in command line without any UI, right? It’s-
But my point is, so many useless features are used by a few people, right? There is a way to watch VLC movies in command line without any UI, right? It’s-
Lex Fridman
I saw that. You can do ASCII.
I saw that. You can do ASCII.
Jean-Baptiste Kempf
ASCII art. Is it useful? Very useful. Imagine you’re debugging… imagine you’re debugging a multicast network, right? You have thousands very complex, very complex networking stack, right? You can SSH to all of the routers and put VLC on it with no UI, and you’re going to see whether it’s black or it’s not black, right? So you see if it’s all green or not all green, right? So you can see-
ASCII art. Is it useful? Very useful. Imagine you’re debugging… imagine you’re debugging a multicast network, right? You have thousands very complex, very complex networking stack, right? You can SSH to all of the routers and put VLC on it with no UI, and you’re going to see whether it’s black or it’s not black, right? So you see if it’s all green or not all green, right? So you can see-
Lex Fridman
Amazing.
Amazing.
Jean-Baptiste Kempf
Yeah, right.
Yeah, right.
Lex Fridman
This is fun.
This is fun.
Jean-Baptiste Kempf
People don’t realize there is so many things in VLC that are useful and they have users, because once you have hundreds of millions of users, you have people who use every feature.
People don’t realize there is so many things in VLC that are useful and they have users, because once you have hundreds of millions of users, you have people who use every feature.
Ultra low latency streaming
Lex Fridman
I would love to sort of zoom in and talk a little bit more about the distinction between kinda downloading a file and watching it offline versus streaming. So the complexities, the challenges of streaming. Is there something we could say about what it takes to stream files? ‘Cause we’ve been talking about codecs- … and I think a lot of that implies encoding and decoding without having to communicate- … over the network. Sure. So can you elaborate, like, what’s required to do over network stuff?
I would love to sort of zoom in and talk a little bit more about the distinction between kinda downloading a file and watching it offline versus streaming. So the complexities, the challenges of streaming. Is there something we could say about what it takes to stream files? ‘Cause we’ve been talking about codecs- … and I think a lot of that implies encoding and decoding without having to communicate- … over the network. Sure. So can you elaborate, like, what’s required to do over network stuff?
Jean-Baptiste Kempf
Yeah, but it is less complex than it seems compared to everything that we’ve talked about. Especially because the most complex thing is not about streaming in terms of streaming services, but it was what was done to actually broadcast through satellites. Because in most of the modern broadcasting services, you can pause and you can go on. But when you’re sending live streaming, whether it’s broadcast or live for streaming services which are live, this is much more difficult because you need to encode in real time. When you go on a satellite, you have a specific size of the link, right? You cannot have a burst-
Yeah, but it is less complex than it seems compared to everything that we’ve talked about. Especially because the most complex thing is not about streaming in terms of streaming services, but it was what was done to actually broadcast through satellites. Because in most of the modern broadcasting services, you can pause and you can go on. But when you’re sending live streaming, whether it’s broadcast or live for streaming services which are live, this is much more difficult because you need to encode in real time. When you go on a satellite, you have a specific size of the link, right? You cannot have a burst-
Jean-Baptiste Kempf
… of bandwidth even for a second, right? Because you don’t have the space for that in your total file. However, there is different types of challenges, which are interesting challenges, but I think they are less complex than the one we’ve seen with late ’90s and early 2000s about broadcasting and streaming through satellite.
… of bandwidth even for a second, right? Because you don’t have the space for that in your total file. However, there is different types of challenges, which are interesting challenges, but I think they are less complex than the one we’ve seen with late ’90s and early 2000s about broadcasting and streaming through satellite.
Lex Fridman
They’re different. They are control systems challenges, whereas some are more mathematical. I think that’s the difference.
They’re different. They are control systems challenges, whereas some are more mathematical. I think that’s the difference.
Jean-Baptiste Kempf
In the streaming world, what you have is called what we call adaptive streaming, because the difficulty– and it’s not really a video problem, it’s mostly a CDN problem, is that you might have too many people watching the same thing at the same time, and it’s a congestion of the network, right? So your player has difficulty downloading things fast enough to play them. So what happens is that locally, the player is going to read a lower resolution- … of it. But there are some very clever algorithms to do that, but most of it is quite basic, to be honest.
In the streaming world, what you have is called what we call adaptive streaming, because the difficulty– and it’s not really a video problem, it’s mostly a CDN problem, is that you might have too many people watching the same thing at the same time, and it’s a congestion of the network, right? So your player has difficulty downloading things fast enough to play them. So what happens is that locally, the player is going to read a lower resolution- … of it. But there are some very clever algorithms to do that, but most of it is quite basic, to be honest.
Lex Fridman
Even on the buffering side, it’s pretty basic.
Even on the buffering side, it’s pretty basic.
Jean-Baptiste Kempf
Yeah, you start to download a segment, what we call a segment, and then you time, right? And if it takes more than 50% of the time to download a segment, you go down to… Right. And the difficulty is more about when do you go up in bandwidth, in quality. But this is not very complex to do. When you encode, you’re going to encode seven resolutions, right? And you’re going to give the bitrate. The difficulty is to have your encoder give the same bitrate, but it’s not as strict as it used to be. So-
Yeah, you start to download a segment, what we call a segment, and then you time, right? And if it takes more than 50% of the time to download a segment, you go down to… Right. And the difficulty is more about when do you go up in bandwidth, in quality. But this is not very complex to do. When you encode, you’re going to encode seven resolutions, right? And you’re going to give the bitrate. The difficulty is to have your encoder give the same bitrate, but it’s not as strict as it used to be. So-
Lex Fridman
Probably YouTube has to figure out the human psychology side of that, like how pissed off do you get when it’s like very low bitrate and how long should it wait before it increases the bitrate even though the connection is better? Because maybe the changes in the bitrate is what, like, affects you psychologically.
Probably YouTube has to figure out the human psychology side of that, like how pissed off do you get when it’s like very low bitrate and how long should it wait before it increases the bitrate even though the connection is better? Because maybe the changes in the bitrate is what, like, affects you psychologically.
Jean-Baptiste Kempf
No, I think actually the interesting one is the audio.
No, I think actually the interesting one is the audio.
Lex Fridman
That’s true.
That’s true.
Jean-Baptiste Kempf
The– you can kind of notice when they move from full fat AAC to the– there are compressed versions of AAC that use spectral band replication. You can kind of see it goes a bit tinny, and that up and down is very jarring. The video side is a lot smoother, and there’s less notice. It’s really the audio you can definitely feel it from when it’s moved you from a different audio profile to one or the other. I don’t know. We’re surprisingly tolerant at skipping audio glitches. I’m surprised people I know who are not video engineers, how tolerant they are to watching sports at 30 FPS, for example, whereas it should really be 60. The world is a lot more tolerant to that, but audio people are very– There’s– It’s an immediate feedback mechanism of, “Oh, something’s changed.”
The– you can kind of notice when they move from full fat AAC to the– there are compressed versions of AAC that use spectral band replication. You can kind of see it goes a bit tinny, and that up and down is very jarring. The video side is a lot smoother, and there’s less notice. It’s really the audio you can definitely feel it from when it’s moved you from a different audio profile to one or the other. I don’t know. We’re surprisingly tolerant at skipping audio glitches. I’m surprised people I know who are not video engineers, how tolerant they are to watching sports at 30 FPS, for example, whereas it should really be 60. The world is a lot more tolerant to that, but audio people are very– There’s– It’s an immediate feedback mechanism of, “Oh, something’s changed.”
Lex Fridman
If you hear a glitch, you realize it directly. I get to fully realize that, I suppose. One of the things I’m afraid of when I listen to audio more and more, that I get to notice every single tiny detail, and that you can over-obsess when people in general are able to kinda blur their consumption. They can look past certain imperfections.
If you hear a glitch, you realize it directly. I get to fully realize that, I suppose. One of the things I’m afraid of when I listen to audio more and more, that I get to notice every single tiny detail, and that you can over-obsess when people in general are able to kinda blur their consumption. They can look past certain imperfections.
Jean-Baptiste Kempf
But then when you combine like an event that is, for example, a sport event that is probably going through satellite or- … somewhere else and goes to a central place for encoding, and then you need to encode this older resolution in real time. You don’t have time for QA. You need to push that to CDNs. You need to add probably DRM for protection. You need to have that over a ton of different devices. Then yes, it is complex. But– And also, like you’re in the web browser or in very much different devices that you use for television, where you had like a defined set-top box or cable box that you know where you control end-to-end. So it’s a challenge, but it’s less… I think the networking part while you agree to have 10, 20 seconds of latency, I don’t think this is very difficult.
But then when you combine like an event that is, for example, a sport event that is probably going through satellite or- … somewhere else and goes to a central place for encoding, and then you need to encode this older resolution in real time. You don’t have time for QA. You need to push that to CDNs. You need to add probably DRM for protection. You need to have that over a ton of different devices. Then yes, it is complex. But– And also, like you’re in the web browser or in very much different devices that you use for television, where you had like a defined set-top box or cable box that you know where you control end-to-end. So it’s a challenge, but it’s less… I think the networking part while you agree to have 10, 20 seconds of latency, I don’t think this is very difficult.
Lex Fridman
Speaking of networking and latency, so your new effort, as we mentioned, is Kyber, which is aimed at ultra-low latency. As you say, every millisecond counts, and you’re applying that to remote control machines like robots, drones, computers. Can you tell me about it?
Speaking of networking and latency, so your new effort, as we mentioned, is Kyber, which is aimed at ultra-low latency. As you say, every millisecond counts, and you’re applying that to remote control machines like robots, drones, computers. Can you tell me about it?
Jean-Baptiste Kempf
Sure. If you start from where we used to be, right? You used to use FFmpeg to encode files, right? And then we used FFmpeg and VLC to encode in streaming services, right? And then you need to go lower and lower. And the question was where up to where we can, can we go?
Sure. If you start from where we used to be, right? You used to use FFmpeg to encode files, right? And then we used FFmpeg and VLC to encode in streaming services, right? And then you need to go lower and lower. And the question was where up to where we can, can we go?
Jean-Baptiste Kempf
And this question is very important because there are many use cases where you need to be fast, and it’s when you have feedback interaction, right? We are not just listening to something, you’re actually controlling it, right? Because– And that’s the biggest difference compared to what we’ve done so far, is that I need video to have a feedback on something that is happening live, whether it’s a drone flying, whether it’s controlling a humanoid robot from distance, whether it’s controlling a hover, whether it’s playing a video game in the cloud gaming, because this is what I did on a previous job, right? I was CTO of a cloud gaming startup. And this is a very interesting topic because you push to the limit the network.
And this question is very important because there are many use cases where you need to be fast, and it’s when you have feedback interaction, right? We are not just listening to something, you’re actually controlling it, right? Because– And that’s the biggest difference compared to what we’ve done so far, is that I need video to have a feedback on something that is happening live, whether it’s a drone flying, whether it’s controlling a humanoid robot from distance, whether it’s controlling a hover, whether it’s playing a video game in the cloud gaming, because this is what I did on a previous job, right? I was CTO of a cloud gaming startup. And this is a very interesting topic because you push to the limit the network.
Jean-Baptiste Kempf
You need to care not about the quality like we’ve done on video, and we’ve talked about with H.264. You care about latency, because a millisecond is meaningful when you’re controlling a car, right? For– Well, you’ve seen, you’ve used Waymos, right? When Waymos don’t work, and that happens even if one percent of the time, there is someone that is basically remote controlling that. And this is exactly the stuff that we’re building. It’s really an SDK platform to do end-to-end control of machines.
You need to care not about the quality like we’ve done on video, and we’ve talked about with H.264. You care about latency, because a millisecond is meaningful when you’re controlling a car, right? For– Well, you’ve seen, you’ve used Waymos, right? When Waymos don’t work, and that happens even if one percent of the time, there is someone that is basically remote controlling that. And this is exactly the stuff that we’re building. It’s really an SDK platform to do end-to-end control of machines.
Lex Fridman
So the– this comes up quite a lot in a lot of different contexts in robotics. So obviously, teleoperation, teleop is becoming more and more important including for training robots via machine learning.
So the– this comes up quite a lot in a lot of different contexts in robotics. So obviously, teleoperation, teleop is becoming more and more important including for training robots via machine learning.
Jean-Baptiste Kempf
Yes. And what we do is a bit different from everyone else, is that we take only one socket, one connection, which is a QUIC protocol based on UDP which is interesting because it’s done for low latency. It doesn’t have two of the, what we call the TCP head-of-line problem and the HTTP head-of-line problem. It’s ciphered by default, but on the same wire, we send multiple streams, like multiple tracks. We send audio, we send video, but we also send the commands, right? Mouse, keyboard, gamepad, and so on. And we do that while maintaining coherence, right? Synchronization. Because what people don’t realize is that all the clocks actually drift.
Yes. And what we do is a bit different from everyone else, is that we take only one socket, one connection, which is a QUIC protocol based on UDP which is interesting because it’s done for low latency. It doesn’t have two of the, what we call the TCP head-of-line problem and the HTTP head-of-line problem. It’s ciphered by default, but on the same wire, we send multiple streams, like multiple tracks. We send audio, we send video, but we also send the commands, right? Mouse, keyboard, gamepad, and so on. And we do that while maintaining coherence, right? Synchronization. Because what people don’t realize is that all the clocks actually drift.
Jean-Baptiste Kempf
And when you’re controlling a robot, a robot is going to have, like, two cameras, five cameras, ten cameras, a ton of sensors, GPS, and so on. And if you want to train correctly your robotic AI model, you need to have all those that are in sync and coherent. And what we’ve done, and it’s all the stuff that we learned on VLC in broadcast in real time, and MPEG-TS that Kierans know well, is that we account for clock drifting. And so when I record a Kyber stream, a robot, I am sure that it’s going to be predictive in the way you play it back. And so when you’re going to do recording and training of your AI model, you need to be sure that every time you retrain based on the data, the data is going to stay coherent. And clocks actually drift.
And when you’re controlling a robot, a robot is going to have, like, two cameras, five cameras, ten cameras, a ton of sensors, GPS, and so on. And if you want to train correctly your robotic AI model, you need to have all those that are in sync and coherent. And what we’ve done, and it’s all the stuff that we learned on VLC in broadcast in real time, and MPEG-TS that Kierans know well, is that we account for clock drifting. And so when I record a Kyber stream, a robot, I am sure that it’s going to be predictive in the way you play it back. And so when you’re going to do recording and training of your AI model, you need to be sure that every time you retrain based on the data, the data is going to stay coherent. And clocks actually drift.
Jean-Baptiste Kempf
Like, the existing solution works with one camera. Once you’re going to five or six, it’s more complex.
Like, the existing solution works with one camera. Once you’re going to five or six, it’s more complex.
Lex Fridman
So you wanna make sure that the visual snapshot perfectly matches the time it actually happened.
So you wanna make sure that the visual snapshot perfectly matches the time it actually happened.
Jean-Baptiste Kempf
Exactly. And also, if you’re going to control, right, I do something on robot, I need to be sure that it is actually happening at that precise time, right? And so we have on the server, which would be a robot, a time of, like, re-timestamping mechanism accounting for clock drift for that, right? So that’s one of the use case of Kyber to control robots. I also think, like, remote drones, remote whether it’s defense or non-defense, remote cars, remote submarines. There is many places in industry or remote surgery where the expert cannot go everywhere the machine is because it’s either dangerous or it’s too costly, right? So you allow people to have machines next to you, right?
Exactly. And also, if you’re going to control, right, I do something on robot, I need to be sure that it is actually happening at that precise time, right? And so we have on the server, which would be a robot, a time of, like, re-timestamping mechanism accounting for clock drift for that, right? So that’s one of the use case of Kyber to control robots. I also think, like, remote drones, remote whether it’s defense or non-defense, remote cars, remote submarines. There is many places in industry or remote surgery where the expert cannot go everywhere the machine is because it’s either dangerous or it’s too costly, right? So you allow people to have machines next to you, right?
Jean-Baptiste Kempf
The goal of Kyber is to make distance disappear because it’s either projection of skills or projection of power, right? So imagine we are all like— you’ve seen the Meta Ray-Ban and everyone else, right? You need to stream there, right? Because you’re not going to run anything over there, right? So you need GPU power whether it’s on a cloud, on a phone to stream that. And so all of these use cases need to be not about extremely low latency, but real-time latency for video. And so that means you need– we’re toying with the encoders so that the encoders encode a frame in four milliseconds. And Kieran with his company also goes under those type of latency, because you need to optimize at max the local latency, right?
The goal of Kyber is to make distance disappear because it’s either projection of skills or projection of power, right? So imagine we are all like— you’ve seen the Meta Ray-Ban and everyone else, right? You need to stream there, right? Because you’re not going to run anything over there, right? So you need GPU power whether it’s on a cloud, on a phone to stream that. And so all of these use cases need to be not about extremely low latency, but real-time latency for video. And so that means you need– we’re toying with the encoders so that the encoders encode a frame in four milliseconds. And Kieran with his company also goes under those type of latency, because you need to optimize at max the local latency, right?
Jean-Baptiste Kempf
Because it’s the decoder, the encoder and so on. Because this time is going to be added to your networking time. And it’s not just about low latency, it’s also about, like, reliability. We do clever things like forward error correction, right? So forward error correction is you over-transmit a bit of data, right, a few percent and while over-transmit, you’re allowed to lose some packets. Because all of that is very difficult over an internet network where you’re going to do things very far away. And if you check that all packets are delivered, you add a ton of latency. If you don’t want latency, what we do is that we over-transmit some data that you can reconstruct on the client side when there is things that are broken, right?
Because it’s the decoder, the encoder and so on. Because this time is going to be added to your networking time. And it’s not just about low latency, it’s also about, like, reliability. We do clever things like forward error correction, right? So forward error correction is you over-transmit a bit of data, right, a few percent and while over-transmit, you’re allowed to lose some packets. Because all of that is very difficult over an internet network where you’re going to do things very far away. And if you check that all packets are delivered, you add a ton of latency. If you don’t want latency, what we do is that we over-transmit some data that you can reconstruct on the client side when there is things that are broken, right?
Jean-Baptiste Kempf
So and a few days, weeks ago, we were doing the demo around Las Vegas for the CES about— we had a rover that is fully 3D printed. It’s very simple. It’s a car, right? It’s a small car with a telescopic arm, and it was actually controlled from France, right? And the video was with a webcam and a very small server, right? A small PCB was basically running and send that to someone that is on the other side of the planet. And so there is so many use cases. You can also think about having AI who are going to control many drones and so on. And technically, we need to be amazing in video, we need to be amazing at networking, we need to care about any milliseconds in networking, in encoding time, in decoding time, and also you need to integrate very low level.
So and a few days, weeks ago, we were doing the demo around Las Vegas for the CES about— we had a rover that is fully 3D printed. It’s very simple. It’s a car, right? It’s a small car with a telescopic arm, and it was actually controlled from France, right? And the video was with a webcam and a very small server, right? A small PCB was basically running and send that to someone that is on the other side of the planet. And so there is so many use cases. You can also think about having AI who are going to control many drones and so on. And technically, we need to be amazing in video, we need to be amazing at networking, we need to care about any milliseconds in networking, in encoding time, in decoding time, and also you need to integrate very low level.
Lex Fridman
So sync everything together well. But how– Like, what kind of latency can you get to? Like, why– When you say milliseconds, what’s the goal?
So sync everything together well. But how– Like, what kind of latency can you get to? Like, why– When you say milliseconds, what’s the goal?
Jean-Baptiste Kempf
So my goal is four milliseconds glass-to-glass latency.
So my goal is four milliseconds glass-to-glass latency.
Lex Fridman
What’s glass-to-glass mean?
What’s glass-to-glass mean?
Jean-Baptiste Kempf
So it’s easy, right? You have a computer which is running a program, right? Probably a video game, and this one is actually running, right? It could be– it’s an example of a robot, right? And you have the replicate that is- … done through the network. And you want, if you take a one thousand hertz camera, you can take a picture, and you want that to be at four milliseconds. Four milliseconds means two hundred and forty hertz, right?
So it’s easy, right? You have a computer which is running a program, right? Probably a video game, and this one is actually running, right? It could be– it’s an example of a robot, right? And you have the replicate that is- … done through the network. And you want, if you take a one thousand hertz camera, you can take a picture, and you want that to be at four milliseconds. Four milliseconds means two hundred and forty hertz, right?
Lex Fridman
Yes. Nuts.
Yes. Nuts.
Jean-Baptiste Kempf
So far we achieve seven milliseconds from a Windows to Windows or Windows to Mac. And if you look in the timing, there is around three point five milliseconds inside the NVIDIA hardware encoder and around two milliseconds on the Intel decoder, right? So, like, the encoder plus the decoder is already six milliseconds, right? So in order to go down, we need either to have some other type of codecs or some better encoder that are faster. But four milliseconds would be the grail.
So far we achieve seven milliseconds from a Windows to Windows or Windows to Mac. And if you look in the timing, there is around three point five milliseconds inside the NVIDIA hardware encoder and around two milliseconds on the Intel decoder, right? So, like, the encoder plus the decoder is already six milliseconds, right? So in order to go down, we need either to have some other type of codecs or some better encoder that are faster. But four milliseconds would be the grail.
Lex Fridman
That’s pretty nuts. I love it, though. I don’t think anyone’s ever achieved that, right? That’s fast.
That’s pretty nuts. I love it, though. I don’t think anyone’s ever achieved that, right? That’s fast.
Jean-Baptiste Kempf
You can achieve that with custom hardware- … with SDI, with professional hardware. But I want that to work over the internet. I want that to work with any robots where you’re going to have a small Jetson Nano in it or a N150, right? I want that because there is going to be millions of robots or- … drones are just rolling robots or flying robots or swimming robots, right? It’s just you, a machine that you control. And in order… Either you need to teleoperate them or when everything will be fully autonomous, you need to teleobserve them, right? You need to check what’s happening.
You can achieve that with custom hardware- … with SDI, with professional hardware. But I want that to work over the internet. I want that to work with any robots where you’re going to have a small Jetson Nano in it or a N150, right? I want that because there is going to be millions of robots or- … drones are just rolling robots or flying robots or swimming robots, right? It’s just you, a machine that you control. And in order… Either you need to teleoperate them or when everything will be fully autonomous, you need to teleobserve them, right? You need to check what’s happening.
Jean-Baptiste Kempf
And in my view, in the future, like, all those remote cars will be teleobserved by an AI model, which is just going to say, “Well, everything is good.” And when it’s not good, say, “Hey, there is a problem,” and then you have an operator, right? And this is going to be about safety, right? When you have your humanoid taking care of your grandma or my grandma, I want to be sure that everything goes well, and I’m not in those type of horrible scenarios where the robot is dangerous. Or when I’m driving, I want, like, the car to stop when it should stop, and if needed, someone takes care of that, right? And so there is so many cases, scenarios about real time, and so the goal of Kyber is to make real time control of machine. Distance disappears.
And in my view, in the future, like, all those remote cars will be teleobserved by an AI model, which is just going to say, “Well, everything is good.” And when it’s not good, say, “Hey, there is a problem,” and then you have an operator, right? And this is going to be about safety, right? When you have your humanoid taking care of your grandma or my grandma, I want to be sure that everything goes well, and I’m not in those type of horrible scenarios where the robot is dangerous. Or when I’m driving, I want, like, the car to stop when it should stop, and if needed, someone takes care of that, right? And so there is so many cases, scenarios about real time, and so the goal of Kyber is to make real time control of machine. Distance disappears.
Lex Fridman
It’s incredible. And some of the same technology, some of the same ideas that we’re talking about is connected to what you’re doing.
It’s incredible. And some of the same technology, some of the same ideas that we’re talking about is connected to what you’re doing.
Jean-Baptiste Kempf
And for me, it’s amazingly challenging, right? Because I would say that on video I’m doing okay, but networking I have so much more to learn, right? It’s about, like, congestion protocols, bitrate adaptation in real time. But it’s quite funny. And so I created this project and we have fundraised in the US, of course. But it’s open source, right? This is important, right? Like, we’ve not said that, right? But everything on Kyber is open source.
And for me, it’s amazingly challenging, right? Because I would say that on video I’m doing okay, but networking I have so much more to learn, right? It’s about, like, congestion protocols, bitrate adaptation in real time. But it’s quite funny. And so I created this project and we have fundraised in the US, of course. But it’s open source, right? This is important, right? Like, we’ve not said that, right? But everything on Kyber is open source.
Lex Fridman
So how do you make money?
So how do you make money?
Jean-Baptiste Kempf
It’s a dual license, commercial and AGPL, right? You remember what you said- … about licenses. Basically, if you want to use Kyber in your product, you must have your full product open source. If you want to use this amazing technology but not open source, you pay the commercial license, right? So the small people or the hobbyist and the very small guys who want to do that, they can use the technology. They build something that is open source and cool.
It’s a dual license, commercial and AGPL, right? You remember what you said- … about licenses. Basically, if you want to use Kyber in your product, you must have your full product open source. If you want to use this amazing technology but not open source, you pay the commercial license, right? So the small people or the hobbyist and the very small guys who want to do that, they can use the technology. They build something that is open source and cool.
Lex Fridman
That’s awesome.
That’s awesome.
Jean-Baptiste Kempf
And if you’re a large company, you’re going to have the support, all the IP, the right modification, and so on. So yeah, it’s really cool and also I’m building robots, and I love that, right? Like we have– Like the rover we have is 3D printed. We are finishing a demo where it’s an actual wing, right? Like a type of drone wings that is also fully 3D printed. We are trying to do a sailboat that is 3D printed. And we’ll work on some humanoids. Of course, they are not going to be very good robots, right? It’s not our job, but we’re here for everyone to make robots. Cool.
And if you’re a large company, you’re going to have the support, all the IP, the right modification, and so on. So yeah, it’s really cool and also I’m building robots, and I love that, right? Like we have– Like the rover we have is 3D printed. We are finishing a demo where it’s an actual wing, right? Like a type of drone wings that is also fully 3D printed. We are trying to do a sailboat that is 3D printed. And we’ll work on some humanoids. Of course, they are not going to be very good robots, right? It’s not our job, but we’re here for everyone to make robots. Cool.
AV2 codec and video patents
Lex Fridman
Ah, you’re talking to the right guy. I love robots. There’s a bunch of them upstairs. And teleop is gonna be really, really important, especially as the number of robots scales across the world. So 100%. Let’s talk about the future of multimedia. FFmpeg, VLC, but some of the codecs, we didn’t really mention AV2. So can we just lay out what is AV2? What is the hope for it? What is H.265, H.266?
Ah, you’re talking to the right guy. I love robots. There’s a bunch of them upstairs. And teleop is gonna be really, really important, especially as the number of robots scales across the world. So 100%. Let’s talk about the future of multimedia. FFmpeg, VLC, but some of the codecs, we didn’t really mention AV2. So can we just lay out what is AV2? What is the hope for it? What is H.265, H.266?
Jean-Baptiste Kempf
So AV1 is this codec that is done by the Alliance for Open Media, right? Where there is Google, Netflix, Amazon, Apple VideoLAN, where we try to make a royalty-free very good codec, right? And now it’s being deployed. But actually, the codec was finished in 2018, but a codec takes years to be used in wide scenarios, right? So AV2 is the next generation of this codec. It’s 30% better, right? So if you keep the same quality, you get 30% bandwidth reduction compared to AV1.
So AV1 is this codec that is done by the Alliance for Open Media, right? Where there is Google, Netflix, Amazon, Apple VideoLAN, where we try to make a royalty-free very good codec, right? And now it’s being deployed. But actually, the codec was finished in 2018, but a codec takes years to be used in wide scenarios, right? So AV2 is the next generation of this codec. It’s 30% better, right? So if you keep the same quality, you get 30% bandwidth reduction compared to AV1.
Lex Fridman
What’s the connection with the dav1d and AV2?
What’s the connection with the dav1d and AV2?
Jean-Baptiste Kempf
We are going to do a dav1d 2, right? That I call Devid, because de is two in French.
We are going to do a dav1d 2, right? That I call Devid, because de is two in French.
Lex Fridman
Ah, well done.
Ah, well done.
Jean-Baptiste Kempf
… And you have to know that dav1d is an actual what we call recursive acronym, right? Because it means D, dav1d, is an AV1 decoder, right? So
… And you have to know that dav1d is an actual what we call recursive acronym, right? Because it means D, dav1d, is an AV1 decoder, right? So
Lex Fridman
Oh, nice. Nice. I didn’t even think of that. And people should know that dav1d is spelled with a one.
Oh, nice. Nice. I didn’t even think of that. And people should know that dav1d is spelled with a one.
Jean-Baptiste Kempf
Yes. It’s… And so dav1d 2-
Yes. It’s… And so dav1d 2-
Lex Fridman
It’s gonna be spelled with a two. Please tell us
It’s gonna be spelled with a two. Please tell us
Jean-Baptiste Kempf
… is going to be D-A-V-2-D. Sorry, I don’t know how you pronounce that. And again, we did a demo at the CES of VLC running the first demo of AV2.
… is going to be D-A-V-2-D. Sorry, I don’t know how you pronounce that. And again, we did a demo at the CES of VLC running the first demo of AV2.
Lex Fridman
So can you clarify to me the specification of AV2? And then the encoding and the decoding
So can you clarify to me the specification of AV2? And then the encoding and the decoding
Jean-Baptiste Kempf
Sure. So the specification is like the document which explains how the codec is supposed to work, right?
Sure. So the specification is like the document which explains how the codec is supposed to work, right?
Lex Fridman
And that’s really AV2.
And that’s really AV2.
Jean-Baptiste Kempf
That is AV2, like H.264. Right? Then you have an encoder. The current encoder is called AVM, and there will probably be other encoders, probably one called SVT-AV2, and those are the encoder. The same way x264 is an encoder to H.264, the same way that x265 is an encoder for the H.265 codec. And the decoders for AV1 is dav1d. The decoder for AV2 is dav2d. The decoder for H.264 is ffh264 inside FFmpeg. The decoder for HEVC is ffhevc inside FFmpeg. And there is a next generation codec from the MPEG world after H.264, H.265. There is one that is called H.266, also known as VVC.
That is AV2, like H.264. Right? Then you have an encoder. The current encoder is called AVM, and there will probably be other encoders, probably one called SVT-AV2, and those are the encoder. The same way x264 is an encoder to H.264, the same way that x265 is an encoder for the H.265 codec. And the decoders for AV1 is dav1d. The decoder for AV2 is dav2d. The decoder for H.264 is ffh264 inside FFmpeg. The decoder for HEVC is ffhevc inside FFmpeg. And there is a next generation codec from the MPEG world after H.264, H.265. There is one that is called H.266, also known as VVC.
Lex Fridman
So HEVC is H.265. VVC is H.266. Why is H.266 super sexy- … and so much better?
So HEVC is H.265. VVC is H.266. Why is H.266 super sexy- … and so much better?
Jean-Baptiste Kempf
So the question often we have is why are there two names? Because most of the time it is a joint work from the ISO world and the ITU, which is the International Telecommunication Union.
So the question often we have is why are there two names? Because most of the time it is a joint work from the ISO world and the ITU, which is the International Telecommunication Union.
Lex Fridman
These are these two regulatory bodies.
These are these two regulatory bodies.
Kieran Kunhya
No, one is a private entity and one is the United Nations.
No, one is a private entity and one is the United Nations.
Lex Fridman
Which one is the private?
Which one is the private?
Kieran Kunhya
ISO is private.
ISO is private.
Jean-Baptiste Kempf
In theory, H.264 is MPEG-4 Part 10, H.264/AVC. And this is the full name.
In theory, H.264 is MPEG-4 Part 10, H.264/AVC. And this is the full name.
Lex Fridman
Nice.
Nice.
Kieran Kunhya
So it’s the concatenation of the ISO name and the ITU name- … even though they work together. This is, this is politics, historical, you know— … gotcha.
So it’s the concatenation of the ISO name and the ITU name- … even though they work together. This is, this is politics, historical, you know— … gotcha.
Jean-Baptiste Kempf
And for HEVC, it’s MPEG-H, H.265, HEVC.
And for HEVC, it’s MPEG-H, H.265, HEVC.
Lex Fridman
Got it.
Got it.
Jean-Baptiste Kempf
And there is H.266, which is also named VVC.
And there is H.266, which is also named VVC.
Lex Fridman
Is there a high-level thing to say about the improvement of-
Is there a high-level thing to say about the improvement of-
Jean-Baptiste Kempf
30% each generation is the best summary.
30% each generation is the best summary.
Lex Fridman
This is true both for the AV- … codecs and the- … H.264, 5, 6.
This is true both for the AV- … codecs and the- … H.264, 5, 6.
Jean-Baptiste Kempf
So the professionals who are listening to us are going to kill us because they say, “No, it’s 35%, 25%—
So the professionals who are listening to us are going to kill us because they say, “No, it’s 35%, 25%—
Kieran Kunhya
“No, it’s 50, 60.”
“No, it’s 50, 60.”
Jean-Baptiste Kempf
… it’s 50,” blah, blah, blah. But globally, you need to know that HEVC is 30% better than H.264. H.266 is 30% better than H.265 because there are so many cases and so many scenarios. For example, there are cases, especially for screen recording, where the gains are humongous because you arrive, you have the right tool that is done for that. And so for a specific video, a new generation is going to give you 70% gain or 80% gain. Right? But there used to be a ton more codecs, but now the two main families for transmission are H.264, H.265, H.266, and the other is AV1, AV2.
… it’s 50,” blah, blah, blah. But globally, you need to know that HEVC is 30% better than H.264. H.266 is 30% better than H.265 because there are so many cases and so many scenarios. For example, there are cases, especially for screen recording, where the gains are humongous because you arrive, you have the right tool that is done for that. And so for a specific video, a new generation is going to give you 70% gain or 80% gain. Right? But there used to be a ton more codecs, but now the two main families for transmission are H.264, H.265, H.266, and the other is AV1, AV2.
Lex Fridman
And I guess the major difference would be the cost of encoding.
And I guess the major difference would be the cost of encoding.
Jean-Baptiste Kempf
Yes, and the royalty of the patents. And this is the reasons why you see the AV version of codecs, is because they try to be as royalty-free—which means no cost for the patents—as much as possible. Because what you need to know, and we’ve not talked about that so far, is that multimedia is what we call a patent minefield. There are two places where you have the most patents. It’s everything related to 3G, 4G, 5G, RF, and multimedia. Because it’s very mathematical, and you can get great gains and so on. So Google and Meta and Netflix wanted something where it was royalty-free. There are people who said that they have patents outside, but they are fringe patents, right? So it’s mostly true that it’s patent-free.
Yes, and the royalty of the patents. And this is the reasons why you see the AV version of codecs, is because they try to be as royalty-free—which means no cost for the patents—as much as possible. Because what you need to know, and we’ve not talked about that so far, is that multimedia is what we call a patent minefield. There are two places where you have the most patents. It’s everything related to 3G, 4G, 5G, RF, and multimedia. Because it’s very mathematical, and you can get great gains and so on. So Google and Meta and Netflix wanted something where it was royalty-free. There are people who said that they have patents outside, but they are fringe patents, right? So it’s mostly true that it’s patent-free.
Kieran Kunhya
Oh, you should extend. Patent, patent checking was done as part of the standardization process in AV1, AV2, whereas patents are not even discussed in the MPEG world.
Oh, you should extend. Patent, patent checking was done as part of the standardization process in AV1, AV2, whereas patents are not even discussed in the MPEG world.
Jean-Baptiste Kempf
The MPEG world.
The MPEG world.
Kieran Kunhya
Patents are off-topic completely.
Patents are off-topic completely.
Lex Fridman
Can you educate me on the patent side?
Can you educate me on the patent side?
Jean-Baptiste Kempf
So usually, MPEG does a format, right? And then everyone comes around and says, “Well, I have all those patents for the format,” and they do usually a union called what’s called MPEG LA, MPEG Licensing Association. And you put all your patents in, and then you ask everyone who’s using this format to pay for it.
So usually, MPEG does a format, right? And then everyone comes around and says, “Well, I have all those patents for the format,” and they do usually a union called what’s called MPEG LA, MPEG Licensing Association. And you put all your patents in, and then you ask everyone who’s using this format to pay for it.
Lex Fridman
Wait, can you elaborate? What does it mean to have a patent for a codec? Why are there many patents?
Wait, can you elaborate? What does it mean to have a patent for a codec? Why are there many patents?
Jean-Baptiste Kempf
Imagine I’m doing something where I’m going to def— instead of doing blocks which are square, I’m going to do rectangles, right?
Imagine I’m doing something where I’m going to def— instead of doing blocks which are square, I’m going to do rectangles, right?
Lex Fridman
Oh, so every idea- … somebody patents it. Oh, man. Oh, man. People and their… How many lawyers are-
Oh, so every idea- … somebody patents it. Oh, man. Oh, man. People and their… How many lawyers are-
Kieran Kunhya
I mean, it pays for a lot of lawyers, right? Like-
I mean, it pays for a lot of lawyers, right? Like-
Jean-Baptiste Kempf
The biggest issue is not the following, right? Because at time of H.264, the patents were, let’s call it, like, sane. But there was so much money in that— … that for HEVC, a lot—there were a ton of things that were pushed inside the specification, which are not useful in 99.9% of the time, but so just one could add a patent on it. And so it became that for HEVC licensing, there was MPEG LA plus another patent pool called HEVC Advance. Plus-
The biggest issue is not the following, right? Because at time of H.264, the patents were, let’s call it, like, sane. But there was so much money in that— … that for HEVC, a lot—there were a ton of things that were pushed inside the specification, which are not useful in 99.9% of the time, but so just one could add a patent on it. And so it became that for HEVC licensing, there was MPEG LA plus another patent pool called HEVC Advance. Plus-
Kieran Kunhya
That one
That one
Jean-Baptiste Kempf
… I think Nokia was outside of the patent pool.
… I think Nokia was outside of the patent pool.
Kieran Kunhya
Yeah, a few of them are outside, and some other one that’s-
Yeah, a few of them are outside, and some other one that’s-
Jean-Baptiste Kempf
And so it was impossible to license, right? And I think that several months ago, HP decided that they were going to remove support from HEVC in their Windows laptops because the, the cost was increasing of those patents. And it arrived—
And so it was impossible to license, right? And I think that several months ago, HP decided that they were going to remove support from HEVC in their Windows laptops because the, the cost was increasing of those patents. And it arrived—
Lex Fridman
Nice
Nice
Jean-Baptiste Kempf
… where a point where—And there was uncapped pay. And so for YouTube or Netflix, we could talk about hundreds of millions of dollars of licensing for patents per year. And they said, “You know what? At hundred million per year, you know, I could create my own codec,” and this is what they did. And so that’s why we have the Alliance for Open Media, where we are part of, that created AV1 and creates AV2. We create also audio codecs. But yes. So the main difference would be that, and because you need to work around the patents or go do some things that are not patented, a lot of things are different, right? The basic things that were done in MPEG-2 thirty years ago are, of course, out of patents. But so for example, there is things like a golden frame, a S-frame, or, or different type of—
… where a point where—And there was uncapped pay. And so for YouTube or Netflix, we could talk about hundreds of millions of dollars of licensing for patents per year. And they said, “You know what? At hundred million per year, you know, I could create my own codec,” and this is what they did. And so that’s why we have the Alliance for Open Media, where we are part of, that created AV1 and creates AV2. We create also audio codecs. But yes. So the main difference would be that, and because you need to work around the patents or go do some things that are not patented, a lot of things are different, right? The basic things that were done in MPEG-2 thirty years ago are, of course, out of patents. But so for example, there is things like a golden frame, a S-frame, or, or different type of—
Lex Fridman
These are all patented ideas.
These are all patented ideas.
Kieran Kunhya
Yeah, no, it’s I can’t believe it’s not butter. I can’t believe it’s not a B-frame. It’s, I mean, I mean, it’s kind of what it is. In some ways, it’s like a-
Yeah, no, it’s I can’t believe it’s not butter. I can’t believe it’s not a B-frame. It’s, I mean, I mean, it’s kind of what it is. In some ways, it’s like a-
Lex Fridman
Oh, so it’s a different variant of a B-frame.
Oh, so it’s a different variant of a B-frame.
Kieran Kunhya
Yeah, that’s to try and sidestep. Things like that.
Yeah, that’s to try and sidestep. Things like that.
Jean-Baptiste Kempf
And so you need to have double creativity, right? Creativity in terms of being more efficient, but creativity of being sure that you don’t infringe existing patents. And so, for example, VVC is, has all the patents of HEVC plus new ones, right? It’s why AV2 tries to be as royalty-free as possible.
And so you need to have double creativity, right? Creativity in terms of being more efficient, but creativity of being sure that you don’t infringe existing patents. And so, for example, VVC is, has all the patents of HEVC plus new ones, right? It’s why AV2 tries to be as royalty-free as possible.
Lex Fridman
To what degree does FFmpeg and VLC have to think about this kind of stuff?
To what degree does FFmpeg and VLC have to think about this kind of stuff?
Jean-Baptiste Kempf
We don’t, and one of the reasons why VLC was in France is that France rejects software patents. So most of those patents are illegal in France because I once made the calculus that if I had to pay all the licensing fee for VLC, I needed to pay more than two hundred euros per user, right? It’s the same in dollars. But most of those patents are invalid in Europe because those are called—it’s basically mathematical patents or idea patents, and they are not valid in Europe.
We don’t, and one of the reasons why VLC was in France is that France rejects software patents. So most of those patents are illegal in France because I once made the calculus that if I had to pay all the licensing fee for VLC, I needed to pay more than two hundred euros per user, right? It’s the same in dollars. But most of those patents are invalid in Europe because those are called—it’s basically mathematical patents or idea patents, and they are not valid in Europe.
VLC backdoors
Lex Fridman
Let me just at a high level, just out of curiosity. So the meme online and the interwebs on X and Twitter and so on, and my own—I have friends in Europe—this, the sense is that Europe is not friendly to entrepreneurship. They over-regulate, there’s too much bureaucracy, and so on. Is, is there any- anything positive to say? Is there hope for entrepreneurship- … in the future of Europe? Is Europe over from a tech perspective?
Let me just at a high level, just out of curiosity. So the meme online and the interwebs on X and Twitter and so on, and my own—I have friends in Europe—this, the sense is that Europe is not friendly to entrepreneurship. They over-regulate, there’s too much bureaucracy, and so on. Is, is there any- anything positive to say? Is there hope for entrepreneurship- … in the future of Europe? Is Europe over from a tech perspective?
Kieran Kunhya
Just, just look at the two of us, right? It’s, it’s notable that there’s two people from the European continent on this podcast talking about video. It’s fair to say the community is weighted heavily.
Just, just look at the two of us, right? It’s, it’s notable that there’s two people from the European continent on this podcast talking about video. It’s fair to say the community is weighted heavily.
Jean-Baptiste Kempf
What you probably don’t see yet is that there is a new generation of entrepreneurs in Europe and mostly in France. UK has done it since a long time because, well, it’s more—it’s more Anglo-Saxon type of business. But especially like what happened in France, and of course, sometimes a bit overdone with everything called French Tech, but today, most of the people who come on the market want to create startups. Fifteen years ago, it wasn’t the case. Everyone wanted to work on big companies because when you failed in France, for example, twenty years ago, fifteen years ago, and you destroy your company, which is normal for a startup, right? You were not allowed to create a new company, right? There was a lot of stigma. The stigma is gone.
What you probably don’t see yet is that there is a new generation of entrepreneurs in Europe and mostly in France. UK has done it since a long time because, well, it’s more—it’s more Anglo-Saxon type of business. But especially like what happened in France, and of course, sometimes a bit overdone with everything called French Tech, but today, most of the people who come on the market want to create startups. Fifteen years ago, it wasn’t the case. Everyone wanted to work on big companies because when you failed in France, for example, twenty years ago, fifteen years ago, and you destroy your company, which is normal for a startup, right? You were not allowed to create a new company, right? There was a lot of stigma. The stigma is gone.
Jean-Baptiste Kempf
… there is so many things happening on AI in France and so on, right? So there is, sure, over-regulations. I know that, right? I’m an entrepreneur. But it has some good things also.
… there is so many things happening on AI in France and so on, right? So there is, sure, over-regulations. I know that, right? I’m an entrepreneur. But it has some good things also.
Lex Fridman
I mean, is there some paralyzing aspects? You know, if I look at, at the case of somebody I’ve become close with, Pavel Durov, you know, he was uh, blamed directly by the French government for the kind of things his, quote, “platform” was hosting. Uh, I could see the same kind of stuff basically, just as an example, VLC being blamed for the kind of videos that people are watching.
I mean, is there some paralyzing aspects? You know, if I look at, at the case of somebody I’ve become close with, Pavel Durov, you know, he was uh, blamed directly by the French government for the kind of things his, quote, “platform” was hosting. Uh, I could see the same kind of stuff basically, just as an example, VLC being blamed for the kind of videos that people are watching.
Jean-Baptiste Kempf
But they tried, right? Like we had issues. Like we-
But they tried, right? Like we had issues. Like we-
Lex Fridman
I mean, is that, that’s the pressure that people worry about because if you have to think about that kind of stuff when you’re kind of just obsessed about-
I mean, is that, that’s the pressure that people worry about because if you have to think about that kind of stuff when you’re kind of just obsessed about-
Jean-Baptiste Kempf
No, you don’t think about it- … and that’s, that’s okay, right? Like-
No, you don’t think about it- … and that’s, that’s okay, right? Like-
Lex Fridman
But what if they come in? When, what if they show up and-
But what if they come in? When, what if they show up and-
Jean-Baptiste Kempf
There is no office. VideoLAN doesn’t have an office.
There is no office. VideoLAN doesn’t have an office.
Lex Fridman
I mean, this is what happened with Pavel. They arrested him, right? So arrested him for particular videos or, or a particular content that’s being shared on the platform.
I mean, this is what happened with Pavel. They arrested him, right? So arrested him for particular videos or, or a particular content that’s being shared on the platform.
Jean-Baptiste Kempf
Sure. I don’t have any platform. Everything is on the client side.
Sure. I don’t have any platform. Everything is on the client side.
Lex Fridman
Yeah, but they’re, they can still arrest you.
Yeah, but they’re, they can still arrest you.
Jean-Baptiste Kempf
On what ground? I’m not sharing anything. I’m not– The content doesn’t go through my stuff.
On what ground? I’m not sharing anything. I’m not– The content doesn’t go through my stuff.
Lex Fridman
For sure, but it’s still lawyer fees. That’s the problem.
For sure, but it’s still lawyer fees. That’s the problem.
Jean-Baptiste Kempf
Yes, that’s correct.
Yes, that’s correct.
Lex Fridman
It’s paperwork. So like, actually, if you had infinite trillions of dollars you would win easily because you’re on the right side. But the thing is, there is a degree to which they suffocate you with paperwork. That’s the downside of bureaucracy, through paperwork, through process. You know, it’s the Kafkaesque thing.
It’s paperwork. So like, actually, if you had infinite trillions of dollars you would win easily because you’re on the right side. But the thing is, there is a degree to which they suffocate you with paperwork. That’s the downside of bureaucracy, through paperwork, through process. You know, it’s the Kafkaesque thing.
Jean-Baptiste Kempf
You have to realize that one of the good things, for example in France or most of Europe, is that the—Answering to a court order does not make you bankrupt, right? It’s not like in the US, where it can actually bankrupt you, right? There is—The way the law system works is that, like I receive lawyers’ letters every week, right? And I can tell you that the cost of lawyer fees for VideoLAN is less than ten thousand dollars per year, right? Right? So that’s not really scary.
You have to realize that one of the good things, for example in France or most of Europe, is that the—Answering to a court order does not make you bankrupt, right? It’s not like in the US, where it can actually bankrupt you, right? There is—The way the law system works is that, like I receive lawyers’ letters every week, right? And I can tell you that the cost of lawyer fees for VideoLAN is less than ten thousand dollars per year, right? Right? So that’s not really scary.
Lex Fridman
I mean, similar with Pavel. The intelligence agencies tried to like say, “Can you put a backdoor in VLC?”
I mean, similar with Pavel. The intelligence agencies tried to like say, “Can you put a backdoor in VLC?”
Jean-Baptiste Kempf
Yes. Two of them.
Yes. Two of them.
Lex Fridman
What, what do you say?
What, what do you say?
Jean-Baptiste Kempf
No. Well, I was a lot less polite.
No. Well, I was a lot less polite.
Lex Fridman
I see… Yeah, yeah. You’re basically saying, “Hell no.”
I see… Yeah, yeah. You’re basically saying, “Hell no.”
Jean-Baptiste Kempf
Like, if we had to compromise our software, we would shut it down. This is clear.
Like, if we had to compromise our software, we would shut it down. This is clear.
Lex Fridman
And what’s the definition of compromise? Like allowing a government to do a backdoor-
And what’s the definition of compromise? Like allowing a government to do a backdoor-
Jean-Baptiste Kempf
There is no code that gets into VLC that we don’t control, and the way we compile VLC, you would call me completely paranoid. Like, we compile on boxes that are offline, where we start by compiling the compiler. We do everything offline on places that have never been connected to the internet. We— The way we do signing, there is double signature. And especially because, for example, we’ve seen, and we believe it’s a governmental agency that is not from the Western world who tried to push a fake binary into our own servers and that scared us a lot. And VideoLAN is open source. How can you kill it? Like, I move to where? I move to Malta.
There is no code that gets into VLC that we don’t control, and the way we compile VLC, you would call me completely paranoid. Like, we compile on boxes that are offline, where we start by compiling the compiler. We do everything offline on places that have never been connected to the internet. We— The way we do signing, there is double signature. And especially because, for example, we’ve seen, and we believe it’s a governmental agency that is not from the Western world who tried to push a fake binary into our own servers and that scared us a lot. And VideoLAN is open source. How can you kill it? Like, I move to where? I move to Malta.
Jean-Baptiste Kempf
I move to, I don’t know, Cayman Islands, and I change the domain name, and I start again, right? Like, VLC is a tool. It’s a tool that is going to help people doing things. We are not a platform. And for patents, well, I’m sorry, but most of the patents… Like, you shouldn’t be able to patent math and matrices. Like, this is wrong.
I move to, I don’t know, Cayman Islands, and I change the domain name, and I start again, right? Like, VLC is a tool. It’s a tool that is going to help people doing things. We are not a platform. And for patents, well, I’m sorry, but most of the patents… Like, you shouldn’t be able to patent math and matrices. Like, this is wrong.
Lex Fridman
So does VLC ever, like, censor the kind of videos it can play and not based on the content of the video?
So does VLC ever, like, censor the kind of videos it can play and not based on the content of the video?
Jean-Baptiste Kempf
No, never. We never do that. Because VLC is completely offline. It doesn’t talk to any server, so we don’t know anything that you’re using the software for.
No, never. We never do that. Because VLC is completely offline. It doesn’t talk to any server, so we don’t know anything that you’re using the software for.
Lex Fridman
So again, there’s no government that can say, you know, like the French government come in and say, “We don’t want… I think anime is destructive to society. We don’t want any anime; not allowed to be…”
So again, there’s no government that can say, you know, like the French government come in and say, “We don’t want… I think anime is destructive to society. We don’t want any anime; not allowed to be…”
Jean-Baptiste Kempf
No, they cannot, they cannot do that. And also what they tried is to say, “Hey, I want to know if that person watched that type of video.” And the answer is like, “No idea.”
No, they cannot, they cannot do that. And also what they tried is to say, “Hey, I want to know if that person watched that type of video.” And the answer is like, “No idea.”
Lex Fridman
So no on that too. So for surveillance, no.
So no on that too. So for surveillance, no.
Jean-Baptiste Kempf
No, no, because the only infrastructure we have is a downloading infrastructure. There is no telemetry in VLC, right?
No, no, because the only infrastructure we have is a downloading infrastructure. There is no telemetry in VLC, right?
Lex Fridman
It would be difficult ’cause of the international nature. It would be difficult for you to incorporate that code because there would be someone in the UK and someone in Germany and someone in the US as part of VideoLAN who’d be able to see that. It would be extremely difficult.
It would be difficult ’cause of the international nature. It would be difficult for you to incorporate that code because there would be someone in the UK and someone in Germany and someone in the US as part of VideoLAN who’d be able to see that. It would be extremely difficult.
Jean-Baptiste Kempf
The only thing that we can do, which happened, is like we had the issue— We had the case with some police in the US who said, “We have a murder case,” right? And the file is destructed or doesn’t play in that version of VLC. Could you help us?” Right? We never have access to the video. It’s like a normal support, right?
The only thing that we can do, which happened, is like we had the issue— We had the case with some police in the US who said, “We have a murder case,” right? And the file is destructed or doesn’t play in that version of VLC. Could you help us?” Right? We never have access to the video. It’s like a normal support, right?
Lex Fridman
Oh, it’s really about playing the file?
Oh, it’s really about playing the file?
Jean-Baptiste Kempf
Yes. And, like, I remember in the middle of the Afghan War, right? I received an email from someone in the army, right? I don’t remember the grades, right? It was just like, “We have a big issue with the latest version of VLC because it doesn’t play correctly the file on an RTSP server that we have where there is all the movies.” And he says VLC is very important for the morale on the troop on the ground, right? Because at night I think it might be boring, right? So they have a collection of videos to watch or movies over there, right? So and, and of course I did an update, and I broke some support of RTSP, right? So I gave them another version just for them, right? Because it was important.
Yes. And, like, I remember in the middle of the Afghan War, right? I received an email from someone in the army, right? I don’t remember the grades, right? It was just like, “We have a big issue with the latest version of VLC because it doesn’t play correctly the file on an RTSP server that we have where there is all the movies.” And he says VLC is very important for the morale on the troop on the ground, right? Because at night I think it might be boring, right? So they have a collection of videos to watch or movies over there, right? So and, and of course I did an update, and I broke some support of RTSP, right? So I gave them another version just for them, right? Because it was important.
Jean-Baptiste Kempf
And because VLC is completely open source, I think it is allowed on the US Army laptops, right? Because the, I guess someone in the US military actually looked at it and say, “Well, okay, this is okay,” right? And the way we document how we process, that was okay, right? So the only way we work with authorities is to help them doing support.
And because VLC is completely open source, I think it is allowed on the US Army laptops, right? Because the, I guess someone in the US military actually looked at it and say, “Well, okay, this is okay,” right? And the way we document how we process, that was okay, right? So the only way we work with authorities is to help them doing support.
Lex Fridman
That’s amazing. That’s an amazing story. Yeah.
That’s amazing. That’s an amazing story. Yeah.
Jean-Baptiste Kempf
We don’t see anything happening on how people use VLC, and this is strong.
We don’t see anything happening on how people use VLC, and this is strong.
Lex Fridman
Do you feel the stress of this? So first of all, millions of people using it. Second of all, the military using it. Maybe sometimes pressure from governments. Does that… That’s a small team, right?
Do you feel the stress of this? So first of all, millions of people using it. Second of all, the military using it. Maybe sometimes pressure from governments. Does that… That’s a small team, right?
Jean-Baptiste Kempf
Yeah, but-
Yeah, but-
Lex Fridman
How big is VLC core… how many?
How big is VLC core… how many?
Jean-Baptiste Kempf
Six, eight. But everything legally is only me. Everything that is legal is only me.
Six, eight. But everything legally is only me. Everything that is legal is only me.
Lex Fridman
You’re not stressed about this?
You’re not stressed about this?
Jean-Baptiste Kempf
I used to stress about that a lot. But the thing is, we’re doing what we can for everyone, for the greater good. We work so that we make some extremely complex technology easy for everyone. We’re a tool, and every tool is going to be used for great things and for bad things, right? You cannot blame a tool, I think. And this is, like, very important for us. I used to be in a lot of stress. I’m not anymore, right?
I used to stress about that a lot. But the thing is, we’re doing what we can for everyone, for the greater good. We work so that we make some extremely complex technology easy for everyone. We’re a tool, and every tool is going to be used for great things and for bad things, right? You cannot blame a tool, I think. And this is, like, very important for us. I used to be in a lot of stress. I’m not anymore, right?
Lex Fridman
What’s the secret to your zen? I mean, over and over in the chats I’ve had with you in the conversation today about every even tense topic, you’re very zen. What’s the source of zen?
What’s the secret to your zen? I mean, over and over in the chats I’ve had with you in the conversation today about every even tense topic, you’re very zen. What’s the source of zen?
Jean-Baptiste Kempf
I have a way of thinking about what is the worst case scenario, always, right? And the answer is, at the end, if I take like a chess player, right? In the end, am I dead? Yes or no? Right? And I do that nonstop, right?
I have a way of thinking about what is the worst case scenario, always, right? And the answer is, at the end, if I take like a chess player, right? In the end, am I dead? Yes or no? Right? And I do that nonstop, right?
Jean-Baptiste Kempf
And that’s also how I do my startups, right? Is that I’m here to get something right. What is the worst case? It goes bankrupt. That’s life. A company lives, a company dies. That’s okay, right? Like, and so my moral way is always like, am I dying in the end? Am I hurting someone? If the answer is no, then too bad, right? Like, oh, some lawyers are going to be unhappy. What are they going to do? Take all the money of VideoLAN? Wow. They’re going to have 50 grand. Amazing, right? What are they going to do with that? The code– the source code is out there. It’s not stoppable. Also because what we do is good and it’s done for everyone.
And that’s also how I do my startups, right? Is that I’m here to get something right. What is the worst case? It goes bankrupt. That’s life. A company lives, a company dies. That’s okay, right? Like, and so my moral way is always like, am I dying in the end? Am I hurting someone? If the answer is no, then too bad, right? Like, oh, some lawyers are going to be unhappy. What are they going to do? Take all the money of VideoLAN? Wow. They’re going to have 50 grand. Amazing, right? What are they going to do with that? The code– the source code is out there. It’s not stoppable. Also because what we do is good and it’s done for everyone.
Video archiving
Lex Fridman
That’s beautiful. Kieran, you said that there’s an active archiving preservation community? I think that’s super fascinating. You wrote that they’re stretched in budget, but they see the extreme importance of FFmpeg as a Rosetta Stone so that multimedia can be played a thousand years from now. I mean, that’s a beautiful way to see FFmpeg and VLC as a tool for preserving visual knowledge.
That’s beautiful. Kieran, you said that there’s an active archiving preservation community? I think that’s super fascinating. You wrote that they’re stretched in budget, but they see the extreme importance of FFmpeg as a Rosetta Stone so that multimedia can be played a thousand years from now. I mean, that’s a beautiful way to see FFmpeg and VLC as a tool for preserving visual knowledge.
Kieran Kunhya
Yes, that’s right. One of the coolest communities in open source multimedia, mainly led by someone called Dave Rice, I’ll give him a shout-out, I think from City University of New York, is the archiving community. They’ve done so much stuff. They value open source, one, because yes, they lack budgets, but two, they see the fact that archiving video is important for the world, and being able to play that is a big problem. Famously in the UK, there was something called the New Domesday Book, and they archived lots of stuff on BBC microcomputers. Within 10 to 15 years, no one had the right software to play that.
Yes, that’s right. One of the coolest communities in open source multimedia, mainly led by someone called Dave Rice, I’ll give him a shout-out, I think from City University of New York, is the archiving community. They’ve done so much stuff. They value open source, one, because yes, they lack budgets, but two, they see the fact that archiving video is important for the world, and being able to play that is a big problem. Famously in the UK, there was something called the New Domesday Book, and they archived lots of stuff on BBC microcomputers. Within 10 to 15 years, no one had the right software to play that.
Kieran Kunhya
I think it was 20 years or something like that, and someone had to go and reverse engineer this, and that was like 20 years. Imagine that in a thousand years. I think one of the great things about FFmpeg is it’s written in C. C is the closest to mathematics you’re probably gonna get. The closest to logic is-
I think it was 20 years or something like that, and someone had to go and reverse engineer this, and that was like 20 years. Imagine that in a thousand years. I think one of the great things about FFmpeg is it’s written in C. C is the closest to mathematics you’re probably gonna get. The closest to logic is-
Jean-Baptiste Kempf
Do you think in 1,000 years we’d still have C compilers?
Do you think in 1,000 years we’d still have C compilers?
Kieran Kunhya
Yes. We have languages that exist that haven’t changed too much. We have mathematical notation that exists. It will be like Latin. C will be like Latin. It will be a thing that you learn from the past, but it will still be usable in certain contexts. So the archiving community are really great practically. They, again, limited funds. They funded the development of the FFV1 codec, so that’s a lossless codec. So the archiving community is really scared about the act of compression losing things, and they have a fair point in this, you know.
Yes. We have languages that exist that haven’t changed too much. We have mathematical notation that exists. It will be like Latin. C will be like Latin. It will be a thing that you learn from the past, but it will still be usable in certain contexts. So the archiving community are really great practically. They, again, limited funds. They funded the development of the FFV1 codec, so that’s a lossless codec. So the archiving community is really scared about the act of compression losing things, and they have a fair point in this, you know.
Kieran Kunhya
If they compress too hard, it could change the view of the material. There could be something slightly different here and there, so they’re really concerned about things. They need to be not just compressed well, but lossless and be fast. And so they worked with FFmpeg to develop a whole new codec designed for fast software-based encoding. They’re really concerned about resilience, so if they’re storing on tapes or other hard disks, I lose some bits, I need to recover quickly. I can’t lose a whole GOP because I’ve lost a bit-
If they compress too hard, it could change the view of the material. There could be something slightly different here and there, so they’re really concerned about things. They need to be not just compressed well, but lossless and be fast. And so they worked with FFmpeg to develop a whole new codec designed for fast software-based encoding. They’re really concerned about resilience, so if they’re storing on tapes or other hard disks, I lose some bits, I need to recover quickly. I can’t lose a whole GOP because I’ve lost a bit-
Kieran Kunhya
… something like that. So they’re a really great bunch of people. They funded GPU encoding in FFmpeg to make FFV1 encode faster. And it’s really about preserving the world’s multimedia heritage in a way that’s usable, and there’s a lot of great teams and a lot of archival groups across the world who’ve chosen FFmpeg and FFV1 as their archiving solution. And they can really provide us also super specialist advice. They can- … explain, “Ah, in the 1950s, colorimetry was done like this on this certain type of tape, and so there is this special case that you need to handle, and you’ll never get this anywhere else.”
… something like that. So they’re a really great bunch of people. They funded GPU encoding in FFmpeg to make FFV1 encode faster. And it’s really about preserving the world’s multimedia heritage in a way that’s usable, and there’s a lot of great teams and a lot of archival groups across the world who’ve chosen FFmpeg and FFV1 as their archiving solution. And they can really provide us also super specialist advice. They can- … explain, “Ah, in the 1950s, colorimetry was done like this on this certain type of tape, and so there is this special case that you need to handle, and you’ll never get this anywhere else.”
Jean-Baptiste Kempf
You see, they know things on video that we don’t. Like, every time I talk to, was it Dave-
You see, they know things on video that we don’t. Like, every time I talk to, was it Dave-
Kieran Kunhya
Dave Rice
Dave Rice
Jean-Baptiste Kempf
… or, or the people from the British, uh-
… or, or the people from the British, uh-
Kieran Kunhya
British Film, uh
British Film, uh
Jean-Baptiste Kempf
… Film, it’s just like every time I just learn something new, and I’ve been doing video for 20 years. They have, especially on colorimetry and colors.
… Film, it’s just like every time I just learn something new, and I’ve been doing video for 20 years. They have, especially on colorimetry and colors.
Kieran Kunhya
Storage, these other things.
Storage, these other things.
Lex Fridman
I mean, they have a deep, deep appreciation of the content itself, of the video itself. And like, especially when you’re thinking of lossless, they’re terrified of losing something essential- … about the thing, and in so doing, they’re deeply understanding the thing that is to be preserved, which you sometimes might not be thinking about when you’re-
I mean, they have a deep, deep appreciation of the content itself, of the video itself. And like, especially when you’re thinking of lossless, they’re terrified of losing something essential- … about the thing, and in so doing, they’re deeply understanding the thing that is to be preserved, which you sometimes might not be thinking about when you’re-
Kieran Kunhya
Absolutely.
Absolutely.
Lex Fridman
… obsessing about the actual technology of the encoding and so on.
… obsessing about the actual technology of the encoding and so on.
Jean-Baptiste Kempf
And when you enter the rabbit hole of film scanners, right? So you take those- … those things to make to digital, like, it’s like- … a huge topic that, like, would take another five hours of podcast- … just on that topic.
And when you enter the rabbit hole of film scanners, right? So you take those- … those things to make to digital, like, it’s like- … a huge topic that, like, would take another five hours of podcast- … just on that topic.
Kieran Kunhya
On film, and there’s a lot of film that needs to be archived. Film is degrading. It’s maybe not stored in the right environment. The other thing is they can… What they also do is, because it’s open source, they give this away, their workflows, to countries who can’t afford to have archiving institutions, where archiving is done by volunteers, it’s done by other things. They go and teach, you know, in India, they teach children to do FFmpeg commands. They’re really great. They’re really, they’re really the model community, the model ethos of what we’re trying to achieve. They are such a great bunch of people, so interested in participating and being part of something much bigger because they realize the work they’re doing in a thousand years is gonna tell a lot.
On film, and there’s a lot of film that needs to be archived. Film is degrading. It’s maybe not stored in the right environment. The other thing is they can… What they also do is, because it’s open source, they give this away, their workflows, to countries who can’t afford to have archiving institutions, where archiving is done by volunteers, it’s done by other things. They go and teach, you know, in India, they teach children to do FFmpeg commands. They’re really great. They’re really, they’re really the model community, the model ethos of what we’re trying to achieve. They are such a great bunch of people, so interested in participating and being part of something much bigger because they realize the work they’re doing in a thousand years is gonna tell a lot.
Kieran Kunhya
You know, in a thousand years we may be drowning in AI slop. This stuff needs to be important and, you know, archived well. What was life like?
You know, in a thousand years we may be drowning in AI slop. This stuff needs to be important and, you know, archived well. What was life like?
Lex Fridman
Yeah, it feels like capturing the 20th century and the 21st century is essential because it feels like a transition point, where we went from scarcity of data to slop- … oceans of slop, and that transition point is good to archive.
Yeah, it feels like capturing the 20th century and the 21st century is essential because it feels like a transition point, where we went from scarcity of data to slop- … oceans of slop, and that transition point is good to archive.
Kieran Kunhya
It’s important, yeah.
It’s important, yeah.
Jean-Baptiste Kempf
But people don’t realize we are losing today a ton of films. There is a ton of things from the ’30s, from the ’40s, and the ’50s that where there is no value-
But people don’t realize we are losing today a ton of films. There is a ton of things from the ’30s, from the ’40s, and the ’50s that where there is no value-
Kieran Kunhya
And tape. ’70s and ’80s, there’s tape, and there’s not enough tape heads in the world-
And tape. ’70s and ’80s, there’s tape, and there’s not enough tape heads in the world-
Jean-Baptiste Kempf
To read all the tapes.
To read all the tapes.
Kieran Kunhya
… left to redo, so they have to decide what they want to archive and throw away the rest of the tapes. There’s huge moral hazard, I guess for want of a better phrase, around this topic because this is a digital record of human history and they have to make decisions that… And there’s digital stewardship, I suppose, for want of… I made that phrase up. That’s not a real phrase. To make sure the world can have this information in something that’s playable by everybody, not- … playable on some device that, well, it doesn’t exist anymore.
… left to redo, so they have to decide what they want to archive and throw away the rest of the tapes. There’s huge moral hazard, I guess for want of a better phrase, around this topic because this is a digital record of human history and they have to make decisions that… And there’s digital stewardship, I suppose, for want of… I made that phrase up. That’s not a real phrase. To make sure the world can have this information in something that’s playable by everybody, not- … playable on some device that, well, it doesn’t exist anymore.
Lex Fridman
And then there’s like, realistically speaking, there’s a needle in a haystack where there’s a lot of value in archiving all that footage, and then over time finding the gems- … that we don’t know are there.
And then there’s like, realistically speaking, there’s a needle in a haystack where there’s a lot of value in archiving all that footage, and then over time finding the gems- … that we don’t know are there.
Kieran Kunhya
Hey, there was something in that corner that we just didn’t- And that would’ve been compressed away because it was some little thing. Oh, wow, there’s something there.
Hey, there was something in that corner that we just didn’t- And that would’ve been compressed away because it was some little thing. Oh, wow, there’s something there.
Lex Fridman
That’s it.
That’s it.
Kieran Kunhya
And that’s… They’ve made sure that it’s lossless. They can prove mathematically that it’s lossless. They can run different trade-offs for if there’s bit… if they lose a bit, a single bit flips, I can make sure that I only lose a portion of a given frame. We can do error recovery on previous frames. They can do all sorts of different things.
And that’s… They’ve made sure that it’s lossless. They can prove mathematically that it’s lossless. They can run different trade-offs for if there’s bit… if they lose a bit, a single bit flips, I can make sure that I only lose a portion of a given frame. We can do error recovery on previous frames. They can do all sorts of different things.
Future of FFmpeg and VLC
Lex Fridman
Do you think VLC and FFmpeg will be here 100 years from now?
Do you think VLC and FFmpeg will be here 100 years from now?
Jean-Baptiste Kempf
FFmpeg, yes.
FFmpeg, yes.
Kieran Kunhya
Yep, FFmpeg, yes.
Yep, FFmpeg, yes.
Jean-Baptiste Kempf
VLC, maybe.
VLC, maybe.
Lex Fridman
What’s the future of… Where is FFmpeg going? Where is VLC going? Like in the next… If you think about, like, five years, 10 years, 20 years.
What’s the future of… Where is FFmpeg going? Where is VLC going? Like in the next… If you think about, like, five years, 10 years, 20 years.
Jean-Baptiste Kempf
Five years, 10 years is easy. The question is after that, right? The question is- … do we arrive at something called holograms, right?
Five years, 10 years is easy. The question is after that, right? The question is- … do we arrive at something called holograms, right?
Lex Fridman
Yeah, so will VLC and FFmpeg expand- … to whatever-
Yeah, so will VLC and FFmpeg expand- … to whatever-
Jean-Baptiste Kempf
Multimedia
Multimedia
Lex Fridman
… multimedia- … so multimedia might become, I’m sorry for the pothead expansion of topic, but, you know, if you look at something like Neuralink with brain-computer interfaces, it’s very possible that we start to consume what multimedia means is whatever codec, whatever data that our brain wants to consume through the brain-computer interfaces. That’s one. Then virtual reality, of course.
… multimedia- … so multimedia might become, I’m sorry for the pothead expansion of topic, but, you know, if you look at something like Neuralink with brain-computer interfaces, it’s very possible that we start to consume what multimedia means is whatever codec, whatever data that our brain wants to consume through the brain-computer interfaces. That’s one. Then virtual reality, of course.
Jean-Baptiste Kempf
You will have VLC for Neuralink.
You will have VLC for Neuralink.
Kieran Kunhya
Yep, and you’ll have FFmpeg -i input format human brain.
Yep, and you’ll have FFmpeg -i input format human brain.
Lex Fridman
Yeah. There’s gonna be codecs for the brain.
Yeah. There’s gonna be codecs for the brain.
Kieran Kunhya
Sure, 100%. Yeah, to compress neural information, yeah.
Sure, 100%. Yeah, to compress neural information, yeah.
Jean-Baptiste Kempf
I mean, today there is like, there are new codecs for- … for example, what we call point cloud, right? Or volumetric videos, right? There is a ton of research on what we call RGB-D, right? So codecs for depth that is useful for robotics and for 3D things. There is a ton of codecs for compression of 3D elements.
I mean, today there is like, there are new codecs for- … for example, what we call point cloud, right? Or volumetric videos, right? There is a ton of research on what we call RGB-D, right? So codecs for depth that is useful for robotics and for 3D things. There is a ton of codecs for compression of 3D elements.
Kieran Kunhya
Compression for astronomy.
Compression for astronomy.
Jean-Baptiste Kempf
For example, on VLC, we also have already a VR and XR version of VLC. And also on Kyber, right? We talk about Kyber. On Kyber, we also like do streaming of XR content on, for the glasses who cannot have enough power or inside the Apple Vision or the Quest. So we already work on streaming 3D, XR, interactive, low latency. There is something called volumetric video, point cloud videos, so it’s not stopping. And yes, at some point it will manage 3D data inside VLC and FFmpeg, right? It’s obvious.
For example, on VLC, we also have already a VR and XR version of VLC. And also on Kyber, right? We talk about Kyber. On Kyber, we also like do streaming of XR content on, for the glasses who cannot have enough power or inside the Apple Vision or the Quest. So we already work on streaming 3D, XR, interactive, low latency. There is something called volumetric video, point cloud videos, so it’s not stopping. And yes, at some point it will manage 3D data inside VLC and FFmpeg, right? It’s obvious.
Lex Fridman
So that’s where it is moving, like the community is open.
So that’s where it is moving, like the community is open.
Jean-Baptiste Kempf
Not everyone in the community sees that, but as Kieran and I, we are entrepreneurs, we know where it’s going. We see that, right?
Not everyone in the community sees that, but as Kieran and I, we are entrepreneurs, we know where it’s going. We see that, right?
Lex Fridman
So I suppose that there is a tension probably inside FFmpeg. It’s like, “Hey, listen, folks, we’re really good at doing video and audio, so like why expand? Like let’s do the thing we’re really good at doing.”
So I suppose that there is a tension probably inside FFmpeg. It’s like, “Hey, listen, folks, we’re really good at doing video and audio, so like why expand? Like let’s do the thing we’re really good at doing.”
Jean-Baptiste Kempf
In order to answer that question, we need to answer the definition of what is multimedia. And multimedia is a digital representation of several streams for the human senses. And we will do that, right? So imagine there is now a way to not have a mic, but have a odor sensor- … and a diffuser of odors. It will get into FFmpeg.
In order to answer that question, we need to answer the definition of what is multimedia. And multimedia is a digital representation of several streams for the human senses. And we will do that, right? So imagine there is now a way to not have a mic, but have a odor sensor- … and a diffuser of odors. It will get into FFmpeg.
Lex Fridman
So your demuxer is coming up.
So your demuxer is coming up.
Jean-Baptiste Kempf
Yes. Yes. Of course, your demuxer has a new track type that is basically odors, right? And you already have-
Yes. Yes. Of course, your demuxer has a new track type that is basically odors, right? And you already have-
Lex Fridman
Smell, touch.
Smell, touch.
Kieran Kunhya
It’s like audio. You’ll have a left and right nose track. You have a left and right audio pair. It’s easy.
It’s like audio. You’ll have a left and right nose track. You have a left and right audio pair. It’s easy.
Jean-Baptiste Kempf
Yes, of course.
Yes, of course.
Lex Fridman
Stereo smell.
Stereo smell.
Kieran Kunhya
Stereo smell, yeah.
Stereo smell, yeah.
Jean-Baptiste Kempf
So in VLC, for example, we already have a plugin for haptic. It’s mostly for what we call 4D cinemas, right? You know, those ones on hydraulic—I don’t know how you say that—all the hydraulic-
So in VLC, for example, we already have a plugin for haptic. It’s mostly for what we call 4D cinemas, right? You know, those ones on hydraulic—I don’t know how you say that—all the hydraulic-
Kieran Kunhya
Hydraulic arms. Hydraulic,
Hydraulic arms. Hydraulic,
Jean-Baptiste Kempf
Arms. And where everything is moving, like you have in theme parks, right? And there is a data feed synchronized where, which is basically transporting this information.
Arms. And where everything is moving, like you have in theme parks, right? And there is a data feed synchronized where, which is basically transporting this information.
Lex Fridman
Is there yet a standard for that?
Is there yet a standard for that?
Jean-Baptiste Kempf
There are many standards, right?
There are many standards, right?
Lex Fridman
This is… You make me so happy.
This is… You make me so happy.
Jean-Baptiste Kempf
And so of course, like we have a plugin which is not in the normal version of VLC-
And so of course, like we have a plugin which is not in the normal version of VLC-
Lex Fridman
That’s good.
That’s good.
Jean-Baptiste Kempf
… that is basically transporting those type of movements, which is physical movements, which is haptic movements, right? It is a human sense, so it will get in.
… that is basically transporting those type of movements, which is physical movements, which is haptic movements, right? It is a human sense, so it will get in.
Lex Fridman
That’s such an exciting future. Was it… I mean, it’s a small community of developers. How do you pull that off? Like if you’re a contributor to FFmpeg or VLC, it feels stressful. Like it, just looking on Twitter, it’s like a huge amount of work to make it work on all these different operating systems, an incredible effort.
That’s such an exciting future. Was it… I mean, it’s a small community of developers. How do you pull that off? Like if you’re a contributor to FFmpeg or VLC, it feels stressful. Like it, just looking on Twitter, it’s like a huge amount of work to make it work on all these different operating systems, an incredible effort.
Jean-Baptiste Kempf
No, see it in the other direction. We are not the contributors. We are the maintainers, right? So we maintain for everyone. Meaning that, for example, every year there is around 150 people who contribute to VLC and maybe 300 on FFmpeg, right? Our goal as a small team is to get all the contribution in. So if there is more usage, there will be more contributions, and those people will do the right module, the new format, and so on. We care about the architecture of VLC, the architecture of FFmpeg, right? Now we’re doing things in VLC, which is spatial audio, right? We did the demo not long ago. There was changes needed on the architecture, and we did the first spatial audio module.
No, see it in the other direction. We are not the contributors. We are the maintainers, right? So we maintain for everyone. Meaning that, for example, every year there is around 150 people who contribute to VLC and maybe 300 on FFmpeg, right? Our goal as a small team is to get all the contribution in. So if there is more usage, there will be more contributions, and those people will do the right module, the new format, and so on. We care about the architecture of VLC, the architecture of FFmpeg, right? Now we’re doing things in VLC, which is spatial audio, right? We did the demo not long ago. There was changes needed on the architecture, and we did the first spatial audio module.
Jean-Baptiste Kempf
When it’s going to add the second one, it’s going to be easy, or the third one is going to be easy, right? Our goal, and it’s going to be the same for others or haptic, right? We need to work the architecture so that modules can be added to add future capabilities. So yes, we are going… We are a multimedia framework, so that’s not just audio and video. It’s everything that is timed and represents something that you can sense. And if it’s brainwaves, it’s going to be brainwaves.
When it’s going to add the second one, it’s going to be easy, or the third one is going to be easy, right? Our goal, and it’s going to be the same for others or haptic, right? We need to work the architecture so that modules can be added to add future capabilities. So yes, we are going… We are a multimedia framework, so that’s not just audio and video. It’s everything that is timed and represents something that you can sense. And if it’s brainwaves, it’s going to be brainwaves.
Kieran Kunhya
I think that’s inevitable. Sorry.
I think that’s inevitable. Sorry.
Lex Fridman
I love this on so many fronts because, so FFmpeg and VLC are pushing companies and pushing the world to standardize. So for example, to standardize- … brainwaves, right? So standardize… It would push, like I hope Neuralink comes up with a standard for multimedia via brain-computer interfaces or for robots with haptic.
I love this on so many fronts because, so FFmpeg and VLC are pushing companies and pushing the world to standardize. So for example, to standardize- … brainwaves, right? So standardize… It would push, like I hope Neuralink comes up with a standard for multimedia via brain-computer interfaces or for robots with haptic.
Jean-Baptiste Kempf
By experience, what happens is always the same, right? You start, it’s a new topic. There is like five different standards because everyone starts to do this. The hype goes down because every time the hype goes down, then people start to say, “Well, you know what? We need to do a standard.” People, because two or three companies, usually not the leader, but the two or three followers do a standard, and then we implement the standard and then it’s the end of the curve. It starts to be more pepper.
By experience, what happens is always the same, right? You start, it’s a new topic. There is like five different standards because everyone starts to do this. The hype goes down because every time the hype goes down, then people start to say, “Well, you know what? We need to do a standard.” People, because two or three companies, usually not the leader, but the two or three followers do a standard, and then we implement the standard and then it’s the end of the curve. It starts to be more pepper.
Lex Fridman
And then the leader’s kind of pressured into it because it is better to do a standard. Yeah.
And then the leader’s kind of pressured into it because it is better to do a standard. Yeah.
Jean-Baptiste Kempf
Example, 3D audio, right? Six or seven years ago, it was everything about 3D. You go, you had the cardboard on Android. You had two audio formats. They’re all dead, right? And now it’s coming back with actual use cases, and we learn from the mistakes of the past standard. So it will be the same everywhere.
Example, 3D audio, right? Six or seven years ago, it was everything about 3D. You go, you had the cardboard on Android. You had two audio formats. They’re all dead, right? And now it’s coming back with actual use cases, and we learn from the mistakes of the past standard. So it will be the same everywhere.
Lex Fridman
And not try to avoid closed? I saw somewhere you didn’t have too many nice things to say about Dolby.
And not try to avoid closed? I saw somewhere you didn’t have too many nice things to say about Dolby.
Jean-Baptiste Kempf
No, I don’t.
No, I don’t.
Lex Fridman
What is… can you educate me on why, where they went, what did they do bad that made you mad?
What is… can you educate me on why, where they went, what did they do bad that made you mad?
Jean-Baptiste Kempf
It’s used to be an amazing company doing tons of great things with amazing engineers. They defined what sound was. And now it’s mostly-
It’s used to be an amazing company doing tons of great things with amazing engineers. They defined what sound was. And now it’s mostly-
Kieran Kunhya
Lawyers.
Lawyers.
Jean-Baptiste Kempf
… lawyers and licensing things.
… lawyers and licensing things.
Lex Fridman
Oh, so they’re, yeah, it’s, they’re, they’re closing stuff off. They’re trying to make money on licensing.
Oh, so they’re, yeah, it’s, they’re, they’re closing stuff off. They’re trying to make money on licensing.
Jean-Baptiste Kempf
No, it’s just like they, they don’t innovate as much as they did- … and so on. It’s a bit like I’m sorry to say, right, like HP, right?
No, it’s just like they, they don’t innovate as much as they did- … and so on. It’s a bit like I’m sorry to say, right, like HP, right?
Kieran Kunhya
Very true.
Very true.
Lex Fridman
Oh, since we talked about Twitter a bunch in a bunch of different contexts, do you have a favorite, and least favorite, most embarrassing tweet on either VideoLAN or FFmpeg Twitters?
Oh, since we talked about Twitter a bunch in a bunch of different contexts, do you have a favorite, and least favorite, most embarrassing tweet on either VideoLAN or FFmpeg Twitters?
Kieran Kunhya
The two, my two favorites are, “Talk is cheap, send patches.” I think that, that- … embodies a lot of the stuff doesn’t get, as, as we’ve talked about, stuff doesn’t get built unless someone does it. It doesn’t just appear from the ether. The other one that I like is “FFmpeg, nothing is beyond our reach.” I think that comes from a US military satellite patch where I think they, they invented some kind of monitoring system that could see the whole world, and this was released.
The two, my two favorites are, “Talk is cheap, send patches.” I think that, that- … embodies a lot of the stuff doesn’t get, as, as we’ve talked about, stuff doesn’t get built unless someone does it. It doesn’t just appear from the ether. The other one that I like is “FFmpeg, nothing is beyond our reach.” I think that comes from a US military satellite patch where I think they, they invented some kind of monitoring system that could see the whole world, and this was released.
Jean-Baptiste Kempf
Wasn’t there something where FFmpeg was running on a rover on Mars also?
Wasn’t there something where FFmpeg was running on a rover on Mars also?
Kieran Kunhya
Yeah, so FFmpeg is used by the Mars rover, the Mars 2020 rover to compress pictures. They really wanted—they wrote a paper about it, and they really wanted to use as much commercial off-the-shelf technology as possible.
Yeah, so FFmpeg is used by the Mars rover, the Mars 2020 rover to compress pictures. They really wanted—they wrote a paper about it, and they really wanted to use as much commercial off-the-shelf technology as possible.
Lex Fridman
Oh, that’s cool.
Oh, that’s cool.
Kieran Kunhya
FFmpeg runs on Mars, so we are, we are a multi-planetary open source library.
FFmpeg runs on Mars, so we are, we are a multi-planetary open source library.
Jean-Baptiste Kempf
Very often we’ve seen- … Tweets for people using VLC in weird places. A lot of the people doing Formula 1 are in all the paddocks, they use VLC to play the live feed. We’ve seen the European Space Agency. We’ve seen SpaceX, like, monitoring the launches with VLC, and it fills you with joy, right? So-
Very often we’ve seen- … Tweets for people using VLC in weird places. A lot of the people doing Formula 1 are in all the paddocks, they use VLC to play the live feed. We’ve seen the European Space Agency. We’ve seen SpaceX, like, monitoring the launches with VLC, and it fills you with joy, right? So-
Lex Fridman
I’ve seen a particle accelerator.
I’ve seen a particle accelerator.
Jean-Baptiste Kempf
Oh, yeah, yeah. We had one of the most amazing thing that I went for was to go to the CERN at the LHC because they were using VLC to monitor all the captors on the ring because the ring is 27 kilometers. And so they had some analog cameras- … and they were using some of the capture cards to go to analog to VLC, so VLC could stream on their multicast network for the whole CERN to access to that. And, like, I visited that in 2010 with Laurent and, like, we fixed their issue in an hour or something like that, right? Because it was some parameters maybe not well documented at that time. And he said, “Okay, for the whole day, what do you want to do?” And we visited everything. Like… things where with antimatter and colliders and so on.
Oh, yeah, yeah. We had one of the most amazing thing that I went for was to go to the CERN at the LHC because they were using VLC to monitor all the captors on the ring because the ring is 27 kilometers. And so they had some analog cameras- … and they were using some of the capture cards to go to analog to VLC, so VLC could stream on their multicast network for the whole CERN to access to that. And, like, I visited that in 2010 with Laurent and, like, we fixed their issue in an hour or something like that, right? Because it was some parameters maybe not well documented at that time. And he said, “Okay, for the whole day, what do you want to do?” And we visited everything. Like… things where with antimatter and colliders and so on.
Lex Fridman
That’s awesome.
That’s awesome.
Jean-Baptiste Kempf
And that was, like, one of the most amazing days of my physics background.
And that was, like, one of the most amazing days of my physics background.
Lex Fridman
Yeah, it’s used, like, everywhere. Any tweets, Kieran, you regret?
Yeah, it’s used, like, everywhere. Any tweets, Kieran, you regret?
Kieran Kunhya
Tweets I regret?
Tweets I regret?
Lex Fridman
Or is it like that, how does the French song go? Regret nothing.
Or is it like that, how does the French song go? Regret nothing.
Kieran Kunhya
“Je ne regrette rien.” Yeah.
“Je ne regrette rien.” Yeah.
Jean-Baptiste Kempf
That’s very important for me, right? Don’t regret anything. No, it’s because regrets are a tax on your mind, right? So learn from your mistakes, but don’t regret. Because you’ve done it, so except if you have a, a time machine to go back in time, don’t regret, right? It’s going to just tax your brain. Learn from your mistake, sure. Don’t regret.
That’s very important for me, right? Don’t regret anything. No, it’s because regrets are a tax on your mind, right? So learn from your mistakes, but don’t regret. Because you’ve done it, so except if you have a, a time machine to go back in time, don’t regret, right? It’s going to just tax your brain. Learn from your mistake, sure. Don’t regret.
Lex Fridman
It’s like it reminds me, it’s beautiful. It’s a tax on your brain. It reminds me of the Johnny Depp quote I saw where he was saying, “Hate, you know, I don’t hate. That’s, hate is a very expensive emotion.”
It’s like it reminds me, it’s beautiful. It’s a tax on your brain. It reminds me of the Johnny Depp quote I saw where he was saying, “Hate, you know, I don’t hate. That’s, hate is a very expensive emotion.”
Jean-Baptiste Kempf
Are you comparing me to Johnny Depp?
Are you comparing me to Johnny Depp?
Lex Fridman
Uh-
Uh-
Jean-Baptiste Kempf
Because that would be your first one.
Because that would be your first one.
Lex Fridman
Well, gentlemen, like I said, I’m eternally grateful for the software that, you know, the two of you and the bigger community have been part of building with FFmpeg and VLC and everything else. I’m eternally grateful for the spicy tweets. Never stop. And I’m grateful that you would talk with me today and give me this sexy hat. I feel like a wizard. I feel special. And I feel special to get a chance to talk and celebrate the piece of software that brought me so much joy over the years. So thank you for everything, and thank you for talking today.
Well, gentlemen, like I said, I’m eternally grateful for the software that, you know, the two of you and the bigger community have been part of building with FFmpeg and VLC and everything else. I’m eternally grateful for the spicy tweets. Never stop. And I’m grateful that you would talk with me today and give me this sexy hat. I feel like a wizard. I feel special. And I feel special to get a chance to talk and celebrate the piece of software that brought me so much joy over the years. So thank you for everything, and thank you for talking today.
Kieran Kunhya
Thank you for having us.
Thank you for having us.
Jean-Baptiste Kempf
Thank you so much.
Thank you so much.
Lex Fridman
Thanks for listening to this conversation with Jean-Baptiste Kempf and Kieran Kunhya. To support this podcast, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now let me leave you with some words from the legendary Linus Torvalds. “Most good programmers do programming not because they expect to get paid or get adulation by the public, but because it is fun to program.” Thank you for listening, and I hope to see you next time.
Thanks for listening to this conversation with Jean-Baptiste Kempf and Kieran Kunhya. To support this podcast, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now let me leave you with some words from the legendary Linus Torvalds. “Most good programmers do programming not because they expect to get paid or get adulation by the public, but because it is fun to program.” Thank you for listening, and I hope to see you next time.
Transcript for Vikings, Ragnar, Berserkers, Valhalla & the Warriors of the Viking Age | Lex Fridman Podcast #495
This is a transcript of Lex Fridman Podcast #495 with Lars Brownworth.
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
- Go back to this episode’s main page
- Watch the full YouTube version of the podcast
Table of Contents
Here are the loose “chapters” in the conversation.
Click link to jump approximately to that part in the transcript:
Click link to jump approximately to that part in the transcript:
- 0:00 – Episode highlight
- 1:17 – Introduction
- 2:37 – The start of the Viking Age
- 12:30 – Viking military strategy, tactics & technology
- 26:13 – Ragnar Lothbrok
- 35:40 – The Great Heathen Army
- 40:23 – Rollo and Normandy
- 50:34 – Viking religion and Valhalla
- 1:01:06 – Viking explorers
- 1:06:13 – Vikings in North America
- 1:19:35 – Vikings in the East
- 1:39:14 – Byzantine Empire
- 1:47:57 – History and human nature
Episode highlight
Lars Brownworth
The Viking long ships could average 70 to 120 miles a day. They could hit a place, raid it, drag off whoever they wanted, and get away before you could get your army there. That’s just absolutely terrifying.
The Viking long ships could average 70 to 120 miles a day. They could hit a place, raid it, drag off whoever they wanted, and get away before you could get your army there. That’s just absolutely terrifying.
Lex Fridman
What do you think it felt like for Alcuin and the monks to see the Viking ships on the horizon?
What do you think it felt like for Alcuin and the monks to see the Viking ships on the horizon?
Lars Brownworth
Honestly, I think it’s the end of the world, and I don’t think they were wrong to think that. The Anglo-Saxon Chronicle says the night before Lindisfarne, the monks saw sheets of lightning in the sky in the shape of dragons, and this was obviously meant to foreshadow the dragon ships coming up. But if you were brave, then you got taken to the House of the Dead, which was Valhalla. Every day you would fight, and whatever wounds you got would be magically healed that night, and then the next morning, you’d get up and do it again, so you’re essentially practicing for Ragnarok, the final battle. You know, there’s this poem by Tennyson, Ulysses, my favorite poem. I think it captures the Viking spirit.
Honestly, I think it’s the end of the world, and I don’t think they were wrong to think that. The Anglo-Saxon Chronicle says the night before Lindisfarne, the monks saw sheets of lightning in the sky in the shape of dragons, and this was obviously meant to foreshadow the dragon ships coming up. But if you were brave, then you got taken to the House of the Dead, which was Valhalla. Every day you would fight, and whatever wounds you got would be magically healed that night, and then the next morning, you’d get up and do it again, so you’re essentially practicing for Ragnarok, the final battle. You know, there’s this poem by Tennyson, Ulysses, my favorite poem. I think it captures the Viking spirit.
Lars Brownworth
The last line of it is to strive, “To seek, to find, and not to yield.” I think that’s very much like the Viking, you know, my purpose holds, to sail beyond the baths of all the Western stars until I die. We may die, but I’m gonna do this, I’m not gonna yield.
The last line of it is to strive, “To seek, to find, and not to yield.” I think that’s very much like the Viking, you know, my purpose holds, to sail beyond the baths of all the Western stars until I die. We may die, but I’m gonna do this, I’m not gonna yield.
Introduction
Lex Fridman
The following is a conversation with Lars Brownworth, a historian and author of many excellent history books, including The Sea Wolves: A History of the Vikings and The Normans: From Raiders to Kings. He’s also the host of two history podcast series. The first, called 12 Byzantine Rulers: The History of the Byzantine Empire, is one of the first, if not the first ever, history podcasts launched over 20 years ago in June 2005. His second series, Norman Centuries, explores the remarkable rise of the Normans from Viking raiders to the rulers of kingdoms stretching from England to Sicily.
The following is a conversation with Lars Brownworth, a historian and author of many excellent history books, including The Sea Wolves: A History of the Vikings and The Normans: From Raiders to Kings. He’s also the host of two history podcast series. The first, called 12 Byzantine Rulers: The History of the Byzantine Empire, is one of the first, if not the first ever, history podcasts launched over 20 years ago in June 2005. His second series, Norman Centuries, explores the remarkable rise of the Normans from Viking raiders to the rulers of kingdoms stretching from England to Sicily.
Lex Fridman
In this conversation, we focus primarily on the Vikings, the seafaring Norse warriors and explorers who, over a period of just 300 years, reshaped the medieval world and the trajectory of Western civilization as we know it. This is a Lex Fridman Podcast. To support it, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Lars Brownworth.
In this conversation, we focus primarily on the Vikings, the seafaring Norse warriors and explorers who, over a period of just 300 years, reshaped the medieval world and the trajectory of Western civilization as we know it. This is a Lex Fridman Podcast. To support it, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Lars Brownworth.
The start of the Viking Age
Lex Fridman
Your writing and podcasts take us from the Vikings to the Normans to Crusades, to the collapse of the East Roman Empire, also known as the Byzantine Empire. There’s a thread, I think, that connects the Vikings through all of it, so let’s start at the beginning. Let’s start with the Vikings.
Your writing and podcasts take us from the Vikings to the Normans to Crusades, to the collapse of the East Roman Empire, also known as the Byzantine Empire. There’s a thread, I think, that connects the Vikings through all of it, so let’s start at the beginning. Let’s start with the Vikings.
Lex Fridman
So the age of the Vikings was intense and violent, as you write about, often dated from 793 AD to 1066 AD. It lasted less than three centuries. So the start is often dated to June 8th, 793. What happened on June 8th, 793?
So the age of the Vikings was intense and violent, as you write about, often dated from 793 AD to 1066 AD. It lasted less than three centuries. So the start is often dated to June 8th, 793. What happened on June 8th, 793?
Lars Brownworth
In June of 793, a group of Vikings, probably originating from Norway, arrived at the Holy Island of Lindisfarne, which was a monastic community, and they essentially slaughtered everyone, burned a couple of buildings, and grabbed everything that had any value and left. And that was the first Viking raid that came in force. And I do think Lindisfarne is a good beginning date because the terror that it brought really signified what was to come for the next two to three centuries.
In June of 793, a group of Vikings, probably originating from Norway, arrived at the Holy Island of Lindisfarne, which was a monastic community, and they essentially slaughtered everyone, burned a couple of buildings, and grabbed everything that had any value and left. And that was the first Viking raid that came in force. And I do think Lindisfarne is a good beginning date because the terror that it brought really signified what was to come for the next two to three centuries.
Lex Fridman
So the word of it has spread. Like there’s a- there’s a bunch of accounts, like the monk Alcuin wrote about this event in a letter to King Ethelred of Northumbria, quote, “It is nearly 350 years that we and our fathers have inhabited this most lovely land, and never before has such terror appeared in Britain as we have now suffered from a pagan race, nor was it thought that such an inroad from the sea could be made.” What made this race so psychologically devastating to this monk and to the many other monks on the island and then to all of Britain?
So the word of it has spread. Like there’s a- there’s a bunch of accounts, like the monk Alcuin wrote about this event in a letter to King Ethelred of Northumbria, quote, “It is nearly 350 years that we and our fathers have inhabited this most lovely land, and never before has such terror appeared in Britain as we have now suffered from a pagan race, nor was it thought that such an inroad from the sea could be made.” What made this race so psychologically devastating to this monk and to the many other monks on the island and then to all of Britain?
Lars Brownworth
That’s a great quote. Alcuin was not just a regular scholar, he was Charlemagne’s favorite scholar and he’s largely responsible, as much as one person can be, for the Carolingian Renaissance that had done so much to elevate the early medieval world. In fact, the spaces we have, the punctuation we have, spaces between words are likely a result of Alcuin’s work. He was an extremely literate man, and you can hear the terror creeping into that.
That’s a great quote. Alcuin was not just a regular scholar, he was Charlemagne’s favorite scholar and he’s largely responsible, as much as one person can be, for the Carolingian Renaissance that had done so much to elevate the early medieval world. In fact, the spaces we have, the punctuation we have, spaces between words are likely a result of Alcuin’s work. He was an extremely literate man, and you can hear the terror creeping into that.
Lars Brownworth
And part of that has to do with monastic communities, the Church and the- what they thought a monastic community was. So the Church was viewed as a sacred place. Everyone in Europe, everyone in quotes, is nominally Christian, and the Church is an area of safety. It’s a literal ark from the troubles of the world that you can flee to. I believe there are even rules in England, for example, that if you had killed someone, you could flee to a church and the civil authorities were not allowed to enter for up to 40 days. So you could have sanctuary there. And to violate this would’ve been the worst possible offense you could have given, which is why, you know, Thomas Becket’s murder is so, so horrible in England.
And part of that has to do with monastic communities, the Church and the- what they thought a monastic community was. So the Church was viewed as a sacred place. Everyone in Europe, everyone in quotes, is nominally Christian, and the Church is an area of safety. It’s a literal ark from the troubles of the world that you can flee to. I believe there are even rules in England, for example, that if you had killed someone, you could flee to a church and the civil authorities were not allowed to enter for up to 40 days. So you could have sanctuary there. And to violate this would’ve been the worst possible offense you could have given, which is why, you know, Thomas Becket’s murder is so, so horrible in England.
Lars Brownworth
And the monks had dedicated themselves to a life of studying the Bible, to copying scriptures to prayer, to removing themselves literally from the temptations of the world. And so they would seek monasteries that were remote, and the most remote locations you could find were islands in the North Atlantic, because it’s just so difficult to get there…. So the ocean was considered a place of safety. Not sailing on the ocean, but these- these islands were- were literal havens of peace and security and closeness to God. And so the fact that the Vikings hit this place of all places you could hit was the worst, the most terrifying kind of offense against medieval sensibilities.
And the monks had dedicated themselves to a life of studying the Bible, to copying scriptures to prayer, to removing themselves literally from the temptations of the world. And so they would seek monasteries that were remote, and the most remote locations you could find were islands in the North Atlantic, because it’s just so difficult to get there…. So the ocean was considered a place of safety. Not sailing on the ocean, but these- these islands were- were literal havens of peace and security and closeness to God. And so the fact that the Vikings hit this place of all places you could hit was the worst, the most terrifying kind of offense against medieval sensibilities.
Lex Fridman
So there’s a kind of line that you understand you don’t cross. Like, everybody agrees.
So there’s a kind of line that you understand you don’t cross. Like, everybody agrees.
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
It’s the kind of thing that there’s a social contract that most societies, most civilizations sign. There’s a line that we don’t cross. Let the scholars do their scholarly work. That’s one line. The other line is more kind of from a military perspective, from a mobility perspective, you just assume the sea is not a place from which a threat could come-
It’s the kind of thing that there’s a social contract that most societies, most civilizations sign. There’s a line that we don’t cross. Let the scholars do their scholarly work. That’s one line. The other line is more kind of from a military perspective, from a mobility perspective, you just assume the sea is not a place from which a threat could come-
Lars Brownworth
Yeah, that’s exactly right.
Yeah, that’s exactly right.
Lex Fridman
… especially the north. So your conception of the world is shattered by, one, the brutality that can come, two, that the sea can bring a threat, and three, that you don’t give a damn about any of the lines that we as a society, as a Christian society, have established.
… especially the north. So your conception of the world is shattered by, one, the brutality that can come, two, that the sea can bring a threat, and three, that you don’t give a damn about any of the lines that we as a society, as a Christian society, have established.
Lars Brownworth
That’s exactly right. I mean, even Alcuin, I think he writes a little later on that the dead were left as dung in the streets. So he’s describing dead monks as literal dung in the streets. And, you know, who would do this to men of God? Inhuman monsters.
That’s exactly right. I mean, even Alcuin, I think he writes a little later on that the dead were left as dung in the streets. So he’s describing dead monks as literal dung in the streets. And, you know, who would do this to men of God? Inhuman monsters.
Lex Fridman
So who were they, the Vikings, coming from the north? How did they think of the violence that they were doing?
So who were they, the Vikings, coming from the north? How did they think of the violence that they were doing?
Lars Brownworth
Now, that’s a very good question because… and it brings up a central problem of looking at the Vikings, which is the story is almost always told from somebody else’s perspective, largely from the pens of those they’re attacking. So they’re not gonna come across well. They’re often portrayed as demonic and inhuman. The Vikings themselves though, as much as we can piece together from archeology, from the stories they wrote later—but that was another problem there, the written alphabet, the runes. It was mostly used for spells, name your sword, things like that, curse someone, but it wasn’t really useful for writing long poetry or literature.
Now, that’s a very good question because… and it brings up a central problem of looking at the Vikings, which is the story is almost always told from somebody else’s perspective, largely from the pens of those they’re attacking. So they’re not gonna come across well. They’re often portrayed as demonic and inhuman. The Vikings themselves though, as much as we can piece together from archeology, from the stories they wrote later—but that was another problem there, the written alphabet, the runes. It was mostly used for spells, name your sword, things like that, curse someone, but it wasn’t really useful for writing long poetry or literature.
Lars Brownworth
So the only Norse literature we have comes at the end of the age when they had adopted the Latin alphabet. So it’s… you can almost never see the Vikings in their own words as they saw themselves. But we can piece certain things together. Most importantly, Viking was not their day job. They were mostly merchants and farmers, mostly farmers who lived in little bays called Viks in Old Norse, which is probably where we get the word Viking from. One other note about how hard it is to tease apart what’s happening here is the English and the Frankish and the Irish writers all call them Danes, no matter where they came from.
So the only Norse literature we have comes at the end of the age when they had adopted the Latin alphabet. So it’s… you can almost never see the Vikings in their own words as they saw themselves. But we can piece certain things together. Most importantly, Viking was not their day job. They were mostly merchants and farmers, mostly farmers who lived in little bays called Viks in Old Norse, which is probably where we get the word Viking from. One other note about how hard it is to tease apart what’s happening here is the English and the Frankish and the Irish writers all call them Danes, no matter where they came from.
Lars Brownworth
They didn’t stop to ask, “Now, excuse me, are you from Norway, or are you from…” So they’re all called Danes or Pagans, heathen, or Northmen. So this is not very helpful in figuring out where they came from.
They didn’t stop to ask, “Now, excuse me, are you from Norway, or are you from…” So they’re all called Danes or Pagans, heathen, or Northmen. So this is not very helpful in figuring out where they came from.
Lars Brownworth
The language was interchangeable. You know, Old Norse was spoken in all three of those Scandinavian countries. But living in the north, so far up near the Arctic Circle, is… that’s at the very limit of where technology of the time could allow humans to survive. And that kinda harsh climate bred, I think, very hard people. Mercy was not a quality they seemed to favor or value. There’s a very famous story of a Swedish Viking putting a sword in the crib of his newborn son, and saying, “May you have nothing in this life but what you can gain with this.” I mean, I can’t imagine doing that. You know, to any of my children, you know, putting a gun in the crib, or-
The language was interchangeable. You know, Old Norse was spoken in all three of those Scandinavian countries. But living in the north, so far up near the Arctic Circle, is… that’s at the very limit of where technology of the time could allow humans to survive. And that kinda harsh climate bred, I think, very hard people. Mercy was not a quality they seemed to favor or value. There’s a very famous story of a Swedish Viking putting a sword in the crib of his newborn son, and saying, “May you have nothing in this life but what you can gain with this.” I mean, I can’t imagine doing that. You know, to any of my children, you know, putting a gun in the crib, or-
Lars Brownworth
… you know, I’d be carted away. But the… I think that kind of underscores the kind of violent life that was… you could expect as a Viking. I mean, strength was valued more than anything else.
… you know, I’d be carted away. But the… I think that kind of underscores the kind of violent life that was… you could expect as a Viking. I mean, strength was valued more than anything else.
Lex Fridman
So the understanding of the world is harsh, and that strength is the way you must face that world. So when you have those people, and especially the ones that self-select to get on a boat, to face the ocean with all the uncertainty, that results in the kind of brutality that we got to see.
So the understanding of the world is harsh, and that strength is the way you must face that world. So when you have those people, and especially the ones that self-select to get on a boat, to face the ocean with all the uncertainty, that results in the kind of brutality that we got to see.
Lars Brownworth
I think so. I mean, the way they would build their ships, they were clinker built, so they were overlaid, like planks overlaying. So they were undecked as well. And so they’d have tents. So can you imagine crossing the Atlantic, the northern Atlantic, you know, with these huge waves splashing over with an inch of oak between you and the ocean? I mean, the amount of bravery that must have taken to undergo is astounding. Plus they didn’t have a compass. They navigated by, “Where’s the sun? Where are the stars? What… are there birds in the sky? Do I see a different color of water? Do I see leaves floating?” I mean, it’s terrible. If you’re traveling 2,000 miles, that’s not great.
I think so. I mean, the way they would build their ships, they were clinker built, so they were overlaid, like planks overlaying. So they were undecked as well. And so they’d have tents. So can you imagine crossing the Atlantic, the northern Atlantic, you know, with these huge waves splashing over with an inch of oak between you and the ocean? I mean, the amount of bravery that must have taken to undergo is astounding. Plus they didn’t have a compass. They navigated by, “Where’s the sun? Where are the stars? What… are there birds in the sky? Do I see a different color of water? Do I see leaves floating?” I mean, it’s terrible. If you’re traveling 2,000 miles, that’s not great.
Lars Brownworth
So it’s kind of an intrepidness to them that I think is part of the reason why they’re so fascinating to us in our sanitized, more or less sanitized world. That this incredible courage to do this, and some horror at what they did on the other end when they arrived. But, you know, we’ll talk a little bit more about their religion, but they do not view the Christian God in particularly flattering terms. I mean, to them, he’s a weak God who won’t protect his adherents, and they can just come in and plunder as they… I mean, they’ll… One Viking famously says, “On land, I’m a Christian. When I’m on the sea, I worship Thor.” It was very much the kind of pragmatic take that the Vikings had.
So it’s kind of an intrepidness to them that I think is part of the reason why they’re so fascinating to us in our sanitized, more or less sanitized world. That this incredible courage to do this, and some horror at what they did on the other end when they arrived. But, you know, we’ll talk a little bit more about their religion, but they do not view the Christian God in particularly flattering terms. I mean, to them, he’s a weak God who won’t protect his adherents, and they can just come in and plunder as they… I mean, they’ll… One Viking famously says, “On land, I’m a Christian. When I’m on the sea, I worship Thor.” It was very much the kind of pragmatic take that the Vikings had.
Lex Fridman
Yeah. There are gods, and they have many, but Odin and Thor are pretty hardcore gods. So everything, just their whole philosophy on life is pretty hardcore. Probably some of the toughest humans to have ever lived.
Yeah. There are gods, and they have many, but Odin and Thor are pretty hardcore gods. So everything, just their whole philosophy on life is pretty hardcore. Probably some of the toughest humans to have ever lived.
Lars Brownworth
I think so. Yeah. I mean, their gods are horrifying. They’re polytheistic. There was no universally accepted head god. I think Marvel has also led people astray in this.
I think so. Yeah. I mean, their gods are horrifying. They’re polytheistic. There was no universally accepted head god. I think Marvel has also led people astray in this.
Viking military strategy, tactics & technology
Lex Fridman
Well, we’ll talk more about religion, but since you mentioned the boats, what do we understand about the technology that they were using? Can you just speak a little bit more to this one-inch-of-oak idea? So, these were these longships that were also able to travel on rivers. So they’re not… Like, what is structurally, do we know about the boats that allowed them to be so flexible in terms of where they can travel?
Well, we’ll talk more about religion, but since you mentioned the boats, what do we understand about the technology that they were using? Can you just speak a little bit more to this one-inch-of-oak idea? So, these were these longships that were also able to travel on rivers. So they’re not… Like, what is structurally, do we know about the boats that allowed them to be so flexible in terms of where they can travel?
Lars Brownworth
Yeah. I mean, and this was the Vikings’ great secret, and I think it’s underappreciated. They built different types of ships, obviously for different purposes, but the thing that blows my mind is that they built these ships that could cross an ocean, cross the Atlantic Ocean, and at the same time, when they had a draft of less than two feet, so they could sail up rivers that were two feet deep. And if they came to an, you know, a block or something, 20 men could pick up the ship and port it around. They were incredibly portable and their speed, the speed was the most frightening thing about the Vikings.
Yeah. I mean, and this was the Vikings’ great secret, and I think it’s underappreciated. They built different types of ships, obviously for different purposes, but the thing that blows my mind is that they built these ships that could cross an ocean, cross the Atlantic Ocean, and at the same time, when they had a draft of less than two feet, so they could sail up rivers that were two feet deep. And if they came to an, you know, a block or something, 20 men could pick up the ship and port it around. They were incredibly portable and their speed, the speed was the most frightening thing about the Vikings.
Lex Fridman
So the… These are the same kind of ship that they sailed the ocean on.
So the… These are the same kind of ship that they sailed the ocean on.
Lars Brownworth
Yeah. I mean, it’s insane.
Yeah. I mean, it’s insane.
Lex Fridman
So they’re pretty sufficiently robust to handle the ocean and sufficiently mobile to travel on rivers and do so really fast, so- … you mentioned speed. That seems to be, from a military perspective, the great advantage of the Vikings- … because they can move much faster than the land armies can. So, and not just the element of surprise, which they often had, but the element of speed was the thing that gave them such an extreme advantage against the British armies.
So they’re pretty sufficiently robust to handle the ocean and sufficiently mobile to travel on rivers and do so really fast, so- … you mentioned speed. That seems to be, from a military perspective, the great advantage of the Vikings- … because they can move much faster than the land armies can. So, and not just the element of surprise, which they often had, but the element of speed was the thing that gave them such an extreme advantage against the British armies.
Lars Brownworth
That was the big one. So, an English army, if it had access to a good Roman road that was well maintained, which frankly there weren’t tons of them, but they could average something like 10 to 15 miles per day- … on a good day, if they didn’t have a large baggage train to slow them down. If you had a cavalry unit that didn’t have to travel with the army, they could average about 20 miles a day. The Viking long ships could average 70 to 120 miles a day. So, they’re just moving in super fast motion. They could hit a place, raid it, drag off whoever they wanted, and get away before you could get your army there. That’s just absolutely terrifying.
That was the big one. So, an English army, if it had access to a good Roman road that was well maintained, which frankly there weren’t tons of them, but they could average something like 10 to 15 miles per day- … on a good day, if they didn’t have a large baggage train to slow them down. If you had a cavalry unit that didn’t have to travel with the army, they could average about 20 miles a day. The Viking long ships could average 70 to 120 miles a day. So, they’re just moving in super fast motion. They could hit a place, raid it, drag off whoever they wanted, and get away before you could get your army there. That’s just absolutely terrifying.
Lex Fridman
What do you think it felt like for Alcuin and the monks to see the Viking ships on the horizon? Do you ever think about trying to put yourself in the mind of those folks and imagining… In that time, you don’t have a full map of the world, right? And the oceans are not mapped, and you have a hazy conception of the world. And so out of the darkness from the ocean where you thought nothing can come, comes this terrifying, this brutal force. What do you think that felt like?
What do you think it felt like for Alcuin and the monks to see the Viking ships on the horizon? Do you ever think about trying to put yourself in the mind of those folks and imagining… In that time, you don’t have a full map of the world, right? And the oceans are not mapped, and you have a hazy conception of the world. And so out of the darkness from the ocean where you thought nothing can come, comes this terrifying, this brutal force. What do you think that felt like?
Lars Brownworth
Honestly, I think it’s the end of the world, and I don’t think that’s… I don’t think they were wrong to think that. The Anglo-Saxon Chronicle says the night before Lindisfarne, the monks saw sheets of lightning in the sky in the shape of dragons. And this obviously meant to, you know, foreshadow the dragon ships coming up. I can’t imagine the horror. It would shake my faith, I’m sure, to have these giant men jumping out of their ships with swords raised. And you’re… What do you have? Your cross?
Honestly, I think it’s the end of the world, and I don’t think that’s… I don’t think they were wrong to think that. The Anglo-Saxon Chronicle says the night before Lindisfarne, the monks saw sheets of lightning in the sky in the shape of dragons. And this obviously meant to, you know, foreshadow the dragon ships coming up. I can’t imagine the horror. It would shake my faith, I’m sure, to have these giant men jumping out of their ships with swords raised. And you’re… What do you have? Your cross?
Lex Fridman
Were the Vikings aware of the fear that they had caused? So, did they use fear as a kind of weapon, or was this just a side consequence of their actions, or did they understand and use it? Like the Mongols, Genghis Khan, the Mongols used the fear and the terror on purpose- … to increase the chance that they wouldn’t have to avoid fights, basically.
Were the Vikings aware of the fear that they had caused? So, did they use fear as a kind of weapon, or was this just a side consequence of their actions, or did they understand and use it? Like the Mongols, Genghis Khan, the Mongols used the fear and the terror on purpose- … to increase the chance that they wouldn’t have to avoid fights, basically.
Lars Brownworth
Yeah, yeah. The Vikings absolutely used terror. It was a main weapon in their arsenal. They would attack specifically on high holy days like Easter, Christmas, because they knew there’d be higher value targets there with richer clothing, richer offerings. There’d be a lot of money available. So they were rather sophisticated, which I think is something also that they don’t get much credit for. It’s like they were just dumb brutes attacking and just destroying. But they were… It was very sophisticated. They would show up. That’s what I mean when I say Viking wasn’t their day jobs. They would be traders in say an English port, kind of looking around.
Yeah, yeah. The Vikings absolutely used terror. It was a main weapon in their arsenal. They would attack specifically on high holy days like Easter, Christmas, because they knew there’d be higher value targets there with richer clothing, richer offerings. There’d be a lot of money available. So they were rather sophisticated, which I think is something also that they don’t get much credit for. It’s like they were just dumb brutes attacking and just destroying. But they were… It was very sophisticated. They would show up. That’s what I mean when I say Viking wasn’t their day jobs. They would be traders in say an English port, kind of looking around.
Lars Brownworth
They’d get everyone’s schedule, then they would sail away and come back as Vikings, and they knew exactly where to go. They knew where all the money was held. They knew where all the, you know, the churches were, when to attack. They knew the entire Christian calendar. They knew when someone’s baptism was, when someone’s confirmation. I mean, they were aware of all of this. And they would… They would definitely attack to increase terror.
They’d get everyone’s schedule, then they would sail away and come back as Vikings, and they knew exactly where to go. They knew where all the money was held. They knew where all the, you know, the churches were, when to attack. They knew the entire Christian calendar. They knew when someone’s baptism was, when someone’s confirmation. I mean, they were aware of all of this. And they would… They would definitely attack to increase terror.
Lex Fridman
One of the signs of the intelligence of the Vikings is that the Viking Age is so short. So what happens is these explorers and these rough men who do the raids, they very quickly are good at conquering and then start state building, or conquering and then establishing trade routes and stop being the quote, unquote, “Vikings.” So basically, they just… They conquer, and then they start doing the usual build the institutions, start a state, and now they’re normal kind of nation, civilization kind of thing. So this kind of force that is the conquering, raid, violent, intense explorers, it’s like a short-lasting thing, a couple of generations at most.
One of the signs of the intelligence of the Vikings is that the Viking Age is so short. So what happens is these explorers and these rough men who do the raids, they very quickly are good at conquering and then start state building, or conquering and then establishing trade routes and stop being the quote, unquote, “Vikings.” So basically, they just… They conquer, and then they start doing the usual build the institutions, start a state, and now they’re normal kind of nation, civilization kind of thing. So this kind of force that is the conquering, raid, violent, intense explorers, it’s like a short-lasting thing, a couple of generations at most.
Lars Brownworth
Yeah, that’s right. I mean, the Vikings were ultimately a pragmatic people who, if it worked, they would keep it, which is frustrating because they disappear so quickly because of that.
Yeah, that’s right. I mean, the Vikings were ultimately a pragmatic people who, if it worked, they would keep it, which is frustrating because they disappear so quickly because of that.
Lex Fridman
With very little trace in the records-
With very little trace in the records-
Lars Brownworth
With very little trace. That’s right.
With very little trace. That’s right.
Lex Fridman
… with very little writing.
… with very little writing.
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
No time for writing it down.
No time for writing it down.
Lars Brownworth
No. Yes. Not doing that.
No. Yes. Not doing that.
Lex Fridman
Yeah. Why were monasteries such good targets for these early raids?
Yeah. Why were monasteries such good targets for these early raids?
Lars Brownworth
This is where I imagine myself as a Viking and one of my ancestors perhaps. And sailing in… I mean, they must have thought they had won the lottery. You got this rich, these rich buildings, rich gold everywhere. Decorated books, jewels, all guarded by old men who don’t know how to fight. You just take it.
This is where I imagine myself as a Viking and one of my ancestors perhaps. And sailing in… I mean, they must have thought they had won the lottery. You got this rich, these rich buildings, rich gold everywhere. Decorated books, jewels, all guarded by old men who don’t know how to fight. You just take it.
Lex Fridman
I mean, we should make clear that the monasteries had… They were used as almost like storage for gold.
I mean, we should make clear that the monasteries had… They were used as almost like storage for gold.
Lars Brownworth
Yeah. And this goes all the way back to, you know, the Roman Empire where, you know, think of, for example, the Emperor Augustus. When he was writing his will, he put it in the Temple of the Vestal Virgins as well as Mark Antony and Cleopatra. They’d all done that because there’s this additional protection of religion-
Yeah. And this goes all the way back to, you know, the Roman Empire where, you know, think of, for example, the Emperor Augustus. When he was writing his will, he put it in the Temple of the Vestal Virgins as well as Mark Antony and Cleopatra. They’d all done that because there’s this additional protection of religion-
Lars Brownworth
… and this taboo against violating that. And the same thing happened when Europe was Christianized. Monasteries were pla— I mean, rich people, their faith had to be an active faith. They had, they couldn’t just say their prayers and go to church on Sunday. They would have to do something to publicly show that they were, you know, worthy of forgiveness or, or whatever. And so they would donate huge sums to the church. I think, you know, by the time of the French Revolution, which is obviously way in the future, the Church is the largest single landowner in France. I mean, the monasteries were… These, these monasteries filled with monks who had taken vows of poverty were some of the richest places in Europe. It’s kind of a strange dichotomy here.
… and this taboo against violating that. And the same thing happened when Europe was Christianized. Monasteries were pla— I mean, rich people, their faith had to be an active faith. They had, they couldn’t just say their prayers and go to church on Sunday. They would have to do something to publicly show that they were, you know, worthy of forgiveness or, or whatever. And so they would donate huge sums to the church. I think, you know, by the time of the French Revolution, which is obviously way in the future, the Church is the largest single landowner in France. I mean, the monasteries were… These, these monasteries filled with monks who had taken vows of poverty were some of the richest places in Europe. It’s kind of a strange dichotomy here.
Lex Fridman
And then we should also say that the Vikings, many of them pragmatic people, so a lot of them would eventually then convert to Christianity, so you get-
And then we should also say that the Vikings, many of them pragmatic people, so a lot of them would eventually then convert to Christianity, so you get-
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
… you integrate yourself into the system.
… you integrate yourself into the system.
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
In some sense, religion creates this backbone of a society that stabilizes it, and then you create a bunch of rules about behavior, how you’re supposed to behave. One of the rules is you don’t mess with the church buildings and- … the religious- … institutions and therefore they become great storage places for gold. And then the Vikings here just test the system. I mean, it’s the fortune of geography for them and the fortune of their way of life to be able to raid, to become extremely rich and therefore this… It both spreads the terror across England, and the message across Scandinavia that there’s a lot of riches to be had. And so the raids, that’s why there’s an explosion of raids.
In some sense, religion creates this backbone of a society that stabilizes it, and then you create a bunch of rules about behavior, how you’re supposed to behave. One of the rules is you don’t mess with the church buildings and- … the religious- … institutions and therefore they become great storage places for gold. And then the Vikings here just test the system. I mean, it’s the fortune of geography for them and the fortune of their way of life to be able to raid, to become extremely rich and therefore this… It both spreads the terror across England, and the message across Scandinavia that there’s a lot of riches to be had. And so the raids, that’s why there’s an explosion of raids.
Lars Brownworth
That’s right. And I think it’s not a coincidence that it happens when it does. I mean, you have both… So there’s two main theories about why the Viking Age starts. The first, Will Durant puts it, I think, the best. He says, “The fertility of the Viking women outstripped the fertility of the Viking land.” It’s basically overpopulation. And then they’re searching for food. And then the second is there’s this technological breakthrough with the keel and maybe pressure put on Charlemagne’s consolidation and a little worries like that. I don’t see why both can’t be true, but I do also think Europe… Like, Charlemagne puts together this vast empire that, you know, fairly approximates the Western Roman Empire. If you squint-
That’s right. And I think it’s not a coincidence that it happens when it does. I mean, you have both… So there’s two main theories about why the Viking Age starts. The first, Will Durant puts it, I think, the best. He says, “The fertility of the Viking women outstripped the fertility of the Viking land.” It’s basically overpopulation. And then they’re searching for food. And then the second is there’s this technological breakthrough with the keel and maybe pressure put on Charlemagne’s consolidation and a little worries like that. I don’t see why both can’t be true, but I do also think Europe… Like, Charlemagne puts together this vast empire that, you know, fairly approximates the Western Roman Empire. If you squint-
Lars Brownworth
… it looks like the Western Roman Empire. He’s calling himself the new Roman emperor. This will eventually mutate into the Holy Roman Empire. But it, it, it’s very much this idea that it’s back. The Roman Empire is back. He’s crowned on Christmas Day in the year 800, and the empire is back. Unfortunately, it was sprawling. It hadn’t been thought through. There was… The communication was terrible. You just couldn’t do it. And so it was wealthy and weak, and that kind of attracts predators. By the time the Vikings crash into it, you also have the added bonus for them of really feckless rulers.
… it looks like the Western Roman Empire. He’s calling himself the new Roman emperor. This will eventually mutate into the Holy Roman Empire. But it, it, it’s very much this idea that it’s back. The Roman Empire is back. He’s crowned on Christmas Day in the year 800, and the empire is back. Unfortunately, it was sprawling. It hadn’t been thought through. There was… The communication was terrible. You just couldn’t do it. And so it was wealthy and weak, and that kind of attracts predators. By the time the Vikings crash into it, you also have the added bonus for them of really feckless rulers.
Lex Fridman
And we should say, going to Perplexity here, that Charlemagne, also known as Charles the Great, is the Frankish king who became emperor in 800 and ruled much of the Western and Central Europe in the late eighth and early ninth centuries. And there’s a theory that the Viking age was also a reaction to the South expanding north, as you’re talking about. You tell the story of Charlemagne weeping because he foresaw the evil his descendants would suffer. Did the Franks accidentally wake the sleeping giant by crushing the Saxons and removing the buffer zone between them and the, and the Vikings?
And we should say, going to Perplexity here, that Charlemagne, also known as Charles the Great, is the Frankish king who became emperor in 800 and ruled much of the Western and Central Europe in the late eighth and early ninth centuries. And there’s a theory that the Viking age was also a reaction to the South expanding north, as you’re talking about. You tell the story of Charlemagne weeping because he foresaw the evil his descendants would suffer. Did the Franks accidentally wake the sleeping giant by crushing the Saxons and removing the buffer zone between them and the, and the Vikings?
Lars Brownworth
I’m sure that had something to do with it. But yeah, as power was consolidated throughout specifically Central Europe, it did put a little pressure on the areas of Denmark. And those are the areas that first kind of erupt down toward… Norway and Denmark contribute most of the early Vikings that hit the Franks. And the Frankish Empire is the most wealthy state in Europe. It’s poured money into religious houses for the reasons you outlined. And all sitting there, easy pickings for people who’ve just developed the keel.
I’m sure that had something to do with it. But yeah, as power was consolidated throughout specifically Central Europe, it did put a little pressure on the areas of Denmark. And those are the areas that first kind of erupt down toward… Norway and Denmark contribute most of the early Vikings that hit the Franks. And the Frankish Empire is the most wealthy state in Europe. It’s poured money into religious houses for the reasons you outlined. And all sitting there, easy pickings for people who’ve just developed the keel.
Lex Fridman
And so they, the word of the raids sent terror through England and through Europe. How much of the raids were reconnaissance, and how much was it just raids, and how much was it preparing for greater scale?
And so they, the word of the raids sent terror through England and through Europe. How much of the raids were reconnaissance, and how much was it just raids, and how much was it preparing for greater scale?
Lars Brownworth
That’s a good… That’s a really good question. I think a lot of the early raids are probing raids, to see what’s there.
That’s a good… That’s a really good question. I think a lot of the early raids are probing raids, to see what’s there.
Lars Brownworth
Definitely when Ragnar Lothbrok, for example, sacks Paris in 845, that definitely results in waves of Viking attacks throughout the 860s, trying to copy that. And he actually is the template which everyone wants to follow. And so that provokes large-scale invasions. And they hit England. They kind of switch off. When France is pretty much exhausted, they switch over to England, and then when England is pretty much conquered, they switch back to France. So I think a lot of these are just probing raids at first, but they’re proof of concept and then they come in force. For example, there was one king in England, his name was Ethelred the Unready, which is a pretty fun, funny… Pun on his name.
Definitely when Ragnar Lothbrok, for example, sacks Paris in 845, that definitely results in waves of Viking attacks throughout the 860s, trying to copy that. And he actually is the template which everyone wants to follow. And so that provokes large-scale invasions. And they hit England. They kind of switch off. When France is pretty much exhausted, they switch over to England, and then when England is pretty much conquered, they switch back to France. So I think a lot of these are just probing raids at first, but they’re proof of concept and then they come in force. For example, there was one king in England, his name was Ethelred the Unready, which is a pretty fun, funny… Pun on his name.
Lars Brownworth
But he paid, in one year, 7.5 million silver pennies to the Vikings to get them to go away, which is a bit like someone’s mugging you, so you pay them more money so they’ll go away. It’s-
But he paid, in one year, 7.5 million silver pennies to the Vikings to get them to go away, which is a bit like someone’s mugging you, so you pay them more money so they’ll go away. It’s-
Lex Fridman
That’s not gonna work, is it?
That’s not gonna work, is it?
Lars Brownworth
It’s not gonna work, but it will bring more muggers. So he paid the equivalent of 50 adult elephants, 48,000 pounds of silver, to get the muggers to go away. And it’s unsurprising that throughout the course of his reign, he paid something like 20 tons of gold and silver, which he had to tax his people for.
It’s not gonna work, but it will bring more muggers. So he paid the equivalent of 50 adult elephants, 48,000 pounds of silver, to get the muggers to go away. And it’s unsurprising that throughout the course of his reign, he paid something like 20 tons of gold and silver, which he had to tax his people for.
Lex Fridman
Yeah, the Vikings are not the kind of people that that would make go away, right?
Yeah, the Vikings are not the kind of people that that would make go away, right?
Lars Brownworth
Nope. That’s not gonna happen.
Nope. That’s not gonna happen.
Lex Fridman
Yeah, they would just come back in force.
Yeah, they would just come back in force.
Lars Brownworth
Yeah, they trust silver to do the work of swords.
Yeah, they trust silver to do the work of swords.
Ragnar Lothbrok
Lex Fridman
You mentioned Ragnar Lothbrok. Who was Ragnar Lothbrok? Did he actually exist? Some people believe he’s a composite from several real ninth century Viking leaders, versus an actual singular human.
You mentioned Ragnar Lothbrok. Who was Ragnar Lothbrok? Did he actually exist? Some people believe he’s a composite from several real ninth century Viking leaders, versus an actual singular human.
Lars Brownworth
Yeah, I’m a romantic. I would like to believe he existed. I think probably he’s a compilation of a lot of different… There probably is a seed of truth there. There probably was someone named Ragnar. The last name’s a little suspicious. Lothbrok means hairy breeches. He supposedly had magic pants that would prevent him from being poisoned by dragons or snakes. That’s maybe a clue. We’re dealing with myth here. But he is really the template for Vikings. You want to figure out, like, what the Vikings wanted, who’s their success story, it’s Ragnar Lothbrok. He’s born Norway, Denmark—countries argue over that. Maybe Sweden. Some sagas say he’s in Uppsala.
Yeah, I’m a romantic. I would like to believe he existed. I think probably he’s a compilation of a lot of different… There probably is a seed of truth there. There probably was someone named Ragnar. The last name’s a little suspicious. Lothbrok means hairy breeches. He supposedly had magic pants that would prevent him from being poisoned by dragons or snakes. That’s maybe a clue. We’re dealing with myth here. But he is really the template for Vikings. You want to figure out, like, what the Vikings wanted, who’s their success story, it’s Ragnar Lothbrok. He’s born Norway, Denmark—countries argue over that. Maybe Sweden. Some sagas say he’s in Uppsala.
Lars Brownworth
Anyway, he is, you know, penniless, and when he is in his late teens or early 20s, he decides to invade, sail up the Seine. There is a well-known city on the Seine and he raids it. Supposedly, he takes the hinge of one of the gates from Paris to prove that he’s been there. The- the Frankish king, I love the Frankish kings because they- their citizens give them names based on how much they hate them. So you have- you have Charles the Great, right? Charles the Great, Charlemagne. He’s followed by Louis the Pious. That’s probably the best one. And Louis the Pious is followed by Charles the Fat- … who’s followed by Charles the Bald- … who’s followed by Charles the Simple or Stupid.
Anyway, he is, you know, penniless, and when he is in his late teens or early 20s, he decides to invade, sail up the Seine. There is a well-known city on the Seine and he raids it. Supposedly, he takes the hinge of one of the gates from Paris to prove that he’s been there. The- the Frankish king, I love the Frankish kings because they- their citizens give them names based on how much they hate them. So you have- you have Charles the Great, right? Charles the Great, Charlemagne. He’s followed by Louis the Pious. That’s probably the best one. And Louis the Pious is followed by Charles the Fat- … who’s followed by Charles the Bald- … who’s followed by Charles the Simple or Stupid.
Lex Fridman
Nice. So you can trust the names-
Nice. So you can trust the names-
Lars Brownworth
You can trust-
You can trust-
Lex Fridman
… to give you the TL;DR of how good of a ruler they were.
… to give you the TL;DR of how good of a ruler they were.
Lars Brownworth
Yeah, exactly. Exactly.
Yeah, exactly. Exactly.
Lex Fridman
So Charles the Great, widely acknowledged as sort of one of the great leaders of the Frankish Empire- … aka Charlemagne.
So Charles the Great, widely acknowledged as sort of one of the great leaders of the Frankish Empire- … aka Charlemagne.
Lex Fridman
So what- what else do we know about him? So, there’s going to Perplexity, “Ragnar’s portrayed as a Scandinavian warlord, often called a Danish or Swedish king,” like you mentioned. Uh, “Active in the ninth century during the height of the Viking raids.” And then descriptions of the raids and the exploits. “Medieval traditions link Ragnar to famous raids on the Frankish realms, especially the attack on Paris in 845, where he reputedly sails up the Seine and extorts a huge ransom from King Charles the Bald.” He’s also associated with repeated attacks on Anglo-Saxon England, embodying the archetypal Viking chieftain, charismatic, brutal, and focused on wealth, fame, and honor in battle. So that, those are the ideals of the- of the Vikings. Charisma, brutality-
So what- what else do we know about him? So, there’s going to Perplexity, “Ragnar’s portrayed as a Scandinavian warlord, often called a Danish or Swedish king,” like you mentioned. Uh, “Active in the ninth century during the height of the Viking raids.” And then descriptions of the raids and the exploits. “Medieval traditions link Ragnar to famous raids on the Frankish realms, especially the attack on Paris in 845, where he reputedly sails up the Seine and extorts a huge ransom from King Charles the Bald.” He’s also associated with repeated attacks on Anglo-Saxon England, embodying the archetypal Viking chieftain, charismatic, brutal, and focused on wealth, fame, and honor in battle. So that, those are the ideals of the- of the Vikings. Charisma, brutality-
Lex Fridman
… and focusing on wealth, fame, and honor, especially honor in battle.
… and focusing on wealth, fame, and honor, especially honor in battle.
Lars Brownworth
Then also, what does he do with it, right? What does he do with it? So, he gets about 7,000 pounds of silver from Charles the Bald, which destroys, essentially destroys Charles the Bald’s kingship. But he goes back home to Denmark, and the Danish king doesn’t want him around, because he’s too powerful; he’s too rich. He’s a ring giver. You know, think Beowulf here, right? He’s got this large personal army which wants to join him for ad- he can do, you know, they’ll follow him, and he is a threat, and so he kind of is encouraged to go elsewhere.
Then also, what does he do with it, right? What does he do with it? So, he gets about 7,000 pounds of silver from Charles the Bald, which destroys, essentially destroys Charles the Bald’s kingship. But he goes back home to Denmark, and the Danish king doesn’t want him around, because he’s too powerful; he’s too rich. He’s a ring giver. You know, think Beowulf here, right? He’s got this large personal army which wants to join him for ad- he can do, you know, they’ll follow him, and he is a threat, and so he kind of is encouraged to go elsewhere.
Lars Brownworth
He ends up raiding England for something like 15 years, and then there’s a, probably the most famous bit of the story is he- he’s shipwrecked, and King Ælla of Northumberland captures him and decides to kill him by throwing him into a pit with vipers.
He ends up raiding England for something like 15 years, and then there’s a, probably the most famous bit of the story is he- he’s shipwrecked, and King Ælla of Northumberland captures him and decides to kill him by throwing him into a pit with vipers.
Lars Brownworth
They throw him in this and the snakes are biting him, but he’s got his hairy breeches on, so it’s not working. So, he’s singing a hymn to Odin and he gets pulled out and he’s asked why he’s not dying and he explains, rather foolishly, that he has these hairy breeches. So, they take the pants off and throw him back, and his last words are “When the boar bleats, the piglets come.”… by which he means, “My- I have sons,” he had 12 of them, “and they will avenge me.” And they do; they lead the the Great Heathen Army to invade and eventually conquer England. Ælla, fun fact, not so fun for him, is the, supposedly was captured by the son of Ragnar, his name is Ivar the Boneless, which is somewhat terrifying of a name. And he is, he’s the first person that a blood eagle was performed on.
They throw him in this and the snakes are biting him, but he’s got his hairy breeches on, so it’s not working. So, he’s singing a hymn to Odin and he gets pulled out and he’s asked why he’s not dying and he explains, rather foolishly, that he has these hairy breeches. So, they take the pants off and throw him back, and his last words are “When the boar bleats, the piglets come.”… by which he means, “My- I have sons,” he had 12 of them, “and they will avenge me.” And they do; they lead the the Great Heathen Army to invade and eventually conquer England. Ælla, fun fact, not so fun for him, is the, supposedly was captured by the son of Ragnar, his name is Ivar the Boneless, which is somewhat terrifying of a name. And he is, he’s the first person that a blood eagle was performed on.
Lex Fridman
What’s the blood eagle?
What’s the blood eagle?
Lars Brownworth
It’s when they remove the lungs, they, while you’re still alive, they cut you open and remove the lungs and put the lungs on your back. And then when you try to breathe, they flutter like wings, so it’s called, like an eagle. It’s called the blood eagle.
It’s when they remove the lungs, they, while you’re still alive, they cut you open and remove the lungs and put the lungs on your back. And then when you try to breathe, they flutter like wings, so it’s called, like an eagle. It’s called the blood eagle.
Lex Fridman
That is horrible.
That is horrible.
Lars Brownworth
It’s disgusting, yes. And this is what Ælla, you know, deserves, according to, you know, Bjorn Ironside and Ivar the Boneless, the sons of Ragnar. Like, this is what they get.
It’s disgusting, yes. And this is what Ælla, you know, deserves, according to, you know, Bjorn Ironside and Ivar the Boneless, the sons of Ragnar. Like, this is what they get.
Lex Fridman
Offense.
Offense.
Lars Brownworth
This is the piglets coming- … to their own boar, you know? One last thing about Ragnar, is his wife is also an important part. He had something like 12 sons, the accounts differ, and probably three marriages. But his most famous wife was named Aslaug, and she fell in love with him. He was on a ship, he was passing through, so- … kind of a glamorous sea king, right? With his, he’s the, he’s living the dream. And she sees him and she wants to be married to him, and- and he- he says no. He says, ’cause he wants a clever wife, and so he says, “If you can accomplish these three things, you can marry me. So tomorrow, I’ll be here tonight, and then tomorrow, I want you to come to my ship. I want you to have no clothes on-
This is the piglets coming- … to their own boar, you know? One last thing about Ragnar, is his wife is also an important part. He had something like 12 sons, the accounts differ, and probably three marriages. But his most famous wife was named Aslaug, and she fell in love with him. He was on a ship, he was passing through, so- … kind of a glamorous sea king, right? With his, he’s the, he’s living the dream. And she sees him and she wants to be married to him, and- and he- he says no. He says, ’cause he wants a clever wife, and so he says, “If you can accomplish these three things, you can marry me. So tomorrow, I’ll be here tonight, and then tomorrow, I want you to come to my ship. I want you to have no clothes on-
Lars Brownworth
… but not be naked. I want you to have not eaten a meal, but not have fasted. And I want you to come without a companion, but not alone.” And so she shows up with a dog. She doesn’t have a companion, but she’s not alone. She’s taken a bite out of an onion, so she’s eaten. She hasn’t fasted, but she hasn’t had a meal. And then she has very long hair, and so she’s using the hair to cover herself. So she has no clothes-
… but not be naked. I want you to have not eaten a meal, but not have fasted. And I want you to come without a companion, but not alone.” And so she shows up with a dog. She doesn’t have a companion, but she’s not alone. She’s taken a bite out of an onion, so she’s eaten. She hasn’t fasted, but she hasn’t had a meal. And then she has very long hair, and so she’s using the hair to cover herself. So she has no clothes-
Lex Fridman
Oh, she shows up naked, but she’s-
Oh, she shows up naked, but she’s-
Lars Brownworth
… but clothed.
… but clothed.
Lex Fridman
Right. Wow.
Right. Wow.
Lars Brownworth
Yeah, so in this, so this is kind of the cleverness that would be expected of a Viking woman. So they’re well matched, they’re like the ideal couple. And then they have 12 kids, 12 sons. Not just 12 kids, 12 sons.
Yeah, so in this, so this is kind of the cleverness that would be expected of a Viking woman. So they’re well matched, they’re like the ideal couple. And then they have 12 kids, 12 sons. Not just 12 kids, 12 sons.
Lex Fridman
And many of them end up…
And many of them end up…
Lars Brownworth
Many of them end up almost as famous as their father.
Many of them end up almost as famous as their father.
Lex Fridman
Ivar the Boneless. Bjorn Ironside, and many others. These sons later appear as leaders of major Viking forces in England, particularly the so-called Great Heathen Army- … that invades in 865.
Ivar the Boneless. Bjorn Ironside, and many others. These sons later appear as leaders of major Viking forces in England, particularly the so-called Great Heathen Army- … that invades in 865.
Lars Brownworth
And they are historical. They are—I mean, there’s no—these were the names of Vikings who attacked and conquered England. They end up attacking Islamic Spain. They go all over Europe.
And they are historical. They are—I mean, there’s no—these were the names of Vikings who attacked and conquered England. They end up attacking Islamic Spain. They go all over Europe.
Lex Fridman
Well, for them, it sounds like glory in battle is really important.
Well, for them, it sounds like glory in battle is really important.
Lars Brownworth
That’s right, yeah.
That’s right, yeah.
Lex Fridman
And so it’s not even- It’s just part of the culture, it’s part-
And so it’s not even- It’s just part of the culture, it’s part-
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
… the honor culture.
… the honor culture.
Lars Brownworth
Men die, but names live forever.
Men die, but names live forever.
Lex Fridman
As a small aside, since Ragnar is the star of the Vikings TV series, I don’t know if you’ve gotten a chance to watch any of it. Is there any accuracy to it?
As a small aside, since Ragnar is the star of the Vikings TV series, I don’t know if you’ve gotten a chance to watch any of it. Is there any accuracy to it?
Lars Brownworth
I think it’s well done. My one quibble, Ragnar’s brother is Rollo in the show, right? They weren’t brothers. In fact, by some accounts, they were born 80 years apart. But as a storytelling device, I applaud that.
I think it’s well done. My one quibble, Ragnar’s brother is Rollo in the show, right? They weren’t brothers. In fact, by some accounts, they were born 80 years apart. But as a storytelling device, I applaud that.
Lex Fridman
Yeah, they basically take all the main Vikings and put them all together. Just so it’s a-
Yeah, they basically take all the main Vikings and put them all together. Just so it’s a-
Lars Brownworth
I mean, I get it. I get, it’s confusing. Honestly, in writing a book about it, the hardest part was coming up with an organizational scheme. Like, what’s the overarching thing that links them together?
I mean, I get it. I get, it’s confusing. Honestly, in writing a book about it, the hardest part was coming up with an organizational scheme. Like, what’s the overarching thing that links them together?
Lex Fridman
Well, there’s certainly an overarching thing, but we don’t have information about it. This is, the problem is we get to see just slivers of the information-
Well, there’s certainly an overarching thing, but we don’t have information about it. This is, the problem is we get to see just slivers of the information-
Lars Brownworth
That’s right, that’s right.
That’s right, that’s right.
Lex Fridman
… from the raids. There might be just this rich history that we know nothing about. Like where did this warrior culture come from? Like, what was the evolution of these ideas of honor and battle? I mean, maybe it’s being overly romantic, but you can imagine the ideals of battle from the Roman Empire, from the Roman Republic and the early imperial period coming up north to Scandinavia. And we just know very little traces about that.
… from the raids. There might be just this rich history that we know nothing about. Like where did this warrior culture come from? Like, what was the evolution of these ideas of honor and battle? I mean, maybe it’s being overly romantic, but you can imagine the ideals of battle from the Roman Empire, from the Roman Republic and the early imperial period coming up north to Scandinavia. And we just know very little traces about that.
Lars Brownworth
Yeah. Even the name Scandinavia is from a Roman author. I mean, they thought it was an island. They thought Scandinavia was an island with one tribe, the Scandia tribe, but you know, close enough.
Yeah. Even the name Scandinavia is from a Roman author. I mean, they thought it was an island. They thought Scandinavia was an island with one tribe, the Scandia tribe, but you know, close enough.
The Great Heathen Army
Lex Fridman
And who was the, what was in this Great Heathen Army that invaded England in 865? What can we say about that?
And who was the, what was in this Great Heathen Army that invaded England in 865? What can we say about that?
Lars Brownworth
Well, there’s this famous scene in the Viking siege of Paris in 845, which is really the Europeans’ introduction, or Europe as a whole, to a Viking army, not just a raid and then what it could do. And the king, the emperor, Charles, said, “You know, let’s find out what they want and how much do I have to pay to get them to leave?” And so his ambassador went to a Viking and said, “Who is your king?” And the Viking looked at him, he didn’t understand, and he said, “We have no king. We are all kings.” So they’re very like decentralized, tough. They only valued leaders who could prove that they had won. You know, could give out the rings.
Well, there’s this famous scene in the Viking siege of Paris in 845, which is really the Europeans’ introduction, or Europe as a whole, to a Viking army, not just a raid and then what it could do. And the king, the emperor, Charles, said, “You know, let’s find out what they want and how much do I have to pay to get them to leave?” And so his ambassador went to a Viking and said, “Who is your king?” And the Viking looked at him, he didn’t understand, and he said, “We have no king. We are all kings.” So they’re very like decentralized, tough. They only valued leaders who could prove that they had won. You know, could give out the rings.
Lex Fridman
So flat organization, very meritocratic.
So flat organization, very meritocratic.
Lars Brownworth
Yes.
Yes.
Lex Fridman
If you’re good at what you do, you demonstrate that skill in battle. That means you get to have maybe a leadership position.
If you’re good at what you do, you demonstrate that skill in battle. That means you get to have maybe a leadership position.
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
And the moment you’re no longer effective, you don’t get to have this leadership position. “We’re all kings.” That’s gangster. Throughout history, the Mongols—Genghis Khan was famous for this meritocracy.
And the moment you’re no longer effective, you don’t get to have this leadership position. “We’re all kings.” That’s gangster. Throughout history, the Mongols—Genghis Khan was famous for this meritocracy.
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
That’s one of the components of an extremely effective military force, is if meritocracy is prized. Same is true for who gets to rule. How do you determine the succession? If you’re just giving it to your oldest son, that’s gonna be a problem.
That’s one of the components of an extremely effective military force, is if meritocracy is prized. Same is true for who gets to rule. How do you determine the succession? If you’re just giving it to your oldest son, that’s gonna be a problem.
Lars Brownworth
Yeah. Yeah, that… I could not agree more. There are some problems with meritocracy in civil war because it tends to… The only way you can find out, like Alexander the Great, right? Who does your empire belong to? To the strongest. That kind of guarantees the civil war. At least with giving it to your older son, you know who’s gonna be… There’s an element of stability there although you may end up with a Caligula. More likely than not, you’re gonna end up with a Caligula, I would say, human nature being what it is.
Yeah. Yeah, that… I could not agree more. There are some problems with meritocracy in civil war because it tends to… The only way you can find out, like Alexander the Great, right? Who does your empire belong to? To the strongest. That kind of guarantees the civil war. At least with giving it to your older son, you know who’s gonna be… There’s an element of stability there although you may end up with a Caligula. More likely than not, you’re gonna end up with a Caligula, I would say, human nature being what it is.
Lex Fridman
It’s- It’s… Yeah, yeah, it always converges to the asshole, and the asshole holds power, a crazy asshole. So yeah, Great Heathen Army, 865.
It’s- It’s… Yeah, yeah, it always converges to the asshole, and the asshole holds power, a crazy asshole. So yeah, Great Heathen Army, 865.
Lars Brownworth
So the Great Heathen Army, they were war bands that… Each followed this guy and this guy. “And I’m gonna sit you down in this room. I’m gonna tell you my plan. You’re gonna listen or you’re gonna push back. I’m gonna push back, and we’ll just have this kind of creative discussion and come up with a plan we all agree on.”
So the Great Heathen Army, they were war bands that… Each followed this guy and this guy. “And I’m gonna sit you down in this room. I’m gonna tell you my plan. You’re gonna listen or you’re gonna push back. I’m gonna push back, and we’ll just have this kind of creative discussion and come up with a plan we all agree on.”
Lex Fridman
So it used to be relatively small Viking groups that were doing raids.
So it used to be relatively small Viking groups that were doing raids.
Lars Brownworth
Right.
Right.
Lex Fridman
And then the Great Heathen Army is this large coalition of Viking groups-
And then the Great Heathen Army is this large coalition of Viking groups-
Lars Brownworth
That’s right
That’s right
Lex Fridman
… without a real leader that was able to somehow stabilize enough to have something like governance. Basically, there seems to be a very rapid evolution of a Viking in every part of the world they touch. You go explore raid, conquer, establish state- … And trade routes, and always maintaining a grand ambition, but no longer doing the violence, and always being sufficiently programmatic and flexible where you can accept a conversion to Christianity, for example, if it’s useful-
… without a real leader that was able to somehow stabilize enough to have something like governance. Basically, there seems to be a very rapid evolution of a Viking in every part of the world they touch. You go explore raid, conquer, establish state- … And trade routes, and always maintaining a grand ambition, but no longer doing the violence, and always being sufficiently programmatic and flexible where you can accept a conversion to Christianity, for example, if it’s useful-
Lars Brownworth
That’s right
That’s right
Lex Fridman
… and then they accept the culture, accept the language. So that’s why they integrate, and the thing that we think of as Viking dis- kind of dissipates and disappears pretty quickly.
… and then they accept the culture, accept the language. So that’s why they integrate, and the thing that we think of as Viking dis- kind of dissipates and disappears pretty quickly.
Lars Brownworth
Yeah, and I think the best example of this is France, right? So the Vikings, which is—we’ll talk about this more probably with Rollo—but, you know, the Vikings settle in France, in the North Man’s Duchy, which is shortened to Normandy. And they, within a generation… I mean, Rollo, whose real name is Hrolf Eric, he names his son William. That’s not a Viking name. And within a generation, the language is gone. The Viking names are gone. The worship of Odin is, as far as we can tell, gone. And the Normans are building churches and marrying into the local aristocracy in there. Their…
Yeah, and I think the best example of this is France, right? So the Vikings, which is—we’ll talk about this more probably with Rollo—but, you know, the Vikings settle in France, in the North Man’s Duchy, which is shortened to Normandy. And they, within a generation… I mean, Rollo, whose real name is Hrolf Eric, he names his son William. That’s not a Viking name. And within a generation, the language is gone. The Viking names are gone. The worship of Odin is, as far as we can tell, gone. And the Normans are building churches and marrying into the local aristocracy in there. Their…
Lars Brownworth
Essentially, their Viking-ness is gone except for one thing, their, like, incredible vitality, which the Normans essentially conquer kingdoms at both ends of Europe, Sicily and England, and found two of the foremost powerful states in medieval Europe.
Essentially, their Viking-ness is gone except for one thing, their, like, incredible vitality, which the Normans essentially conquer kingdoms at both ends of Europe, Sicily and England, and found two of the foremost powerful states in medieval Europe.
Lex Fridman
Yeah, so the ambition is there.
Yeah, so the ambition is there.
Lars Brownworth
Is there.
Is there.
Lex Fridman
The vitality is there, but it’s-
The vitality is there, but it’s-
Lars Brownworth
The methods have changed.
The methods have changed.
Rollo and Normandy
Lex Fridman
Yeah, and they change rapidly, which is fascinating. So you have a book. You have a podcast series on the Normans, so let’s talk about Rollo. Who was Rollo? The famous Viking war leader who became the first ruler of Normandy, Northern France.
Yeah, and they change rapidly, which is fascinating. So you have a book. You have a podcast series on the Normans, so let’s talk about Rollo. Who was Rollo? The famous Viking war leader who became the first ruler of Normandy, Northern France.
Lars Brownworth
Well, first I should say, as someone of Norwegian descent, I’m gonna fall down on the Norwegian side of the argument here because- … Norway and Denmark almost came to blows over which was the birthplace of Rollo. But the consensus seems to be Norway- Not just biased.
Well, first I should say, as someone of Norwegian descent, I’m gonna fall down on the Norwegian side of the argument here because- … Norway and Denmark almost came to blows over which was the birthplace of Rollo. But the consensus seems to be Norway- Not just biased.
Lars Brownworth
So he was… The only thing we… The only glimpse we get of Rollo as a young man is he was very tall, so he’s called Hrolf Walker, Hrolf Ganger, because he was so tall he couldn’t ride the little Viking ponies. So he had to walk everywhere. But… Kinda poor, probably raised on stories of Ragnar and the other Viking lords, and he goes. He may have participated in some of the earlier, like the 860 raids that the Vikings did on Paris or the Seine, you know, and then he eventually ends up plundering what will become the Norman coast. And in the year 911, he makes a treaty, the Treaty of Saint-Clair-sur-Epte, with the Frankish King, Charles the Simple, which is not stupid. It’s more like straightforward.
So he was… The only thing we… The only glimpse we get of Rollo as a young man is he was very tall, so he’s called Hrolf Walker, Hrolf Ganger, because he was so tall he couldn’t ride the little Viking ponies. So he had to walk everywhere. But… Kinda poor, probably raised on stories of Ragnar and the other Viking lords, and he goes. He may have participated in some of the earlier, like the 860 raids that the Vikings did on Paris or the Seine, you know, and then he eventually ends up plundering what will become the Norman coast. And in the year 911, he makes a treaty, the Treaty of Saint-Clair-sur-Epte, with the Frankish King, Charles the Simple, which is not stupid. It’s more like straightforward.
Lars Brownworth
There’s no guile in how he talks. And Charles makes a really interesting deal with Rollo, which is, “Why don’t you settle here, integrate into the local aristocracy and defend the Nor- the French coast against the Vikings?” Which I don’t know. It’s like putting a burglar in charge of your security or some- I don’t know, but it works. It works. And Rollo, by the time he makes that deal, he’s probably in his mid-fifties to mid-sixties. It’s unclear when he was born, but the point is he’s lived the Viking life. He’s got something like 20 or 30, if you add up all the sagas they say that he… they gave him this many coins or whatever. He has probably 20 or 30 tons of silver that he has acquired and then probably given out to whatever. So-
There’s no guile in how he talks. And Charles makes a really interesting deal with Rollo, which is, “Why don’t you settle here, integrate into the local aristocracy and defend the Nor- the French coast against the Vikings?” Which I don’t know. It’s like putting a burglar in charge of your security or some- I don’t know, but it works. It works. And Rollo, by the time he makes that deal, he’s probably in his mid-fifties to mid-sixties. It’s unclear when he was born, but the point is he’s lived the Viking life. He’s got something like 20 or 30, if you add up all the sagas they say that he… they gave him this many coins or whatever. He has probably 20 or 30 tons of silver that he has acquired and then probably given out to whatever. So-
Lex Fridman
So yeah, so he’s done the full-
So yeah, so he’s done the full-
Lars Brownworth
He’s done the thing.
He’s done the thing.
Lex Fridman
… the raid- … and then the conquering and then-
… the raid- … and then the conquering and then-
Lars Brownworth
And then the king says, “Can you settle here?” “Can I give you legitimacy?”
And then the king says, “Can you settle here?” “Can I give you legitimacy?”
Lex Fridman
So he does the diplomacy of a treaty. … Then he does the good state craft and state building and then becomes, I mean, European. In one life, he goes through the full journey.
So he does the diplomacy of a treaty. … Then he does the good state craft and state building and then becomes, I mean, European. In one life, he goes through the full journey.
Lars Brownworth
It’s… Yeah, yeah.
It’s… Yeah, yeah.
Lex Fridman
And then his son, William Longsword-
And then his son, William Longsword-
Lars Brownworth
William Longsword
William Longsword
Lex Fridman
… succeeded him and …
… succeeded him and …
Lars Brownworth
Gets assassinated, but he does enlarge Normandy. So basically, every ruler after Rollo enlarges Normandy until it essentially becomes more powerful than the king- … By far. There’s a wonderful scene when- when Rollo signs the treaty. He becomes a liege lord of the French king, and there’s this- this great scene ’cause Rollo has to bend down and kiss the foot of the king. So Rollo’s probably, you know, he’s a Norwegian Viking. He’s probably, I don’t know, six foot.
Gets assassinated, but he does enlarge Normandy. So basically, every ruler after Rollo enlarges Normandy until it essentially becomes more powerful than the king- … By far. There’s a wonderful scene when- when Rollo signs the treaty. He becomes a liege lord of the French king, and there’s this- this great scene ’cause Rollo has to bend down and kiss the foot of the king. So Rollo’s probably, you know, he’s a Norwegian Viking. He’s probably, I don’t know, six foot.
Lars Brownworth
Charles, this little Frank, he’s probably five ten. So he’s like Rollo’s towering over him, and he’s- there’s a large- both armies are watching. There’s a bunch of people who have come in from the countryside. They’ve heard something’s going on, and this important part of this feudal ceremony, you have to kiss your lord’s foot to- to, you know, be in a subservient role. And Rollo says, “I’m not gonna do that.” So he turns to one of his guards and says, “You kiss the foot,” and the guard’s probably taller than he is. So he bends down and he picks the king’s foot up to his mouth-
Charles, this little Frank, he’s probably five ten. So he’s like Rollo’s towering over him, and he’s- there’s a large- both armies are watching. There’s a bunch of people who have come in from the countryside. They’ve heard something’s going on, and this important part of this feudal ceremony, you have to kiss your lord’s foot to- to, you know, be in a subservient role. And Rollo says, “I’m not gonna do that.” So he turns to one of his guards and says, “You kiss the foot,” and the guard’s probably taller than he is. So he bends down and he picks the king’s foot up to his mouth-
Lex Fridman
That’s the way to do it.
That’s the way to do it.
Lars Brownworth
… which Charles goes falling on the back. I mean- … I can’t think of a better example of the relationship between the Norman dukes and the French kings. I mean, it’s perfect. It’s perfect.
… which Charles goes falling on the back. I mean- … I can’t think of a better example of the relationship between the Norman dukes and the French kings. I mean, it’s perfect. It’s perfect.
Lex Fridman
Oh God, I love the Vikings. So as you’ve covered, and maybe you could speak to that a bit more, for a long time to come, Normans have influence on Europe and beyond.
Oh God, I love the Vikings. So as you’ve covered, and maybe you could speak to that a bit more, for a long time to come, Normans have influence on Europe and beyond.
Lars Brownworth
Yeah, it’s hard to overstate Normandy’s impact on Europe in the Middle Ages. Of course, they’ll go on to conquer England as well. But Rollo, when he signs the treaty, it’s an ambiguous treaty. He’s given a title which is rather ambiguous. He’s not a duke, and it’s not clear. He’s not an earl. He’s not a duke. He’s just subservient to the king. Which means Normandy is not a duchy. It’s not a principality. It’s kind of this ambiguous, no one really knows what it is. And so Rollo, being a good Viking, and his descendants being good Vikings, despite becoming French, they just call themselves Duke.
Yeah, it’s hard to overstate Normandy’s impact on Europe in the Middle Ages. Of course, they’ll go on to conquer England as well. But Rollo, when he signs the treaty, it’s an ambiguous treaty. He’s given a title which is rather ambiguous. He’s not a duke, and it’s not clear. He’s not an earl. He’s not a duke. He’s just subservient to the king. Which means Normandy is not a duchy. It’s not a principality. It’s kind of this ambiguous, no one really knows what it is. And so Rollo, being a good Viking, and his descendants being good Vikings, despite becoming French, they just call themselves Duke.
Lars Brownworth
And they essentially seize whatever power they want. There’s one Norman duke, I think he’s the grandson of Rollo. He’s kidnapped by the French king when he’s 14. He escapes the captivity and kidnaps the king. As a 14-year-old, I mean, it’s just—these are—these guys are crazy.
And they essentially seize whatever power they want. There’s one Norman duke, I think he’s the grandson of Rollo. He’s kidnapped by the French king when he’s 14. He escapes the captivity and kidnaps the king. As a 14-year-old, I mean, it’s just—these are—these guys are crazy.
Lex Fridman
How far geographically and in time does the influence of the Normans and Normandy go? So what should we understand about the impact of Normans- … in history?
How far geographically and in time does the influence of the Normans and Normandy go? So what should we understand about the impact of Normans- … in history?
Lars Brownworth
I’m a romantic, so I, when I read history, I usually end up rooting for the losers. I want Harold Godwinson to beat William the Conqueror. You know, I want Hector to beat Achilles. Never works, no matter how many times I read it. But I was always interested in the Normans because of the Norman conquest of England. And I have a twin brother, and he asked me, we were taking a walk and he asked me, “How did Europe…” Because I was reading about the Dark Ages at the time, the early Middle Ages. “And how did Europe, this kind of backwards place, become the dominant-“
I’m a romantic, so I, when I read history, I usually end up rooting for the losers. I want Harold Godwinson to beat William the Conqueror. You know, I want Hector to beat Achilles. Never works, no matter how many times I read it. But I was always interested in the Normans because of the Norman conquest of England. And I have a twin brother, and he asked me, we were taking a walk and he asked me, “How did Europe…” Because I was reading about the Dark Ages at the time, the early Middle Ages. “And how did Europe, this kind of backwards place, become the dominant-“
Lars Brownworth
“… force in the world?” And I started thinking about that, and my answer really is the Normans. The Normans, that’s the great change between Europe as a backwards, inward-looking place, and Europe as a kind of confident, outward-looking place. And that change happens under the Normans. I mean, the Normans, it’s not a coincidence that they lead the charge in the First Crusade. They create the state of England. If you look at England before the Vikings arrive, there are seven, it’s the Heptarchy, there are seven kingdoms in England, and the Vikings destroyed all but one. Only Wessex is preserved, and they’ve conquered about half of Wessex. And there’s a young king. What’s he gonna do?
“… force in the world?” And I started thinking about that, and my answer really is the Normans. The Normans, that’s the great change between Europe as a backwards, inward-looking place, and Europe as a kind of confident, outward-looking place. And that change happens under the Normans. I mean, the Normans, it’s not a coincidence that they lead the charge in the First Crusade. They create the state of England. If you look at England before the Vikings arrive, there are seven, it’s the Heptarchy, there are seven kingdoms in England, and the Vikings destroyed all but one. Only Wessex is preserved, and they’ve conquered about half of Wessex. And there’s a young king. What’s he gonna do?
Lars Brownworth
But that king is Alfred the Great, and he conquers the rest. And then his grandson, Athelstan, is the first man called King of England, king of all Angles. And then they do the same thing almost wherever they go. They help create modern France by ripping apart Charlemagne’s empire, which was unwieldy. It looked good on paper, but it was unwieldy. It was replaced by this leaner, meaner, compact thing. They figure out how to deal with the Vikings by essentially building fortified bridges, changes to their army, and so forth. The Vikings, I like to call it creative destruction. They, by destroying the things they destroyed, they cleared the ground for something stronger to grow.
But that king is Alfred the Great, and he conquers the rest. And then his grandson, Athelstan, is the first man called King of England, king of all Angles. And then they do the same thing almost wherever they go. They help create modern France by ripping apart Charlemagne’s empire, which was unwieldy. It looked good on paper, but it was unwieldy. It was replaced by this leaner, meaner, compact thing. They figure out how to deal with the Vikings by essentially building fortified bridges, changes to their army, and so forth. The Vikings, I like to call it creative destruction. They, by destroying the things they destroyed, they cleared the ground for something stronger to grow.
Lex Fridman
That’s brilliant. The creative destruction engine that created Europe was the Normans and the Vikings. And then you also, you have another book that talks about the Byzantine Empire, so you have the creative destruction that resulted in Europe, that Europe led to this Western, quote-unquote, civilization that we think of now. And the thing that protected Europe for centuries was the existence of the Byzantine Empire, the East Roman Empire, because of all the threats- … That came towards Europe. This strong, stable empire that is the Byzantine Empire protected the forces from everything that came from the east. They were a buffer.
That’s brilliant. The creative destruction engine that created Europe was the Normans and the Vikings. And then you also, you have another book that talks about the Byzantine Empire, so you have the creative destruction that resulted in Europe, that Europe led to this Western, quote-unquote, civilization that we think of now. And the thing that protected Europe for centuries was the existence of the Byzantine Empire, the East Roman Empire, because of all the threats- … That came towards Europe. This strong, stable empire that is the Byzantine Empire protected the forces from everything that came from the east. They were a buffer.
Lars Brownworth
They were a buffer, giving Europe this kind of vital time to develop the way it needed to develop.
They were a buffer, giving Europe this kind of vital time to develop the way it needed to develop.
Lex Fridman
So it’s interesting to think that the world as we see now was a results- … of a sequence of quite lucky geographical and leadership decisions in history. I mean, it really does pivot on a few points of geography and a few special leaders- … that conquer.
So it’s interesting to think that the world as we see now was a results- … of a sequence of quite lucky geographical and leadership decisions in history. I mean, it really does pivot on a few points of geography and a few special leaders- … that conquer.
Lars Brownworth
Yeah. Had Constantine chosen his side a little less wisely, the world’s gonna be very different.
Yeah. Had Constantine chosen his side a little less wisely, the world’s gonna be very different.
Lex Fridman
Yeah, so Constantine is the guy who moved the capital of the empire from Rome to Constantinople, thereby giving a lot more focus to the east. Thereby protecting Europe from the gigantic threats that- … loomed in the east.
Yeah, so Constantine is the guy who moved the capital of the empire from Rome to Constantinople, thereby giving a lot more focus to the east. Thereby protecting Europe from the gigantic threats that- … loomed in the east.
Lars Brownworth
That’s right. And the Islamic invasions of the seventh century, they couldn’t get past that choke point of Constantinople. So they had to take the long way across Africa. You know, and by the time they get to Spain and conquer Spain and into—that’s the Battle of Tours, you know—Charles Martel is able to stop them, and they’re, they’re massively overextended. You know, I think it’s a very different story if they can come in through the Black Sea.
That’s right. And the Islamic invasions of the seventh century, they couldn’t get past that choke point of Constantinople. So they had to take the long way across Africa. You know, and by the time they get to Spain and conquer Spain and into—that’s the Battle of Tours, you know—Charles Martel is able to stop them, and they’re, they’re massively overextended. You know, I think it’s a very different story if they can come in through the Black Sea.
Lex Fridman
And all the times the East Roman Empire almost died from all the invasions, all of those invaders would have just conquered the entirety of Europe.
And all the times the East Roman Empire almost died from all the invasions, all of those invaders would have just conquered the entirety of Europe.
Lars Brownworth
Yeah, I mean, I don’t think they would’ve met much resistance.
Yeah, I mean, I don’t think they would’ve met much resistance.
Viking religion and Valhalla
Lex Fridman
Yeah. So rewinding back, what was the religion, the religious beliefs, the gods that the Vikings believed that we’ve mentioned a little bit of? Thor and Odin, how did they see this, this world and the universe?
Yeah. So rewinding back, what was the religion, the religious beliefs, the gods that the Vikings believed that we’ve mentioned a little bit of? Thor and Odin, how did they see this, this world and the universe?
Lars Brownworth
It’s, so the Viking gods are… I mean, they’ve been sanitized, but they’re quite terrifying. But at their basic conception of the universe is an eternal struggle between chaos and order, which chaos will eventually win. So I think the best view of cosmology is of concentric circles, with Utgard as the outer realm, and that’s where the chaos is. And those are the, that’s where the frost giants are, all the monsters that seek to destroy. The gods represent order and stability, and the monsters represent chaos, and it’s an eternal war between the two of them.
It’s, so the Viking gods are… I mean, they’ve been sanitized, but they’re quite terrifying. But at their basic conception of the universe is an eternal struggle between chaos and order, which chaos will eventually win. So I think the best view of cosmology is of concentric circles, with Utgard as the outer realm, and that’s where the chaos is. And those are the, that’s where the frost giants are, all the monsters that seek to destroy. The gods represent order and stability, and the monsters represent chaos, and it’s an eternal war between the two of them.
Lars Brownworth
So there are different categories of gods depending on which circle you come from. The gods don’t all like each other. They’re not… Sometimes they engage in wars. Some of the most famous gods, the Norse gods, you know, Loki or Freya, come from outside the Aesir, the main gods. So it was kind of a fluid thing.
So there are different categories of gods depending on which circle you come from. The gods don’t all like each other. They’re not… Sometimes they engage in wars. Some of the most famous gods, the Norse gods, you know, Loki or Freya, come from outside the Aesir, the main gods. So it was kind of a fluid thing.
Lex Fridman
It’s more a way to understand the world.
It’s more a way to understand the world.
Lars Brownworth
I think so, yeah. The thunder is Thor fighting the ice giants, and that’s what that is.
I think so, yeah. The thunder is Thor fighting the ice giants, and that’s what that is.
Lex Fridman
Going to Perplexity. Vikings followed the polytheistic, ritual-heavy religion centered on a pantheon of gods and spirits with no single holy book or unified church, and practices varied a lot by region and family. And so the major gods were Odin and Thor and Freya. Odin was—his domain was war, kingship, wisdom, death. Thor was protection, thunder, fertility. Freya was love, magic, battle dead. Typical worshipers for Odin were chieftains and elite warriors and poets. Typical worshipers for Thor were farmers and “ordinary people,” and typical worshipers of Freya were women, magic practitioners, and lovers.
Going to Perplexity. Vikings followed the polytheistic, ritual-heavy religion centered on a pantheon of gods and spirits with no single holy book or unified church, and practices varied a lot by region and family. And so the major gods were Odin and Thor and Freya. Odin was—his domain was war, kingship, wisdom, death. Thor was protection, thunder, fertility. Freya was love, magic, battle dead. Typical worshipers for Odin were chieftains and elite warriors and poets. Typical worshipers for Thor were farmers and “ordinary people,” and typical worshipers of Freya were women, magic practitioners, and lovers.
Lars Brownworth
Yeah, yeah. I mean, I’ve heard it… I think you can break it down saying like, Odin was the elite. He’s kind of more aristocratic, right? He, he’s the god of poetry; you need to read, et cetera. Only the elite would know how to do that. A farmer wouldn’t really care about that. Wherein Thor is a more earthy god. You know, you want the waves to be less, you know, pray to Thor. I find Odin, I think, most disturbing. He’s the god of madness and the god of poetry, which, I guess those are related. But in battle, I mean, the Berserkers, probably the most famous type of Viking warriors, were considered to be Odin’s chosen warriors. They would show no pain, and they’d just run at the enemy and attack with their nails and their teeth.
Yeah, yeah. I mean, I’ve heard it… I think you can break it down saying like, Odin was the elite. He’s kind of more aristocratic, right? He, he’s the god of poetry; you need to read, et cetera. Only the elite would know how to do that. A farmer wouldn’t really care about that. Wherein Thor is a more earthy god. You know, you want the waves to be less, you know, pray to Thor. I find Odin, I think, most disturbing. He’s the god of madness and the god of poetry, which, I guess those are related. But in battle, I mean, the Berserkers, probably the most famous type of Viking warriors, were considered to be Odin’s chosen warriors. They would show no pain, and they’d just run at the enemy and attack with their nails and their teeth.
Lars Brownworth
Even if they could have their arms hacked off, they would still keep going. Like they would just… And they would attack other Vikings. They were berserk. That’s where we get the word from.
Even if they could have their arms hacked off, they would still keep going. Like they would just… And they would attack other Vikings. They were berserk. That’s where we get the word from.
Lex Fridman
What do we understand the mindset leads to that? I mean, it wasn’t religious in nature. There’s not this kinda ideology. It’s just the way of life and then the prized honor and intensity in battle.
What do we understand the mindset leads to that? I mean, it wasn’t religious in nature. There’s not this kinda ideology. It’s just the way of life and then the prized honor and intensity in battle.
Lars Brownworth
Yeah. I mean, one of Odin’s names is the raven feeder. I mean, you were, by creating corpses, which ravens feed on, you are, you’re doing the work of Odin. And, you know, the Viking view of the afterlife was unique. There weren’t really punishments, not really, for doing bad things. Unless you did something really bad. Then you ended up as basically an evil spirit haunting your grave. But if you were brave, then you got taken to the house of the dead, which is Valhalla to… And you were resurrected. Every day you would fight, and whatever wounds you got would be magically healed that night. And then the next morning you’d get up and do it again. So you’re essentially practicing for Ragnarok- … the final battle, which you would lose. So I’m not sure. It seems, it’s rather pessimistic.
Yeah. I mean, one of Odin’s names is the raven feeder. I mean, you were, by creating corpses, which ravens feed on, you are, you’re doing the work of Odin. And, you know, the Viking view of the afterlife was unique. There weren’t really punishments, not really, for doing bad things. Unless you did something really bad. Then you ended up as basically an evil spirit haunting your grave. But if you were brave, then you got taken to the house of the dead, which is Valhalla to… And you were resurrected. Every day you would fight, and whatever wounds you got would be magically healed that night. And then the next morning you’d get up and do it again. So you’re essentially practicing for Ragnarok- … the final battle, which you would lose. So I’m not sure. It seems, it’s rather pessimistic.
Lex Fridman
The battle’s what… I mean, it sounds like losing is not a thing. The battle itself is what matters. So Valhalla… It’s a place where you fight a battle every day.
The battle’s what… I mean, it sounds like losing is not a thing. The battle itself is what matters. So Valhalla… It’s a place where you fight a battle every day.
Lars Brownworth
Every day.
Every day.
Lex Fridman
Unlimited food, there’s like a boar or whatever.
Unlimited food, there’s like a boar or whatever.
Lars Brownworth
Yeah. Yeah, that’s right.
Yeah. Yeah, that’s right.
Lex Fridman
There’s unlimited wine.
There’s unlimited wine.
Lars Brownworth
Yep.
Yep.
Lex Fridman
And you can die as much as you want-
And you can die as much as you want-
Lars Brownworth
As much as you want, yeah.
As much as you want, yeah.
Lex Fridman
… and you’ll be born again.
… and you’ll be born again.
Lars Brownworth
Just like video games.
Just like video games.
Lex Fridman
And this is the idea of the highest… This, I guess, if there’s such a thing as heaven in this kind of construction of the universe, this is heaven.
And this is the idea of the highest… This, I guess, if there’s such a thing as heaven in this kind of construction of the universe, this is heaven.
Lars Brownworth
This is heaven, yeah.
This is heaven, yeah.
Lex Fridman
This is the highest form. This is the highest place you can go to: is Valhalla. Is fight every day, eat as much as you want, drink as much as you want, die, and are reborn the next day. And this is forever, preparing yourself for the final battle of Ragnarok.
This is the highest form. This is the highest place you can go to: is Valhalla. Is fight every day, eat as much as you want, drink as much as you want, die, and are reborn the next day. And this is forever, preparing yourself for the final battle of Ragnarok.
Lars Brownworth
Ragnarok.
Ragnarok.
Lex Fridman
So this is where… this is the end of the world, this is the cataclysm.
So this is where… this is the end of the world, this is the cataclysm.
Lars Brownworth
Mm-hmm. That’s right. Odin’s gonna die, Thor will die. He’ll get killed by one of Loki’s children, the Midgard Serpent. Odin will be devoured by a wolf. The sun and moon, which are being chased by monsters, by giants, will be caught and swallowed by the giants, plunging the world into eternal darkness. Essentially all the gods will die and darkness and chaos will then ensue. And then at the very end the… This is mostly from a guy named Snorri Sturluson who was living right at the end of the Viking Age and writing this. And he was, I believe, a Christian. So there’s… I think we’re fusing things here. So then there would be a new Earth and a new heaven and a new god, who’s all powerful.
Mm-hmm. That’s right. Odin’s gonna die, Thor will die. He’ll get killed by one of Loki’s children, the Midgard Serpent. Odin will be devoured by a wolf. The sun and moon, which are being chased by monsters, by giants, will be caught and swallowed by the giants, plunging the world into eternal darkness. Essentially all the gods will die and darkness and chaos will then ensue. And then at the very end the… This is mostly from a guy named Snorri Sturluson who was living right at the end of the Viking Age and writing this. And he was, I believe, a Christian. So there’s… I think we’re fusing things here. So then there would be a new Earth and a new heaven and a new god, who’s all powerful.
Lex Fridman
Yeah, if you think of religion as a kind of technology, a social technology that stabilizes or helps guide the evolution of a society, it’s interesting to see what the Vikings came up with. And do you ever think from a history, the grand view of history, how effective these different technologies of religion have been?
Yeah, if you think of religion as a kind of technology, a social technology that stabilizes or helps guide the evolution of a society, it’s interesting to see what the Vikings came up with. And do you ever think from a history, the grand view of history, how effective these different technologies of religion have been?
Lars Brownworth
Yeah, I mean, I think that’s certainly… I’m thinking of the Viking rituals. Hospitality is very important in a northern climate where, you know, food is scarce, winters are long and harsh. And if you don’t share your hearth with, you know, someone knocking on your door, then someone else might not share it with you and you could be facing death. So in this case hospitality becomes a core belief and- … you know, the idea was that Odin would travel incognito, knocking on people’s doors, and he would remember if you let him in or not. And if you were hospitable, he would bless you, and if you were inhospitable, he would murder you. And- … you know, I think these rituals are obviously intended for, “How do we survive this winter?”
Yeah, I mean, I think that’s certainly… I’m thinking of the Viking rituals. Hospitality is very important in a northern climate where, you know, food is scarce, winters are long and harsh. And if you don’t share your hearth with, you know, someone knocking on your door, then someone else might not share it with you and you could be facing death. So in this case hospitality becomes a core belief and- … you know, the idea was that Odin would travel incognito, knocking on people’s doors, and he would remember if you let him in or not. And if you were hospitable, he would bless you, and if you were inhospitable, he would murder you. And- … you know, I think these rituals are obviously intended for, “How do we survive this winter?”
Lex Fridman
Yeah, how do we effectively spread the message that hospitality is pretty? good thing, and it’s characteristic of religion. If you do a good thing, you’ll be rewarded. If you do a bad thing, you’ll be punished. And then different religions play the different ways of communicating that.
Yeah, how do we effectively spread the message that hospitality is pretty? good thing, and it’s characteristic of religion. If you do a good thing, you’ll be rewarded. If you do a bad thing, you’ll be punished. And then different religions play the different ways of communicating that.
Lars Brownworth
Yeah. I mean, I think also religion gives you, it gives you a world view, right? It gives you a morality and these are core parts of society.
Yeah. I mean, I think also religion gives you, it gives you a world view, right? It gives you a morality and these are core parts of society.
Lex Fridman
And the beautiful thing about religion is it interplays with human nature and it guides humans. But then of course, human nature and humans project themselves onto the religion, sometimes they use their religion. It’s to accomplish goals in a pragmatic sense, in a political sense, in a geopolitical sense, in a military sense, in a social sense. And so there’s that dance of how religion invigorates and guides the peoples, and then how the peoples use the religion to guide the direction of the world.
And the beautiful thing about religion is it interplays with human nature and it guides humans. But then of course, human nature and humans project themselves onto the religion, sometimes they use their religion. It’s to accomplish goals in a pragmatic sense, in a political sense, in a geopolitical sense, in a military sense, in a social sense. And so there’s that dance of how religion invigorates and guides the peoples, and then how the peoples use the religion to guide the direction of the world.
Lex Fridman
And that’s certainly the history of Christianity has a big role to play in the history of Europe, in the history of the Byzantine Empire and that part of the world. And it was an incredibly effective religion once Constantine converted. It spread extremely quickly, relatively speaking, across a couple of centuries. Just to linger on the Viking views of the world and the afterlife. So we mentioned Valhalla. There’s the Norns, which are the three spirits that represent the past, the present, and the necessity. They spin the fates of all men and gods at the Roots of Yggdrasil.
And that’s certainly the history of Christianity has a big role to play in the history of Europe, in the history of the Byzantine Empire and that part of the world. And it was an incredibly effective religion once Constantine converted. It spread extremely quickly, relatively speaking, across a couple of centuries. Just to linger on the Viking views of the world and the afterlife. So we mentioned Valhalla. There’s the Norns, which are the three spirits that represent the past, the present, and the necessity. They spin the fates of all men and gods at the Roots of Yggdrasil.
Lars Brownworth
Yggdrasil.
Yggdrasil.
Lex Fridman
Yggdrasil. So there’s a notion of like determinism and fate to the Viking life. And there’s Valhalla, there’s Hel, Niflheim. This was the destination for the vast majority of people. So if you don’t make it to Valhalla, this is where you go.
Yggdrasil. So there’s a notion of like determinism and fate to the Viking life. And there’s Valhalla, there’s Hel, Niflheim. This was the destination for the vast majority of people. So if you don’t make it to Valhalla, this is where you go.
Lars Brownworth
That’s where you go.
That’s where you go.
Lex Fridman
Unless you’re a real bad person, then there’s some punishment for the truly wicked.
Unless you’re a real bad person, then there’s some punishment for the truly wicked.
Lars Brownworth
And we should point out that Hel, spelled with one L was a daughter of Loki- … And was not the same as the…
And we should point out that Hel, spelled with one L was a daughter of Loki- … And was not the same as the…
Lex Fridman
Hell with the two L’s.
Hell with the two L’s.
Lars Brownworth
Hell with the two L’s. Very different.
Hell with the two L’s. Very different.
Lex Fridman
It’s more like purgatory type of situation.
It’s more like purgatory type of situation.
Lars Brownworth
Yeah, so it’s the house of the, it’s like the house of the dead, the house of the underworld.
Yeah, so it’s the house of the, it’s like the house of the dead, the house of the underworld.
Lex Fridman
A colorless twilight, not necessarily a place of punishment but simply the inevitable end for most, unless you end up in Valhalla, which means you’re a great warrior dying in battle.
A colorless twilight, not necessarily a place of punishment but simply the inevitable end for most, unless you end up in Valhalla, which means you’re a great warrior dying in battle.
Lars Brownworth
It reminds me of the Greek view of the afterlife, right? Where you essentially get amnesia and forget who you are unless someone makes a sacrifice and says your name, and only then you’ll remember it. So your- your destiny is ultimately to just become gray and fade away. So you might as well- you might as well be brave. You might as well run at that spear.
It reminds me of the Greek view of the afterlife, right? Where you essentially get amnesia and forget who you are unless someone makes a sacrifice and says your name, and only then you’ll remember it. So your- your destiny is ultimately to just become gray and fade away. So you might as well- you might as well be brave. You might as well run at that spear.
Lex Fridman
So that was the engine of their- the warrior culture that was core to their society.
So that was the engine of their- the warrior culture that was core to their society.
Lars Brownworth
I think probably.
I think probably.
Viking explorers
Lex Fridman
I have to ask about Vikings as explorers. They were… truly one of the greatest explorers in history. What can you say to, what is it in their spirit that motivated them? I mean, they sailed, they reached North America 500 years before Columbus. They sailed obviously to England, Spain, Italy, Russia, North Africa, the Middle East, Paris, and I’m just showing here a map of the ocean routes and the river systems that they connected to and sailed. What do you think drove them to explore the unknown?
I have to ask about Vikings as explorers. They were… truly one of the greatest explorers in history. What can you say to, what is it in their spirit that motivated them? I mean, they sailed, they reached North America 500 years before Columbus. They sailed obviously to England, Spain, Italy, Russia, North Africa, the Middle East, Paris, and I’m just showing here a map of the ocean routes and the river systems that they connected to and sailed. What do you think drove them to explore the unknown?
Lars Brownworth
This boggles my mind. This, like this map here, just… it messes with me because they didn’t have a compass. I mean, can you imagine shoving off from some fjord in Norway west? That’s your only—west. And there was a Viking named Naddodd. He’s actually the first Norseman to reach Iceland, though it was a total accident. But here’s the, here’s the mind-blowing part. He decides to land and explore, and he gets off and he sees two humans. They’re monks from Ireland. They got there in a canoe. You look at Ireland, look at Iceland, that’s even more impressive. They got in a canoe, a skin boat- … and they just went north because they were trying to get away from the world.
This boggles my mind. This, like this map here, just… it messes with me because they didn’t have a compass. I mean, can you imagine shoving off from some fjord in Norway west? That’s your only—west. And there was a Viking named Naddodd. He’s actually the first Norseman to reach Iceland, though it was a total accident. But here’s the, here’s the mind-blowing part. He decides to land and explore, and he gets off and he sees two humans. They’re monks from Ireland. They got there in a canoe. You look at Ireland, look at Iceland, that’s even more impressive. They got in a canoe, a skin boat- … and they just went north because they were trying to get away from the world.
Lars Brownworth
They found Iceland, and in a very excellent move on their part, they ran away as soon as the Vikings arrived, which is, you know, pretty smart.
They found Iceland, and in a very excellent move on their part, they ran away as soon as the Vikings arrived, which is, you know, pretty smart.
Lex Fridman
I don’t know if you know there’s this video of the deranged penguin with the Werner Herzog documentary, where Werner Herzog is like overdubbing, explaining the thinking of the penguin. But the penguin leaves the tribe and he just goes out into the mountains. I have to show you this video. This is my favorite video of all time. There’s this low-key documentary where they’re talking about penguins, and then there’s one penguin that leaves-
I don’t know if you know there’s this video of the deranged penguin with the Werner Herzog documentary, where Werner Herzog is like overdubbing, explaining the thinking of the penguin. But the penguin leaves the tribe and he just goes out into the mountains. I have to show you this video. This is my favorite video of all time. There’s this low-key documentary where they’re talking about penguins, and then there’s one penguin that leaves-
Lars Brownworth
I’m out.
I’m out.
Lex Fridman
… leaves the tribe and just goes towards the mountains, and as Werner Herzog says, “Towards certain death.” It always reminds me of this kind of Viking spirit or the- … or the monk spirit. There’s something—one human or a small group of humans just decide to go.
… leaves the tribe and just goes towards the mountains, and as Werner Herzog says, “Towards certain death.” It always reminds me of this kind of Viking spirit or the- … or the monk spirit. There’s something—one human or a small group of humans just decide to go.
Lars Brownworth
Just go, yeah.
Just go, yeah.
Lex Fridman
And not look back.
And not look back.
Lars Brownworth
Are there sea monsters out there? Maybe.
Are there sea monsters out there? Maybe.
Lex Fridman
Maybe.
Maybe.
Lars Brownworth
Is there any land? Are we gonna fall off the edge of the earth? Maybe. I-
Is there any land? Are we gonna fall off the edge of the earth? Maybe. I-
Lex Fridman
And just as Werner Herzog says, you know, “There’s certain death.” Now, he doesn’t romanticize it. He says the penguin is just deranged and crazy. But look, the penguin did look back briefly.
And just as Werner Herzog says, you know, “There’s certain death.” Now, he doesn’t romanticize it. He says the penguin is just deranged and crazy. But look, the penguin did look back briefly.
Lars Brownworth
Right.
Right.
Lex Fridman
He did think about this. So this- … there’s multiple ways, but you just highlighted two ways to explore. One is ’cause you’re this hardcore dude that just is looking to raid and just goes and goes and just you have the resilience and the will- … to keep going. And then there’s the monks that just want to leave.
He did think about this. So this- … there’s multiple ways, but you just highlighted two ways to explore. One is ’cause you’re this hardcore dude that just is looking to raid and just goes and goes and just you have the resilience and the will- … to keep going. And then there’s the monks that just want to leave.
Lars Brownworth
Escape, yeah. Yeah.
Escape, yeah. Yeah.
Lex Fridman
They just go toward the, they want to leave far away- … so they could be closer to God. They could be closer to themselves and to-
They just go toward the, they want to leave far away- … so they could be closer to God. They could be closer to themselves and to-
Lars Brownworth
And away from sin. Yeah. You know, there’s this poem by Tennyson, “Ulysses,” my favorite poem. I think it captures the Viking spirit. The last line of it is “to strive, to seek, to find, and not to yield.” I think that’s very much like the Viking, you know, “My purpose holds, to sail beyond the baths of all the western stars until I die.” You may die, but I’m gonna do this, and I’m not gonna yield.
And away from sin. Yeah. You know, there’s this poem by Tennyson, “Ulysses,” my favorite poem. I think it captures the Viking spirit. The last line of it is “to strive, to seek, to find, and not to yield.” I think that’s very much like the Viking, you know, “My purpose holds, to sail beyond the baths of all the western stars until I die.” You may die, but I’m gonna do this, and I’m not gonna yield.
Lex Fridman
That spirit is one of my favorite aspects of human beings.
That spirit is one of my favorite aspects of human beings.
Lars Brownworth
I think that’s why the Vikings remain so popular today, you know? We name our satellites, our football teams, you know, our cruise ships. There’s this like, there’s this romantic hook- … of a people who did not yield.
I think that’s why the Vikings remain so popular today, you know? We name our satellites, our football teams, you know, our cruise ships. There’s this like, there’s this romantic hook- … of a people who did not yield.
Lex Fridman
Yeah, they embodied the part, the flame that burns in all of us that we admire most about human beings. It’s that, like, unyielding focus on going out there, of taking a leap into the unknown, into the scary, and never stopping.
Yeah, they embodied the part, the flame that burns in all of us that we admire most about human beings. It’s that, like, unyielding focus on going out there, of taking a leap into the unknown, into the scary, and never stopping.
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
That.
That.
Lars Brownworth
It’s not too late to seek a newer world.
It’s not too late to seek a newer world.
Lex Fridman
I have to ask you about, speaking of a newer world, America.
I have to ask you about, speaking of a newer world, America.
Lars Brownworth
Yes.
Yes.
Lex Fridman
And Leif Ericson. But first, a quick bathroom break if it’s okay. Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Larridin for measuring AI adoption in your business, BetterHelp for mental health, LMNT for electrolytes, Fin for customer service AI agents, Shopify for selling stuff online, and Perplexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Lars Brownworth. All right, we’re back.
And Leif Ericson. But first, a quick bathroom break if it’s okay. Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Larridin for measuring AI adoption in your business, BetterHelp for mental health, LMNT for electrolytes, Fin for customer service AI agents, Shopify for selling stuff online, and Perplexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Lars Brownworth. All right, we’re back.
Vikings in North America
Lex Fridman
Let’s talk about this incredible fact of the Vikings, that Leif Ericson, who was a Viking explorer, was the first European to reach North America around the year 1000, five centuries before Columbus reached North America. Tell the story of his journey. What do we know about him?
Let’s talk about this incredible fact of the Vikings, that Leif Ericson, who was a Viking explorer, was the first European to reach North America around the year 1000, five centuries before Columbus reached North America. Tell the story of his journey. What do we know about him?
Lars Brownworth
So let’s begin with his dad. His dad’s name is Erik the Red, who was forced to flee Norway when he was probably 10 years old because his dad had killed some people. It’s kind of hilarious. In the saga it says, “For a few killings…” Okay, I guess that’s a thing. So he went to Iceland and he got a farm in Iceland which was already starting to become overpopulated. They had cut down all the trees. There were some climate problems of deforestation and farms just blowing away, so the population was essentially beginning to crash in Iceland. And he got into a fight with his neighbor and ended up killing his neighbor, and so he was exiled from Iceland. He was exiled from the place his father had been exiled from.
So let’s begin with his dad. His dad’s name is Erik the Red, who was forced to flee Norway when he was probably 10 years old because his dad had killed some people. It’s kind of hilarious. In the saga it says, “For a few killings…” Okay, I guess that’s a thing. So he went to Iceland and he got a farm in Iceland which was already starting to become overpopulated. They had cut down all the trees. There were some climate problems of deforestation and farms just blowing away, so the population was essentially beginning to crash in Iceland. And he got into a fight with his neighbor and ended up killing his neighbor, and so he was exiled from Iceland. He was exiled from the place his father had been exiled from.
Lex Fridman
So it runs in the family, this whole outlaw thing.
So it runs in the family, this whole outlaw thing.
Lars Brownworth
What also ran in the family apparently was this streak, this courageous streak. And he had heard that there had been people… So the Norwegian Vikings, they were aiming for England, and they hit the Hebrides, which are these kind of treeless islands above Scotland, and they found they were good for refueling ’cause they’d pick up water or whatever, and then on your way to Scotland to raid. And then a Viking had missed the Hebrides and discovered Iceland, and then another Viking had aimed for Iceland, missed, and hit Greenland. And a little fun fact about Greenland, it is both north, south, east, and west of Iceland. So it’s any direction, you’re gonna hit Greenland.
What also ran in the family apparently was this streak, this courageous streak. And he had heard that there had been people… So the Norwegian Vikings, they were aiming for England, and they hit the Hebrides, which are these kind of treeless islands above Scotland, and they found they were good for refueling ’cause they’d pick up water or whatever, and then on your way to Scotland to raid. And then a Viking had missed the Hebrides and discovered Iceland, and then another Viking had aimed for Iceland, missed, and hit Greenland. And a little fun fact about Greenland, it is both north, south, east, and west of Iceland. So it’s any direction, you’re gonna hit Greenland.
Lex Fridman
So Greenland is hard to miss.
So Greenland is hard to miss.
Lars Brownworth
It’s hard to miss, which is not to take away anything from the extraordinary danger, the certain death of going further west. But there was this… By this time, there was this idea that, you know, enough people had become famous by sailing west into the unknown and discovering things, that I think there was a general idea of there’s more out there to the west. And so he had talked to someone who had seen Greenland and reported that there was this good land further west. And so he hired the ship’s crew of that Viking. So it’s kind of the deck was loaded, and he went to Greenland where he was able to settle two different colonies. One was called the Western Settlement in the west, and one was called the Eastern Settlement in essentially the extreme south.
It’s hard to miss, which is not to take away anything from the extraordinary danger, the certain death of going further west. But there was this… By this time, there was this idea that, you know, enough people had become famous by sailing west into the unknown and discovering things, that I think there was a general idea of there’s more out there to the west. And so he had talked to someone who had seen Greenland and reported that there was this good land further west. And so he hired the ship’s crew of that Viking. So it’s kind of the deck was loaded, and he went to Greenland where he was able to settle two different colonies. One was called the Western Settlement in the west, and one was called the Eastern Settlement in essentially the extreme south.
Lars Brownworth
And that was essentially the edges of where Viking technology could be. A cool factoid is that the Vikings practiced husbandry, raised animals, and obviously this is not an option in Greenland, although they couldn’t have known it at the time. But they brought plants with them. So, and then they were able to trade with the native Inuit for walrus blubber and things like that, and they made a go of it. But what’s obvious, you know, anyone who’s seen Greenland, there are no trees. It’s almost impossible to survive by practicing husbandry. It is impossible to survive, as it turns out, just practicing husbandry. And by this point, I think this extraordinary Viking pragmatism is beginning to be played out.
And that was essentially the edges of where Viking technology could be. A cool factoid is that the Vikings practiced husbandry, raised animals, and obviously this is not an option in Greenland, although they couldn’t have known it at the time. But they brought plants with them. So, and then they were able to trade with the native Inuit for walrus blubber and things like that, and they made a go of it. But what’s obvious, you know, anyone who’s seen Greenland, there are no trees. It’s almost impossible to survive by practicing husbandry. It is impossible to survive, as it turns out, just practicing husbandry. And by this point, I think this extraordinary Viking pragmatism is beginning to be played out.
Lars Brownworth
Because one of the reasons the Greenland experiment fails ultimately in 300 years is they fail to adapt. It’s clearly they should, they should focus more on fishing, on other sources than, than just raising pigs and cows. But-
Because one of the reasons the Greenland experiment fails ultimately in 300 years is they fail to adapt. It’s clearly they should, they should focus more on fishing, on other sources than, than just raising pigs and cows. But-
Lex Fridman
Oh, so we hit the limit of the Viking adaptability which they have demonstrated throughout the world, I’d say. Interesting.
Oh, so we hit the limit of the Viking adaptability which they have demonstrated throughout the world, I’d say. Interesting.
Lars Brownworth
So Erik the Red is this, he makes his name by exploring, and he does in fact, once he discovers Greenland, he calls it green. He says there’s so many salmon in the rivers of, in the fjords that you can just scoop them out with your hands. You don’t even have to fish.
So Erik the Red is this, he makes his name by exploring, and he does in fact, once he discovers Greenland, he calls it green. He says there’s so many salmon in the rivers of, in the fjords that you can just scoop them out with your hands. You don’t even have to fish.
Lex Fridman
Was this real?
Was this real?
Lars Brownworth
It’s a lie.
It’s a lie.
Lex Fridman
Okay.
Okay.
Lars Brownworth
That’s not true at all.
That’s not true at all.
Lex Fridman
So he’s doing propaganda.
So he’s doing propaganda.
Lars Brownworth
He’s doing propaganda.
He’s doing propaganda.
Lex Fridman
So is that… Is this story true that he called it green just so he can attract-
So is that… Is this story true that he called it green just so he can attract-
Lars Brownworth
It is.
It is.
Lex Fridman
So-
So-
Lars Brownworth
The greatest real estate scam in history. Yeah. Yeah.
The greatest real estate scam in history. Yeah. Yeah.
Lex Fridman
Okay. Genius. I mean, just stuck to this day.
Okay. Genius. I mean, just stuck to this day.
Lars Brownworth
Yeah. It’s the most misnamed place in the world. But in the Europe of the time, even in Iceland, the dream was to have land. I mean, land equaled wealth in Europe. And here he says there’s enough land for the taking, like anyone who wants it, which is true. It’s the largest island on earth. I mean, it’s unusable, but it should be called Iceland, Glacier Land or something. But it worked. He took 500 men with him from Iceland. It’s gotta be a significant chunk of the population, but there’s enough people, kind of land hungry, there’s no more room in Iceland. It’s too restrictive. We’re gonna go further west. So he takes 25 ships and then 14 make it, which is pretty good. And then those 14 ships with their 300 or so people start the western colony.
Yeah. It’s the most misnamed place in the world. But in the Europe of the time, even in Iceland, the dream was to have land. I mean, land equaled wealth in Europe. And here he says there’s enough land for the taking, like anyone who wants it, which is true. It’s the largest island on earth. I mean, it’s unusable, but it should be called Iceland, Glacier Land or something. But it worked. He took 500 men with him from Iceland. It’s gotta be a significant chunk of the population, but there’s enough people, kind of land hungry, there’s no more room in Iceland. It’s too restrictive. We’re gonna go further west. So he takes 25 ships and then 14 make it, which is pretty good. And then those 14 ships with their 300 or so people start the western colony.
Lars Brownworth
And then word gets back to Norway, but Norway’s 2,000 miles away, 2,000-plus miles away. So it’s, you know, contact… They’re having to get resupplied. Obviously in the first winter, all their cattle die. That’s not a great, that’s not a great start for people who practice husbandry. So they’ve got to get resupplied from Norway, but, you know, the chances of making it to Norway and back are actually not that great if you’re sailing without a compass. You’re just kind of hoping. But they do it. They do it. And the colonies survive until the 1400s where they just go silent.
And then word gets back to Norway, but Norway’s 2,000 miles away, 2,000-plus miles away. So it’s, you know, contact… They’re having to get resupplied. Obviously in the first winter, all their cattle die. That’s not a great, that’s not a great start for people who practice husbandry. So they’ve got to get resupplied from Norway, but, you know, the chances of making it to Norway and back are actually not that great if you’re sailing without a compass. You’re just kind of hoping. But they do it. They do it. And the colonies survive until the 1400s where they just go silent.
Lex Fridman
So let’s talk about Erik the Red’s son, Leif Ericson. How does the journey continue west?
So let’s talk about Erik the Red’s son, Leif Ericson. How does the journey continue west?
Lars Brownworth
So Erik is getting a little older. The Greenland settlements are becoming filled up. Erik is happy where he is. He’s been kicked out of enough places. He’s made his home here, and this is where he wants to be. But his son… They’re running out of resources. There’s no wood. You know, there’s limited food, et cetera, et cetera. And so his son proposes going west because he’s heard stories that there are other lands. So another Viking had gotten lost, aimed for Greenland and missed, and had seen something. He said he saw clouds and mountains and there’s land there. And then he had turned around. And Leif again did the same thing, he hired the man’s crew.
So Erik is getting a little older. The Greenland settlements are becoming filled up. Erik is happy where he is. He’s been kicked out of enough places. He’s made his home here, and this is where he wants to be. But his son… They’re running out of resources. There’s no wood. You know, there’s limited food, et cetera, et cetera. And so his son proposes going west because he’s heard stories that there are other lands. So another Viking had gotten lost, aimed for Greenland and missed, and had seen something. He said he saw clouds and mountains and there’s land there. And then he had turned around. And Leif again did the same thing, he hired the man’s crew.
Lars Brownworth
He asked his dad to come, his dad wouldn’t. He went with his half sister Freydis, who was a whole nother story by herself, and a bunch of other colonists, and they went, and they landed in a place. He called it Vinland because he found things that he could ferment. So, of course, the Vikings, they made wine, or a wine-like alcohol. So Leif Ericsson is, he’s landed. He doesn’t know this, but he’s landed on a new continent with essentially inexhaustible stores of food and timber and everything he needs. It’s the perfect place. Unfortunately for him, it’s also inhabited by some natives, probably the Algonquin tribe. He calls them the Skraelings, which is just Norse for screechers because he can’t understand their language.
He asked his dad to come, his dad wouldn’t. He went with his half sister Freydis, who was a whole nother story by herself, and a bunch of other colonists, and they went, and they landed in a place. He called it Vinland because he found things that he could ferment. So, of course, the Vikings, they made wine, or a wine-like alcohol. So Leif Ericsson is, he’s landed. He doesn’t know this, but he’s landed on a new continent with essentially inexhaustible stores of food and timber and everything he needs. It’s the perfect place. Unfortunately for him, it’s also inhabited by some natives, probably the Algonquin tribe. He calls them the Skraelings, which is just Norse for screechers because he can’t understand their language.
Lars Brownworth
They just yell at them and attack immediately. They stay there for three years and then give up and go back home. So ultimately, it… and then really don’t tell anyone about it. They just keep it in their northern sagas.
They just yell at them and attack immediately. They stay there for three years and then give up and go back home. So ultimately, it… and then really don’t tell anyone about it. They just keep it in their northern sagas.
Lex Fridman
Why do you think they left? Why do you think they didn’t stick around longer?
Why do you think they left? Why do you think they didn’t stick around longer?
Lars Brownworth
I think there are a number of things working against them. Of course, I would like to believe there’s an alternate history where the Vikings successfully make it down, you know, maybe down to Maryland or something, and there’s an alternate history of the US and Canada here, but I think there’s a number of things working against them. The first is they stubbornly refused to give up husbandry, so they’re trying to make this work. L’Anse aux Meadows I think is where they were, in Newfoundland. It doesn’t work. The climate’s too cold. It’s not even—the grasses aren’t appropriate, you know, it’s just not gonna work and they do not adapt, number one.
I think there are a number of things working against them. Of course, I would like to believe there’s an alternate history where the Vikings successfully make it down, you know, maybe down to Maryland or something, and there’s an alternate history of the US and Canada here, but I think there’s a number of things working against them. The first is they stubbornly refused to give up husbandry, so they’re trying to make this work. L’Anse aux Meadows I think is where they were, in Newfoundland. It doesn’t work. The climate’s too cold. It’s not even—the grasses aren’t appropriate, you know, it’s just not gonna work and they do not adapt, number one.
Lars Brownworth
Number two, they’re 2,000-plus miles away from Norway and getting resupplied, and although they are extremely good sailors and explorers and traders, I think this is a little too far. And then thirdly is the native resistance. It’s just too incessant. They are, they are outnumbered, you know, millions to one. And the Algonquin do not want them there. It’s clear, and they’re not gonna stop attacking.
Number two, they’re 2,000-plus miles away from Norway and getting resupplied, and although they are extremely good sailors and explorers and traders, I think this is a little too far. And then thirdly is the native resistance. It’s just too incessant. They are, they are outnumbered, you know, millions to one. And the Algonquin do not want them there. It’s clear, and they’re not gonna stop attacking.
Lex Fridman
It’s so fascinating because they really didn’t understand the full scale of the land they’ve encountered, right?
It’s so fascinating because they really didn’t understand the full scale of the land they’ve encountered, right?
Lars Brownworth
That’s right. That’s right. I mean, had they known, had he known- … what he had found-
That’s right. That’s right. I mean, had they known, had he known- … what he had found-
Lex Fridman
That there’s more south. Maybe they, their intuition-
That there’s more south. Maybe they, their intuition-
Lars Brownworth
That’s right
That’s right
Lex Fridman
… was like there’s not… it’s just all northern land, it’s void of resources. We can’t do the whole husbandry thing. But you would think they could go down the coast.
… was like there’s not… it’s just all northern land, it’s void of resources. We can’t do the whole husbandry thing. But you would think they could go down the coast.
Lars Brownworth
I mean, if they could have gotten enough people from Norway, you know, or Iceland or whatever, you know, a sizable enough colony, and build some kind of defenses to fight off the incessant attacks- … then I think that’s the difference there, ’cause there certainly the resources are all there.
I mean, if they could have gotten enough people from Norway, you know, or Iceland or whatever, you know, a sizable enough colony, and build some kind of defenses to fight off the incessant attacks- … then I think that’s the difference there, ’cause there certainly the resources are all there.
Lex Fridman
Mm-hmm. Or just keep staying in the water, keep going down the coast- … not necessarily camp out until, until you get further south. It is fascinating to think about that alternate history where they would have discovered America and settled there. So this is 500 years before Columbus. There’s… first of all, they could have done a lot of the stuff we think about the European nations doing, including brutality towards the natives. But there could have been a coexistence also, and some of the diseases that come with them could have done the damage that they did 500 years later. But now, it would have stabilized the populations to where the Europeans, the French, the Spanish, and so on who come, the natives would be more ready. So they would…
Mm-hmm. Or just keep staying in the water, keep going down the coast- … not necessarily camp out until, until you get further south. It is fascinating to think about that alternate history where they would have discovered America and settled there. So this is 500 years before Columbus. There’s… first of all, they could have done a lot of the stuff we think about the European nations doing, including brutality towards the natives. But there could have been a coexistence also, and some of the diseases that come with them could have done the damage that they did 500 years later. But now, it would have stabilized the populations to where the Europeans, the French, the Spanish, and so on who come, the natives would be more ready. So they would…
Lex Fridman
Europe would then encounter a sizable population of the Viking descendants and the natives to where the, the two could hold onto the land and bring a different kind of civilization there. Because ultimately Europe, with the European ways, of the Western civilization, expanded out into North America, but there could be this whole Scandinavian vibe- … that would have taken over.
Europe would then encounter a sizable population of the Viking descendants and the natives to where the, the two could hold onto the land and bring a different kind of civilization there. Because ultimately Europe, with the European ways, of the Western civilization, expanded out into North America, but there could be this whole Scandinavian vibe- … that would have taken over.
Lars Brownworth
Just a hair’s breadth. My favorite museum in New York is called The Cloisters. It’s part of the Met, and in the Cloisters, there’s an ivory cross. And the ivory cross has been richly carved with Christian scenes. It was carved in England, but it’s made of walrus ivory, and they got it from the New World. And the Viking, you know, Viking traders. It represents, you know, the great arc of the northern trade. So it’s walrus ivory from the New World via Norway to England to New York. It’s a great symbol of that trade.
Just a hair’s breadth. My favorite museum in New York is called The Cloisters. It’s part of the Met, and in the Cloisters, there’s an ivory cross. And the ivory cross has been richly carved with Christian scenes. It was carved in England, but it’s made of walrus ivory, and they got it from the New World. And the Viking, you know, Viking traders. It represents, you know, the great arc of the northern trade. So it’s walrus ivory from the New World via Norway to England to New York. It’s a great symbol of that trade.
Lex Fridman
This whole just period of thousands of years of exploration that we no longer can do, so it’s kind of geographic exploration of the world, is fascinating. It takes true courage. It takes true wonder. The kind of exploration we could do now is more in the scientific realm and the realm of ideas and then maybe in terms of geography out into space and exploring the universe.
This whole just period of thousands of years of exploration that we no longer can do, so it’s kind of geographic exploration of the world, is fascinating. It takes true courage. It takes true wonder. The kind of exploration we could do now is more in the scientific realm and the realm of ideas and then maybe in terms of geography out into space and exploring the universe.
Lars Brownworth
Yeah, I think the closest analog is probably Mars, right? I mean, what would it take for you to be like, “All right, I’m gonna leave and I’m gonna go to Mars”? You’re never coming back. There’s nothing there as far as you know. You know, all the accoutrements of civilization are not there. That’s the kind of courage you would have taken.
Yeah, I think the closest analog is probably Mars, right? I mean, what would it take for you to be like, “All right, I’m gonna leave and I’m gonna go to Mars”? You’re never coming back. There’s nothing there as far as you know. You know, all the accoutrements of civilization are not there. That’s the kind of courage you would have taken.
Lex Fridman
Yeah, but there’s, on top of that, with Greenland, with Iceland, with Finland, there’s just so much uncertainty, like literally what’s beyond this hill. So with Mars, everything is mapped. So it’s really you understand the full harshness of the situation.
Yeah, but there’s, on top of that, with Greenland, with Iceland, with Finland, there’s just so much uncertainty, like literally what’s beyond this hill. So with Mars, everything is mapped. So it’s really you understand the full harshness of the situation.
Lars Brownworth
Of what you’re gonna face, yeah.
Of what you’re gonna face, yeah.
Lex Fridman
So it’s more, that’s more akin to like, all right, I’m running an ultra-marathon. I understand the challenge. I think more akin would be like traveling out into like the Oort cloud, beyond the solar system.
So it’s more, that’s more akin to like, all right, I’m running an ultra-marathon. I understand the challenge. I think more akin would be like traveling out into like the Oort cloud, beyond the solar system.
Lars Brownworth
What’s scarier, the known or the unknown?
What’s scarier, the known or the unknown?
Lex Fridman
I think that deeply, the human nature pulls us towards the unknown.
I think that deeply, the human nature pulls us towards the unknown.
Lars Brownworth
No, it’s true. Yeah.
No, it’s true. Yeah.
Vikings in the East
Lex Fridman
All right. Speaking of which, going to the East. So like we mentioned, the Vikings really went all over. And one of the directions they went that ended up touching the Byzantine Empire in Constantinople, is they went East. What can you say about the 8th century journey East in the river networks that the Vikings did, the Swedish Vikings, the Varangians, as they began to explore the river systems of Russia?
All right. Speaking of which, going to the East. So like we mentioned, the Vikings really went all over. And one of the directions they went that ended up touching the Byzantine Empire in Constantinople, is they went East. What can you say about the 8th century journey East in the river networks that the Vikings did, the Swedish Vikings, the Varangians, as they began to explore the river systems of Russia?
Lars Brownworth
So this was the most surprising part for me when I was first thinking about writing the book and, you know, discovering where the Vikings went. I never… In a million years, it would’ve never occurred to me that the Vikings went East. But a good way to think of this is the Vikings launched themselves in whatever direction their country is facing. So Sweden goes to the East, Denmark goes down toward Germany, and Norway goes England and the New World. So there’s a Viking named Rorik who goes East and manages to set up an encampment on this lake called Staraya Ladoga.
So this was the most surprising part for me when I was first thinking about writing the book and, you know, discovering where the Vikings went. I never… In a million years, it would’ve never occurred to me that the Vikings went East. But a good way to think of this is the Vikings launched themselves in whatever direction their country is facing. So Sweden goes to the East, Denmark goes down toward Germany, and Norway goes England and the New World. So there’s a Viking named Rorik who goes East and manages to set up an encampment on this lake called Staraya Ladoga.
Lex Fridman
Which is a launchpad to both the Volga River and the Dnieper River.
Which is a launchpad to both the Volga River and the Dnieper River.
Lars Brownworth
Yeah, and these are major river systems in the East that take you all the way down to the Black Sea and the Caspian Sea. Because the Vikings, you know, such sea-born people, they can sail up rivers. This allows them access to the caliphates in the East and to the Byzantine Empire, where they, being Vikings, immediately decide to attack the city. The Byzantines essentially set the Sea of Marmara outside of Constantinople on fire- … and burn up all the Viking ships. So then the Vikings decide, “Okay, we can’t, we can’t take Constantinople, so we might as well join ’em if we can’t beat ’em.” And they end up as probably the most famous guard in Byzantine history, the Varangian Guard. Varangian means the men of the oath.
Yeah, and these are major river systems in the East that take you all the way down to the Black Sea and the Caspian Sea. Because the Vikings, you know, such sea-born people, they can sail up rivers. This allows them access to the caliphates in the East and to the Byzantine Empire, where they, being Vikings, immediately decide to attack the city. The Byzantines essentially set the Sea of Marmara outside of Constantinople on fire- … and burn up all the Viking ships. So then the Vikings decide, “Okay, we can’t, we can’t take Constantinople, so we might as well join ’em if we can’t beat ’em.” And they end up as probably the most famous guard in Byzantine history, the Varangian Guard. Varangian means the men of the oath.
Lars Brownworth
Or the men who’ve sworn an oath. This is kind of an analog of the Praetorian Guard in Ancient Rome. They were famously loyal to the throne, but not necessarily to the person sitting on the throne. They’re major power players. The last of the great Byzantine emperors, Basil the Bulgar-Slayer forms them in the late 900s. And they’re there with the history all the way up until the end of it. In fact, many of our famous Vikings, Harald Hardrada, serve in the Varangian Guard. If you go to Constantinople today, inside the Church of the Hagia Sophia, on the second floor there’s a marble balcony, and on the railings, you can find Norse runes that are carved in by Varangian Guards who were bored during a particularly long sermon, in a language they didn’t understand, but they had to stand there.
Or the men who’ve sworn an oath. This is kind of an analog of the Praetorian Guard in Ancient Rome. They were famously loyal to the throne, but not necessarily to the person sitting on the throne. They’re major power players. The last of the great Byzantine emperors, Basil the Bulgar-Slayer forms them in the late 900s. And they’re there with the history all the way up until the end of it. In fact, many of our famous Vikings, Harald Hardrada, serve in the Varangian Guard. If you go to Constantinople today, inside the Church of the Hagia Sophia, on the second floor there’s a marble balcony, and on the railings, you can find Norse runes that are carved in by Varangian Guards who were bored during a particularly long sermon, in a language they didn’t understand, but they had to stand there.
Lex Fridman
So, that’s a fascinating thing, which is the Varangian Guard, guarding the emperor of the East Roman Empire, is made up initially, for quite a bit of time, of Vikings.
So, that’s a fascinating thing, which is the Varangian Guard, guarding the emperor of the East Roman Empire, is made up initially, for quite a bit of time, of Vikings.
Lars Brownworth
Yeah. Yeah.
Yeah. Yeah.
Lex Fridman
I mean, like speaking of pragmatic, they just integrate into everything. Now eventually, the Varangian Guard became less and less Viking over time.
I mean, like speaking of pragmatic, they just integrate into everything. Now eventually, the Varangian Guard became less and less Viking over time.
Lex Fridman
But this whole… You fast-forwarded the story, we should mention that Staraya Ladoga in 753 AD is when it was established, opening the connection to the two rivers, and they began trading on the rivers and establishing more stable states along the rivers, including the Kievan Rus in 862, 882, where the Varangians, so it’s the Swedish Vikings, they took Novgorod, they took Kiev, and they established the Kievan Rus there. And that is what led to the connection to the Byzantine Empire, where they started to… Again, the Vikings went from being Vikings. They go through this process of trading and then establishing a state, now they’re doing treaties of different kinds, and they’re also waging or trying to wage war.
But this whole… You fast-forwarded the story, we should mention that Staraya Ladoga in 753 AD is when it was established, opening the connection to the two rivers, and they began trading on the rivers and establishing more stable states along the rivers, including the Kievan Rus in 862, 882, where the Varangians, so it’s the Swedish Vikings, they took Novgorod, they took Kiev, and they established the Kievan Rus there. And that is what led to the connection to the Byzantine Empire, where they started to… Again, the Vikings went from being Vikings. They go through this process of trading and then establishing a state, now they’re doing treaties of different kinds, and they’re also waging or trying to wage war.
Lex Fridman
And going all the way to Constantinople, and having a deep admiration for Constantinople enough to then begin to dream of sacking Constantinople.
And going all the way to Constantinople, and having a deep admiration for Constantinople enough to then begin to dream of sacking Constantinople.
Lars Brownworth
Yeah. I mean, once they’re alerted to the wealth that’s there, you know, Vikings being Vikings, they show up.
Yeah. I mean, once they’re alerted to the wealth that’s there, you know, Vikings being Vikings, they show up.
Lex Fridman
Can you speak to the Greek Fire? So this was 941 and 944 when they tried, and then Greek Fire was this technology developed by the Romans.
Can you speak to the Greek Fire? So this was 941 and 944 when they tried, and then Greek Fire was this technology developed by the Romans.
Lars Brownworth
We don’t really know what it was, Greek Fire. It was a form of napalm, obviously. We have the ingredients what made it up, naphtha and oil and things like that. But it was this very flammable material that would ignite on contact. So the Byzantines would fill it into clay pots and then throw the clay pots. As soon as it’s exposed to oxygen, it would start burning. They also had siphons. They would carry, like, flamethrowers on their back and they would just spray it at enemies. And the- the real devious thing about it is that if you launch this clay pot at a ship and the material, you know, pooled across the wood and then dripped off into the water, being oil, it would float on top of the water and continue to burn.
We don’t really know what it was, Greek Fire. It was a form of napalm, obviously. We have the ingredients what made it up, naphtha and oil and things like that. But it was this very flammable material that would ignite on contact. So the Byzantines would fill it into clay pots and then throw the clay pots. As soon as it’s exposed to oxygen, it would start burning. They also had siphons. They would carry, like, flamethrowers on their back and they would just spray it at enemies. And the- the real devious thing about it is that if you launch this clay pot at a ship and the material, you know, pooled across the wood and then dripped off into the water, being oil, it would float on top of the water and continue to burn.
Lars Brownworth
So that if you were a sailor and you jumped off the ship ’cause it’s on fire and jumped into this oil patch that’s on fire, you’d be coated with it and you’d burn underneath the water. It was a horrible way to go. So this was a state secret, closely guarded secret. So closely guarded—it remains a mystery to this day of what exactly it was.
So that if you were a sailor and you jumped off the ship ’cause it’s on fire and jumped into this oil patch that’s on fire, you’d be coated with it and you’d burn underneath the water. It was a horrible way to go. So this was a state secret, closely guarded secret. So closely guarded—it remains a mystery to this day of what exactly it was.
Lex Fridman
Which is incredible, right?
Which is incredible, right?
Lars Brownworth
Yeah. But it, in the 944 attack on Constantinople, I mean the Vikings are coming on their ships. They brought these ships from Sweden. I mean that’s crazy. They’re in the Black Sea. They’ve sailed and they kind of swarm at the Byzantines. The Byzantines launch a bunch of decrepit old ships toward them that have Greek fire on them, and that turns the tide. But the Byzantine emperor so appreciates the strength of these horrifying Vikings that he forms a bodyguard of them.
Yeah. But it, in the 944 attack on Constantinople, I mean the Vikings are coming on their ships. They brought these ships from Sweden. I mean that’s crazy. They’re in the Black Sea. They’ve sailed and they kind of swarm at the Byzantines. The Byzantines launch a bunch of decrepit old ships toward them that have Greek fire on them, and that turns the tide. But the Byzantine emperor so appreciates the strength of these horrifying Vikings that he forms a bodyguard of them.
Lex Fridman
And hence we get just a few years later, again, tried to sack Constantinople and then join them.
And hence we get just a few years later, again, tried to sack Constantinople and then join them.
Lars Brownworth
Join ’em, yep.
Join ’em, yep.
Lex Fridman
The Varangian Guard in 988 with Basil II and Vladimir, they make the Varangian Guard into an institution, and then the word of mouth spreads that this is a real career path for the Viking, is to join the guard.
The Varangian Guard in 988 with Basil II and Vladimir, they make the Varangian Guard into an institution, and then the word of mouth spreads that this is a real career path for the Viking, is to join the guard.
Lars Brownworth
Yeah, that’s right. ‘Cause not only do you get paid very, you’re compensated very well obviously for defending the emperor, particularly if you do a good job, but you also have opportunities ’cause the emperor sends you, “Let’s go attack, you know, this tribe,” and you get to keep whatever you take. So there’s tremendous amounts of war profiteering you can accomplish. And the other great river system, the Volga, that brings you to the great enemy of the Byzantines, the Abbasid Caliphate. And they had a lot of trading links with the north. So you get things like fur and amber, lots of slaves from the Islamic world going up. You even have in a Swedish coin hoard, there’s a Buddha that’s been found. I mean… it’s Sweden.
Yeah, that’s right. ‘Cause not only do you get paid very, you’re compensated very well obviously for defending the emperor, particularly if you do a good job, but you also have opportunities ’cause the emperor sends you, “Let’s go attack, you know, this tribe,” and you get to keep whatever you take. So there’s tremendous amounts of war profiteering you can accomplish. And the other great river system, the Volga, that brings you to the great enemy of the Byzantines, the Abbasid Caliphate. And they had a lot of trading links with the north. So you get things like fur and amber, lots of slaves from the Islamic world going up. You even have in a Swedish coin hoard, there’s a Buddha that’s been found. I mean… it’s Sweden.
Lex Fridman
Yeah. So these networks of trade, just how incredible are they with geography, right? You can transform your understanding of land from the geography of the land to the geography of the river networks, because the way they raid and then invade and then conquer England is through the rivers. It’s an incredibly different way of seeing the world.
Yeah. So these networks of trade, just how incredible are they with geography, right? You can transform your understanding of land from the geography of the land to the geography of the river networks, because the way they raid and then invade and then conquer England is through the rivers. It’s an incredibly different way of seeing the world.
Lars Brownworth
Yeah. And if you look at the kingdoms the Vikings created, I’m thinking particularly of like Eric Blood-Axe in, you know, in York, he’s controlling parts of Ireland, parts of Scotland, Wales, England. Like there’s no… That doesn’t make sense unless you’re a Viking. You know, he’s… That also added tremendously to the terror that the Vikings brought, because I mean you should probably be a little careful with absolute statements here, but I can’t think of a major European city that’s not on a river. Which meant now with the Vikings, ’cause they could travel up, you know, rivers, the shallow rivers and then carry their boats whenever, everything was on the table now, even hundreds of miles inland is on the table.
Yeah. And if you look at the kingdoms the Vikings created, I’m thinking particularly of like Eric Blood-Axe in, you know, in York, he’s controlling parts of Ireland, parts of Scotland, Wales, England. Like there’s no… That doesn’t make sense unless you’re a Viking. You know, he’s… That also added tremendously to the terror that the Vikings brought, because I mean you should probably be a little careful with absolute statements here, but I can’t think of a major European city that’s not on a river. Which meant now with the Vikings, ’cause they could travel up, you know, rivers, the shallow rivers and then carry their boats whenever, everything was on the table now, even hundreds of miles inland is on the table.
Lex Fridman
And an incredible speed, much faster than the land armies. It’s terrifying.
And an incredible speed, much faster than the land armies. It’s terrifying.
Lars Brownworth
It’s terrifying.
It’s terrifying.
Lex Fridman
So you’re living in a constant state of fear.
So you’re living in a constant state of fear.
Lars Brownworth
Constant state of fear.
Constant state of fear.
Lex Fridman
We’ve talked about this transition in several different contexts, but you’ve written about this. It’s really interesting. Is the Vikings, like Ragnar, going from this mode of sea kings with no territory to the mode of land kings? So you have like somebody like Harald Bluetooth, 10th century Viking king of Denmark, you go from being these grand explorers that are free to being state builders. Was this always inevitable for all of these Vikings? Could we speak to the different transitions, maybe in England?
We’ve talked about this transition in several different contexts, but you’ve written about this. It’s really interesting. Is the Vikings, like Ragnar, going from this mode of sea kings with no territory to the mode of land kings? So you have like somebody like Harald Bluetooth, 10th century Viking king of Denmark, you go from being these grand explorers that are free to being state builders. Was this always inevitable for all of these Vikings? Could we speak to the different transitions, maybe in England?
Lars Brownworth
I think in one way, it’s inevitable. There’s so many examples of destroyers who just wreck civilizations. The builders are much more rare, you know? So I think it’s one of the reasons I think Augustus is a much more interesting person than Julius Caesar is. Augustus was a builder, and I like to see that. I like to see not just can you pull down, but can you build up? You know, just to take Ireland for example, Dublin, Limerick, almost every major city in Ireland was founded by the Vikings. So I don’t think it’s just a given that it would have happened. I think there’s something about the Vikings, and it’s probably tied to their pragmatism, their—like this pragmatic streak of, “We’re gonna use whatever. Oh, this system of king works.
I think in one way, it’s inevitable. There’s so many examples of destroyers who just wreck civilizations. The builders are much more rare, you know? So I think it’s one of the reasons I think Augustus is a much more interesting person than Julius Caesar is. Augustus was a builder, and I like to see that. I like to see not just can you pull down, but can you build up? You know, just to take Ireland for example, Dublin, Limerick, almost every major city in Ireland was founded by the Vikings. So I don’t think it’s just a given that it would have happened. I think there’s something about the Vikings, and it’s probably tied to their pragmatism, their—like this pragmatic streak of, “We’re gonna use whatever. Oh, this system of king works.
Lars Brownworth
This taxation system’s pretty good, let’s keep it.” You know, “Oh, this doesn’t work. Let’s ditch it.”
This taxation system’s pretty good, let’s keep it.” You know, “Oh, this doesn’t work. Let’s ditch it.”
Lex Fridman
Yeah, they went from destroyer to builder very naturally and very quickly.
Yeah, they went from destroyer to builder very naturally and very quickly.
Lars Brownworth
Yeah. There’s a natural process from conquering to building, but it does take talent and it does take a certain something.
Yeah. There’s a natural process from conquering to building, but it does take talent and it does take a certain something.
Lex Fridman
Can we talk about, so, one of the great Vikings, Canute the Great?
Can we talk about, so, one of the great Vikings, Canute the Great?
Lars Brownworth
I love Canute. I love Canute. I think he never, he doesn’t get his due. He’s one of those unsung heroes, I think, of the Viking world. He had a reputation. He was called the Emperor of the North. He had this massive, you know, England, Scotland, Wales, Ireland, Norway, Sweden, Denmark. I mean he’s just tying it all together. He was an extremely effective English king. I believe he introduced the penny, sadly discontinued, but I—
I love Canute. I love Canute. I think he never, he doesn’t get his due. He’s one of those unsung heroes, I think, of the Viking world. He had a reputation. He was called the Emperor of the North. He had this massive, you know, England, Scotland, Wales, Ireland, Norway, Sweden, Denmark. I mean he’s just tying it all together. He was an extremely effective English king. I believe he introduced the penny, sadly discontinued, but I—
Lex Fridman
Oh wait, really? Discontinued?
Oh wait, really? Discontinued?
Lars Brownworth
Discontinued. They’re no longer making…
Discontinued. They’re no longer making…
Lex Fridman
The penny is discontinued.
The penny is discontinued.
Lars Brownworth
2025’s the last, the last penny.
2025’s the last, the last penny.
Lex Fridman
Oh, no.
Oh, no.
Lars Brownworth
Everything’s gonna go up by five.
Everything’s gonna go up by five.
Lex Fridman
So going to Perplexity. Canute the Great was an early 11th century Danish ruler who became king of England, Denmark, and Norway… creating what historians call the North Sea Empire. He’s often regarded as one of the most effective kings in Anglo-Saxon English history for stabilizing the realm after decades of Viking warfare. Again, an example of a destroyer becoming a state builder.
So going to Perplexity. Canute the Great was an early 11th century Danish ruler who became king of England, Denmark, and Norway… creating what historians call the North Sea Empire. He’s often regarded as one of the most effective kings in Anglo-Saxon English history for stabilizing the realm after decades of Viking warfare. Again, an example of a destroyer becoming a state builder.
Lars Brownworth
Yeah. He was extremely strong. He was effective. You know, England went from being the whipping boy of the Vikings to controlling the Vikings.
Yeah. He was extremely strong. He was effective. You know, England went from being the whipping boy of the Vikings to controlling the Vikings.
Lex Fridman
And ended up on a pilgrimage to Rome.
And ended up on a pilgrimage to Rome.
Lars Brownworth
Went to Rome. Yeah.
Went to Rome. Yeah.
Lex Fridman
So he, although a Viking war leader, Canute ruled as a Christian king- … Patronizing churches and monasteries and going on pilgrimage to Rome in 1027 where he attended the Holy Roman Emperor’s coronation.
So he, although a Viking war leader, Canute ruled as a Christian king- … Patronizing churches and monasteries and going on pilgrimage to Rome in 1027 where he attended the Holy Roman Emperor’s coronation.
Lars Brownworth
Yes, he was recognized by his contemporaries as something special, right? You don’t get invited to those coronations if you’re a nobody. But the most famous story of Canute that I know, my favorite story, is, you know, being in positions of power, being famous, a lot of people sucking up to you, a lot of people telling you whatever they think you wanna hear. And so people are telling him all the time how wonderful he is, and he takes his whole court down to the seashore and orders his, his courtiers to carry him on his throne into the water, and then he commands the seas to stop, the waves to stop and to retreat.
Yes, he was recognized by his contemporaries as something special, right? You don’t get invited to those coronations if you’re a nobody. But the most famous story of Canute that I know, my favorite story, is, you know, being in positions of power, being famous, a lot of people sucking up to you, a lot of people telling you whatever they think you wanna hear. And so people are telling him all the time how wonderful he is, and he takes his whole court down to the seashore and orders his, his courtiers to carry him on his throne into the water, and then he commands the seas to stop, the waves to stop and to retreat.
Lars Brownworth
And they don’t, obviously, and everyone thinks he’s a little… But his point is that, “Y’all are saying how great I am. I have no control.” I mean, this is his active humility to kind of embarrass… “I have no control over anything. Stop telling me I’m the greatest thing since sliced bread.”
And they don’t, obviously, and everyone thinks he’s a little… But his point is that, “Y’all are saying how great I am. I have no control.” I mean, this is his active humility to kind of embarrass… “I have no control over anything. Stop telling me I’m the greatest thing since sliced bread.”
Lex Fridman
I like the leaders, and there’s a few of them in history that rise to the very top, and they’re still able to maintain humility. Marcus Aurelius in the Roman Empire is an example that, you know, reading Meditations is also just an insight into the mind of a man who’s to himself—’cause Meditations is not supposed to be work that’s published. It’s just a diary. To himself, he’s deeply humbled. And- … and one of the most powerful humans in history- … is still humble.
I like the leaders, and there’s a few of them in history that rise to the very top, and they’re still able to maintain humility. Marcus Aurelius in the Roman Empire is an example that, you know, reading Meditations is also just an insight into the mind of a man who’s to himself—’cause Meditations is not supposed to be work that’s published. It’s just a diary. To himself, he’s deeply humbled. And- … and one of the most powerful humans in history- … is still humble.
Lars Brownworth
The two most famous stoics, one was an emperor and one was a slave.
The two most famous stoics, one was an emperor and one was a slave.
Lex Fridman
So in the other part of the world, you’ve written a book and you did an—a legendary podcast series on The Byzantine Empire, the East Roman Empire, AKA, The Roman Empire. Well, let me actually just as a tangent of a tangent ask you about the podcast. So you’re—you created what is widely considered to be the first history podcast. This is before Dan Carlin, before all the amazing podcasts we all know and love. So the podcast series of course is The Twelve Byzantine Rulers: The History of the Byzantine Empire. What motivated you to explore this medium of podcasting? What… In—in the early, this must’ve been 2005, something like this.
So in the other part of the world, you’ve written a book and you did an—a legendary podcast series on The Byzantine Empire, the East Roman Empire, AKA, The Roman Empire. Well, let me actually just as a tangent of a tangent ask you about the podcast. So you’re—you created what is widely considered to be the first history podcast. This is before Dan Carlin, before all the amazing podcasts we all know and love. So the podcast series of course is The Twelve Byzantine Rulers: The History of the Byzantine Empire. What motivated you to explore this medium of podcasting? What… In—in the early, this must’ve been 2005, something like this.
Lars Brownworth
It was ’05, yeah.
It was ’05, yeah.
Lex Fridman
And people should go listen to it because it, it’s still, I mean, it’s like we’re talking about like ancient times or something, ’cause it is now a long time ago, but it’s still an incredibly good podcast. It’s a great podcast series.
And people should go listen to it because it, it’s still, I mean, it’s like we’re talking about like ancient times or something, ’cause it is now a long time ago, but it’s still an incredibly good podcast. It’s a great podcast series.
Lars Brownworth
Thank you. At the time, there’s a series that I would get at the library called The Great Courses. I don’t know if you’re familiar with-
Thank you. At the time, there’s a series that I would get at the library called The Great Courses. I don’t know if you’re familiar with-
Lex Fridman
Yes. Great Courses, yes.
Yes. Great Courses, yes.
Lars Brownworth
There was one particular professor. His name is Bob Brier, and he was—he’s an Egyptologist, lives on Long Island where I’m from, and he—I mean, it’s a massive thing. It’s like 24 hours- of lectures about the entire history of Egypt, and it was fascinating, ’cause he’s such a good storyteller. And I was reading… As a kid I could never figure out if I liked the medieval period better or the Roman period better. I was constantly going back and forth, and I stumbled across a book which referred to the medieval Roman Empire, and it was a bit like discovering your favorite TV show had 12 extra seasons you didn’t know about.
There was one particular professor. His name is Bob Brier, and he was—he’s an Egyptologist, lives on Long Island where I’m from, and he—I mean, it’s a massive thing. It’s like 24 hours- of lectures about the entire history of Egypt, and it was fascinating, ’cause he’s such a good storyteller. And I was reading… As a kid I could never figure out if I liked the medieval period better or the Roman period better. I was constantly going back and forth, and I stumbled across a book which referred to the medieval Roman Empire, and it was a bit like discovering your favorite TV show had 12 extra seasons you didn’t know about.
Lars Brownworth
And they were just as good. So I… It really was a labor of love. I would not shut up about The Byzantine Empire. So my older brother, we would go on walks together and I would be like, “And then Justinian, you know, da, da, da.” And he stopped me, he said, “I have no idea what you’re talking about. I have no idea, like, I need a framework. Give me a framework for this.” So I went home and I recorded myself giving a framework, which turned out to be episode one, but I think I did it in a British accent, a really bad British accent.
And they were just as good. So I… It really was a labor of love. I would not shut up about The Byzantine Empire. So my older brother, we would go on walks together and I would be like, “And then Justinian, you know, da, da, da.” And he stopped me, he said, “I have no idea what you’re talking about. I have no idea, like, I need a framework. Give me a framework for this.” So I went home and I recorded myself giving a framework, which turned out to be episode one, but I think I did it in a British accent, a really bad British accent.
Lars Brownworth
I was just messing around. And I gave him the… Luckily, I did it in my regular voice as well as this goofy accent, and I gave it to him and then I forgot about it. And that summer I was on a dig in Petra excavating the Temple of the Winged Lions, which was like a dream come true for me. And I get this email from my brother and he said, “Oh, I just submitted it as a podcast.” So he had to tell me what that was. But I was going for, to the extent that I had put thought into it, I was going for kind of a longer form lecture Great Courses series on the Byzantines. And then a bunch of people started emailing me saying, “When’s episode two coming out?” Oh, okay. So I guess there has to be an episode two. And then the thing kinda snowballed from there. I had no idea what I was doing.
I was just messing around. And I gave him the… Luckily, I did it in my regular voice as well as this goofy accent, and I gave it to him and then I forgot about it. And that summer I was on a dig in Petra excavating the Temple of the Winged Lions, which was like a dream come true for me. And I get this email from my brother and he said, “Oh, I just submitted it as a podcast.” So he had to tell me what that was. But I was going for, to the extent that I had put thought into it, I was going for kind of a longer form lecture Great Courses series on the Byzantines. And then a bunch of people started emailing me saying, “When’s episode two coming out?” Oh, okay. So I guess there has to be an episode two. And then the thing kinda snowballed from there. I had no idea what I was doing.
Lex Fridman
Your brother, by the way, is super tech savvy.
Your brother, by the way, is super tech savvy.
Lars Brownworth
He is. It wouldn’t have happened without Anders. So Anders, thank you.
He is. It wouldn’t have happened without Anders. So Anders, thank you.
Lex Fridman
But like looking back now, what do you think about that medium? Why do you think it connected so much to people? Because you’ve also written several amazing books. One of them is on The Byzantine Empire. Just looking back in a retrospective kind of way, because that from there blew up an entire industry of incredible other history podcasts and podcasts in general.
But like looking back now, what do you think about that medium? Why do you think it connected so much to people? Because you’ve also written several amazing books. One of them is on The Byzantine Empire. Just looking back in a retrospective kind of way, because that from there blew up an entire industry of incredible other history podcasts and podcasts in general.
Lars Brownworth
Yeah, I’ve been… That’s a great question. I’ve been trying to think for the past 20 years, like why it’s such a niche field, right? Why would people be interested in it? I think number one, it’s a great story. And people are people, and we haven’t changed much, which is one of the reasons why it’s accessible, because it’s very… These are people you could meet today. But I think podcasting in general, because there’s such a low bar to get in, or there was at the time. I mean, there’s nobody else, so just by virtue of being first, you know, it attracted attention. Whatever its merits, being first was the strongest one.
Yeah, I’ve been… That’s a great question. I’ve been trying to think for the past 20 years, like why it’s such a niche field, right? Why would people be interested in it? I think number one, it’s a great story. And people are people, and we haven’t changed much, which is one of the reasons why it’s accessible, because it’s very… These are people you could meet today. But I think podcasting in general, because there’s such a low bar to get in, or there was at the time. I mean, there’s nobody else, so just by virtue of being first, you know, it attracted attention. Whatever its merits, being first was the strongest one.
Lex Fridman
Which is to say you also did another series on the Normans, who no longer had the benefit of being first and were still nevertheless very good, so…
Which is to say you also did another series on the Normans, who no longer had the benefit of being first and were still nevertheless very good, so…
Lars Brownworth
Oh, I appreciate that. Thank you. It was… But I think podcasting in a way democratizes learning. You know, it unlocked the potential of all these armchair historians. I’m one of them who’s like, “Hey, this is really cool. I’m passionate about this.” You know, anything that allows you to tap into your passion, I think is gonna be great.
Oh, I appreciate that. Thank you. It was… But I think podcasting in a way democratizes learning. You know, it unlocked the potential of all these armchair historians. I’m one of them who’s like, “Hey, this is really cool. I’m passionate about this.” You know, anything that allows you to tap into your passion, I think is gonna be great.
Byzantine Empire
Lex Fridman
And the Byzantine Empire’s an interesting one. I don’t understand maybe… And you articulated this well, but it doesn’t get like the love that it maybe deserves in history. I think the framing of the book you wrote on the topic is the reason we have Western civilization as we know, or European-based Western civilization. In a sense because you have… They… Let’s see, maybe you can articulate the different ways they connected the thread, but one of them is they preserved the knowledge when the West was—when Europe was going through a dark period; they protected Europe in all those ways.
And the Byzantine Empire’s an interesting one. I don’t understand maybe… And you articulated this well, but it doesn’t get like the love that it maybe deserves in history. I think the framing of the book you wrote on the topic is the reason we have Western civilization as we know, or European-based Western civilization. In a sense because you have… They… Let’s see, maybe you can articulate the different ways they connected the thread, but one of them is they preserved the knowledge when the West was—when Europe was going through a dark period; they protected Europe in all those ways.
Lars Brownworth
And then eventually they jumpstart the Renaissance, ’cause people are… Constantinople’s gonna fall. It’s inevitable. It’s surrounded by hostile powers, and so they start migrating to Italy. Just at the moment Italy is receptive to its Greco-Roman past. Greek had died out in the West actually as early as the time of Justinian in the 500s, 560s. They needed… If you wanted to travel between the eastern and western parts of the empire, you needed guidebooks with helpful Latin or Greek phrases. So Latin had died out in the East and Greek had died out in the West by the 14th century, so you needed Byzantine teachers to be able to read Plato and Aristotle.
And then eventually they jumpstart the Renaissance, ’cause people are… Constantinople’s gonna fall. It’s inevitable. It’s surrounded by hostile powers, and so they start migrating to Italy. Just at the moment Italy is receptive to its Greco-Roman past. Greek had died out in the West actually as early as the time of Justinian in the 500s, 560s. They needed… If you wanted to travel between the eastern and western parts of the empire, you needed guidebooks with helpful Latin or Greek phrases. So Latin had died out in the East and Greek had died out in the West by the 14th century, so you needed Byzantine teachers to be able to read Plato and Aristotle.
Lex Fridman
The book also emphasizes as we’ve mentioned, a kind of great man view of history. So celebrating people like Constantine and Justinian. Or Justinian, who would be your number one top emperor in the history of the East Roman Empire, Byzantine Empire?
The book also emphasizes as we’ve mentioned, a kind of great man view of history. So celebrating people like Constantine and Justinian. Or Justinian, who would be your number one top emperor in the history of the East Roman Empire, Byzantine Empire?
Lars Brownworth
Hmm, that’s a good question. I mean, romantically, it’s gotta be Justinian. He dreams big. He dreams big. He doesn’t always get there, but he dreams big.
Hmm, that’s a good question. I mean, romantically, it’s gotta be Justinian. He dreams big. He dreams big. He doesn’t always get there, but he dreams big.
Lex Fridman
He dreamed and tried to reconquer the Western Roman Empire. I mean, he was a lot of wars of conquest- … and built the…
He dreamed and tried to reconquer the Western Roman Empire. I mean, he was a lot of wars of conquest- … and built the…
Lars Brownworth
Built the Hagia Sophia. I mean, I think this is… You know, we’re interested in the Egyptians because they built the pyramids. We’re not interested in the pyramids because they were built by the Egyptians, right? It’s like what is the great thing that your society has created? I think the Hagia Sophia is that for the Byzantine Empire. I mean, to go in it today is still the closest you can come to the fifth century. You know, and it peeled back the imperial splendor of what it must’ve been like. You know, you can still see it. You can smell it. You can feel it. Like, it’s there.
Built the Hagia Sophia. I mean, I think this is… You know, we’re interested in the Egyptians because they built the pyramids. We’re not interested in the pyramids because they were built by the Egyptians, right? It’s like what is the great thing that your society has created? I think the Hagia Sophia is that for the Byzantine Empire. I mean, to go in it today is still the closest you can come to the fifth century. You know, and it peeled back the imperial splendor of what it must’ve been like. You know, you can still see it. You can smell it. You can feel it. Like, it’s there.
Lex Fridman
There’s actually a really nice video on YouTube of you going from I think 50 to 60 years ago. I don’t know.
There’s actually a really nice video on YouTube of you going from I think 50 to 60 years ago. I don’t know.
Lars Brownworth
Seems like that. It does seem like that, yeah. We actually were kicked out.
Seems like that. It does seem like that, yeah. We actually were kicked out.
Lex Fridman
Oh, what’d you do?
Oh, what’d you do?
Lars Brownworth
My brother and I went.
My brother and I went.
Lex Fridman
What would you do?
What would you do?
Lars Brownworth
Well, you know, they—as you know, they’re very strict as to guides. They want to promote the local economy, so you have to have a local guide. You can’t go in there and look like you’re being a tour guide without a license. You have 15 different organizations. So we went there early, the hour it opened, and we had the entire cathedral to ourselves. And so we went around and my brother’s holding this camera and I’m, you know, goofily pointing things out. And one of the guards noticed us, and, you know, we had to remove ourselves from the building.
Well, you know, they—as you know, they’re very strict as to guides. They want to promote the local economy, so you have to have a local guide. You can’t go in there and look like you’re being a tour guide without a license. You have 15 different organizations. So we went there early, the hour it opened, and we had the entire cathedral to ourselves. And so we went around and my brother’s holding this camera and I’m, you know, goofily pointing things out. And one of the guards noticed us, and, you know, we had to remove ourselves from the building.
Lex Fridman
And so one of the things, I mean Justinian was a critical person in this too. He overhauled the Roman law. The legal system, the law… First of all, the Roman Empire in general, the East Roman Empire propagated it. They believed in the law. They held onto the law. And that’s many of the legal ideas we take for granted is grounded in everything developed in the Roman Empire and stabilizing the Roman Empire, so they carried that flag forward.
And so one of the things, I mean Justinian was a critical person in this too. He overhauled the Roman law. The legal system, the law… First of all, the Roman Empire in general, the East Roman Empire propagated it. They believed in the law. They held onto the law. And that’s many of the legal ideas we take for granted is grounded in everything developed in the Roman Empire and stabilizing the Roman Empire, so they carried that flag forward.
Lars Brownworth
Yeah. I mean, outside of Great Britain, all European legal systems are based on, ultimately based on the Code of Justinian, and then weirdly, because of the French connections, the State of Louisiana. Actually, if you want to be a lawyer, you have to—you have to pass a different bar in Louisiana than in everywhere else in the US.
Yeah. I mean, outside of Great Britain, all European legal systems are based on, ultimately based on the Code of Justinian, and then weirdly, because of the French connections, the State of Louisiana. Actually, if you want to be a lawyer, you have to—you have to pass a different bar in Louisiana than in everywhere else in the US.
Lex Fridman
Why do you think the—the Western Roman Empire and then the Eastern Roman Empire collapsed? Just looking at the grand picture of the history of the Roman Empire, it’s 2,200 years starting from the kingdom to the republic… and to the imperial period, to the East Roman Empire period. Why do societies rise and fall?
Why do you think the—the Western Roman Empire and then the Eastern Roman Empire collapsed? Just looking at the grand picture of the history of the Roman Empire, it’s 2,200 years starting from the kingdom to the republic… and to the imperial period, to the East Roman Empire period. Why do societies rise and fall?
Lars Brownworth
That’s a really interesting question, and there are probably as many answers as there are different kingdoms. But just the Roman Empire. My take on it is that the collapse really starts at the end of the reign of Basil II. So the year is 1025. Basil is the last monarch of the Macedonian dynasty, which had seen the empire become the most powerful state in the Mediterranean, much more powerful and advanced than its Muslim or Christian neighbors. He had expanded the empire essentially as large as it was going to be after Justinian. It was wealthy, it was glittering, it was educated. I mean, courtiers had to memorize the works of Plato by heart.
That’s a really interesting question, and there are probably as many answers as there are different kingdoms. But just the Roman Empire. My take on it is that the collapse really starts at the end of the reign of Basil II. So the year is 1025. Basil is the last monarch of the Macedonian dynasty, which had seen the empire become the most powerful state in the Mediterranean, much more powerful and advanced than its Muslim or Christian neighbors. He had expanded the empire essentially as large as it was going to be after Justinian. It was wealthy, it was glittering, it was educated. I mean, courtiers had to memorize the works of Plato by heart.
Lars Brownworth
The emperor, one of his favorite activities was to go, and he would begin a quote, and you would have to finish it, but you didn’t know where he would begin or what he was thinking that day. This is kind of what amused him, so they’re incredibly literate. I mean, inside Constantinople itself, the literacy rate was close to 100, which is…
The emperor, one of his favorite activities was to go, and he would begin a quote, and you would have to finish it, but you didn’t know where he would begin or what he was thinking that day. This is kind of what amused him, so they’re incredibly literate. I mean, inside Constantinople itself, the literacy rate was close to 100, which is…
Lars Brownworth
… crazy. But when he died, the court, which had been this magnificent court, this bureaucracy which had been running the empire, and which is vital to the workings of the empire, they convinced themselves that they could run the empire, they didn’t actually need the emperor. And so they specifically selected weak rulers, and then that led directly to the disastrous Battle of Manzikert in 1071, where the Turks enter the story and defeat, destroy the Roman army under Romanos Diogenes, who’s attempting to break free of the bureaucratic constraints. And then Anatolia gets flooded by these nomadic warriors, and the Byzantine gets pushed out on the…
… crazy. But when he died, the court, which had been this magnificent court, this bureaucracy which had been running the empire, and which is vital to the workings of the empire, they convinced themselves that they could run the empire, they didn’t actually need the emperor. And so they specifically selected weak rulers, and then that led directly to the disastrous Battle of Manzikert in 1071, where the Turks enter the story and defeat, destroy the Roman army under Romanos Diogenes, who’s attempting to break free of the bureaucratic constraints. And then Anatolia gets flooded by these nomadic warriors, and the Byzantine gets pushed out on the…
Lars Brownworth
So once they’ve lost the heartland, they’ve lost their source of troops, they’ve lost their source of taxation, they’ve lost their source of food. At this point it’s impossible to recover, and the Crusades are an attempt, the First Crusade anyway is an attempt by the Eastern Emperor Alexius to recover Asia Minor, more than Jerusalem. He wants to recover Asia Minor, and obviously it doesn’t work out. So I think at that point, it’s on a trajectory that can only end in collapse. And I think that’s, you can see that same kind of thing in the Viking world that we talked about, this stultifying, bureaucratic, this inflexibility.
So once they’ve lost the heartland, they’ve lost their source of troops, they’ve lost their source of taxation, they’ve lost their source of food. At this point it’s impossible to recover, and the Crusades are an attempt, the First Crusade anyway is an attempt by the Eastern Emperor Alexius to recover Asia Minor, more than Jerusalem. He wants to recover Asia Minor, and obviously it doesn’t work out. So I think at that point, it’s on a trajectory that can only end in collapse. And I think that’s, you can see that same kind of thing in the Viking world that we talked about, this stultifying, bureaucratic, this inflexibility.
Lex Fridman
Combined with the growing threats from all directions.
Combined with the growing threats from all directions.
Lars Brownworth
Growing threats in all directions. Maybe your own success is beginning to be a problem. And you can’t adapt as quickly, you’re not as lean and mean anymore. There’s too many traditions, too many, too much, the weight of history breaks you.
Growing threats in all directions. Maybe your own success is beginning to be a problem. And you can’t adapt as quickly, you’re not as lean and mean anymore. There’s too many traditions, too many, too much, the weight of history breaks you.
Lex Fridman
You sort of mentioned the Macedonian period, the dynasty where the East Roman Empire flourished once again, but like, they have gone through so many periods like that, and they lasted.
You sort of mentioned the Macedonian period, the dynasty where the East Roman Empire flourished once again, but like, they have gone through so many periods like that, and they lasted.
Lars Brownworth
That’s true. That is true.
That’s true. That is true.
Lex Fridman
I don’t know what the reason is, but you can really trace the Roman spirit, the Roman state, the core of whatever that is, through that 2,200-year period. There’s a real connection there, a thread that connects all of it, and so that, there’s lessons. That’s why we do need to study the Byzantium Empire for lessons of what makes societies last. Eventually everything collapses, but like that one lasted-
I don’t know what the reason is, but you can really trace the Roman spirit, the Roman state, the core of whatever that is, through that 2,200-year period. There’s a real connection there, a thread that connects all of it, and so that, there’s lessons. That’s why we do need to study the Byzantium Empire for lessons of what makes societies last. Eventually everything collapses, but like that one lasted-
Lars Brownworth
That one lasted a long time.
That one lasted a long time.
Lex Fridman
… longer. It’s easier to last when you’re hidden away somewhere, but they were in the middle of everything. Everybody wanted what they had.
… longer. It’s easier to last when you’re hidden away somewhere, but they were in the middle of everything. Everybody wanted what they had.
Lars Brownworth
Yeah, they were getting hit on all sides. There was, in their entire 2200-year history, there was not a single year they were at peace on all frontiers. Like, crazy.
Yeah, they were getting hit on all sides. There was, in their entire 2200-year history, there was not a single year they were at peace on all frontiers. Like, crazy.
Lex Fridman
And it wasn’t always because they were looking for trouble.
And it wasn’t always because they were looking for trouble.
Lars Brownworth
No.
No.
Lex Fridman
They’re, a lot of it is defensive.
They’re, a lot of it is defensive.
Lars Brownworth
Yep, including with those pesky Normans.
Yep, including with those pesky Normans.
History and human nature
Lex Fridman
Yeah, yeah. On the topic of great men in history, so where do you land on this great debate? How important are individual humans versus systems? So what do you think turns the tides of history? Can individual rulers or individual warriors or individual humans have the power to change the course of history?
Yeah, yeah. On the topic of great men in history, so where do you land on this great debate? How important are individual humans versus systems? So what do you think turns the tides of history? Can individual rulers or individual warriors or individual humans have the power to change the course of history?
Lars Brownworth
Yeah, that’s the question, isn’t it? I… The short answer is I subscribe to the great man or great woman theory. I think there’s moments—I can’t imagine the Protestant Reformation. I don’t think you can just swap out Martin Luther and have a Protestant Reformation. I don’t think you can swap out Augustus and have the Roman Empire. I mean, there are… I don’t think you can swap out Genghis—and so on and so forth. I think ultimately these impersonal forces are insufficient for explaining, because we are people. We are humans. We are, you know, everything is kind of a relational thing. And but at the same time, you know, the moment needs the man, but the man also needs the moment, you know?
Yeah, that’s the question, isn’t it? I… The short answer is I subscribe to the great man or great woman theory. I think there’s moments—I can’t imagine the Protestant Reformation. I don’t think you can just swap out Martin Luther and have a Protestant Reformation. I don’t think you can swap out Augustus and have the Roman Empire. I mean, there are… I don’t think you can swap out Genghis—and so on and so forth. I think ultimately these impersonal forces are insufficient for explaining, because we are people. We are humans. We are, you know, everything is kind of a relational thing. And but at the same time, you know, the moment needs the man, but the man also needs the moment, you know?
Lex Fridman
Some of it is timing, some of it is the environment- … the system around it. But yeah, I’ve just seen so many incredible humans that persevere through things that would break basically everybody. And they, the power of the belief they have. We were talking offline about Napoleon. Here’s a guy who was a student of all the great military generals of the past. Extremely competent in being able to micromanage every aspect of military affairs of a nation, but also extremely confident in his vision of the world and ability to conquer anyone. And you have the same thing with Genghis Khan. This boy that came from nothing. Everything was taken away. He united all of Mongolia, and then conquered most of the known world to them, including eventually China.
Some of it is timing, some of it is the environment- … the system around it. But yeah, I’ve just seen so many incredible humans that persevere through things that would break basically everybody. And they, the power of the belief they have. We were talking offline about Napoleon. Here’s a guy who was a student of all the great military generals of the past. Extremely competent in being able to micromanage every aspect of military affairs of a nation, but also extremely confident in his vision of the world and ability to conquer anyone. And you have the same thing with Genghis Khan. This boy that came from nothing. Everything was taken away. He united all of Mongolia, and then conquered most of the known world to them, including eventually China.
Lex Fridman
And it’s like, well, can you possibly have the great Mongol Empire without Genghis Khan?
And it’s like, well, can you possibly have the great Mongol Empire without Genghis Khan?
Lars Brownworth
No.
No.
Lex Fridman
And the same, and we as Americans ask ourselves that question about the founders. I mean, George Washington, not to romanticize it, but to give away power symbolically is a really powerful statement like we’ll mention with Augustus. There’s… When somebody’s given power, and in some sense absolute power, what they do with that power can reverberate through generations, and that’s in the hands of an individual.
And the same, and we as Americans ask ourselves that question about the founders. I mean, George Washington, not to romanticize it, but to give away power symbolically is a really powerful statement like we’ll mention with Augustus. There’s… When somebody’s given power, and in some sense absolute power, what they do with that power can reverberate through generations, and that’s in the hands of an individual.
Lars Brownworth
Yeah. Yeah, that’s exactly right. That’s well put. You know, Cincinnatus in Ancient Rome, same thing.
Yeah. Yeah, that’s exactly right. That’s well put. You know, Cincinnatus in Ancient Rome, same thing.
Lex Fridman
What lessons from… This is a big, ridiculous question, but-
What lessons from… This is a big, ridiculous question, but-
Lars Brownworth
Go at it.
Go at it.
Lex Fridman
… what lessons from all the things we’ve talked about, the exploration of the Vikings? What lessons do you learn from Vikings?
… what lessons from all the things we’ve talked about, the exploration of the Vikings? What lessons do you learn from Vikings?
Lars Brownworth
Lessons, lessons to learn from the Viking Age?
Lessons, lessons to learn from the Viking Age?
Lex Fridman
By the way, I should mention one thing. It’s a very practical lesson that we didn’t talk about that you taught me is the, the Vikings were, like, groomed themselves.
By the way, I should mention one thing. It’s a very practical lesson that we didn’t talk about that you taught me is the, the Vikings were, like, groomed themselves.
Lars Brownworth
Oh yeah.
Oh yeah.
Lex Fridman
They were like clean. This is so very surprising to me. That they like washed themselves and then both the men and the women- … really took care of themselves. You don’t often think about that.
They were like clean. This is so very surprising to me. That they like washed themselves and then both the men and the women- … really took care of themselves. You don’t often think about that.
Lars Brownworth
There was this whole… Like, the Vikings, everyone at this… Everyone has this very clear picture of what a Viking looked like- … and also has no idea what a Viking looked like somehow at the same time. Like, almost everything about them is wrong- … that we think of. You know, almost everything about them is wrong. They didn’t wear horned helmets. They, their hair probably was blonde disproportionately, but that was more because they used lye to dye it because it would kill the lice. And then they would take baths on a more regular basis than… I mean, this depended on where you were. So in England, for example, they were mocked as being soft, which always blows my mind. Like, really? You’re gonna mock the Vikings for being soft? Because they took too many baths.
There was this whole… Like, the Vikings, everyone at this… Everyone has this very clear picture of what a Viking looked like- … and also has no idea what a Viking looked like somehow at the same time. Like, almost everything about them is wrong- … that we think of. You know, almost everything about them is wrong. They didn’t wear horned helmets. They, their hair probably was blonde disproportionately, but that was more because they used lye to dye it because it would kill the lice. And then they would take baths on a more regular basis than… I mean, this depended on where you were. So in England, for example, they were mocked as being soft, which always blows my mind. Like, really? You’re gonna mock the Vikings for being soft? Because they took too many baths.
Lars Brownworth
But then in the Muslim East, one Muslim traveler writes that they were God’s filthiest creatures because of their habits of kind of disgusting shared bathing.
But then in the Muslim East, one Muslim traveler writes that they were God’s filthiest creatures because of their habits of kind of disgusting shared bathing.
Lex Fridman
Oh, that aspect of it.
Oh, that aspect of it.
Lars Brownworth
That, yeah.
That, yeah.
Lex Fridman
Yeah. So, it’s not that they didn’t bathe, they bathed a little too much and together.
Yeah. So, it’s not that they didn’t bathe, they bathed a little too much and together.
Lars Brownworth
They bathed, but they also like would brush their teeth- … using like recycled… Like, they would then spit into a cup and pass it to the next guy.
They bathed, but they also like would brush their teeth- … using like recycled… Like, they would then spit into a cup and pass it to the next guy.
Lex Fridman
Got it.
Got it.
Lars Brownworth
It was… It’s not awesome.
It was… It’s not awesome.
Lex Fridman
I read that… This is, this could be propaganda, but I read that in England there was worry that the Vikings were a bit too attractive to the women of England because of how much the Vikings took care of themselves in terms of grooming.
I read that… This is, this could be propaganda, but I read that in England there was worry that the Vikings were a bit too attractive to the women of England because of how much the Vikings took care of themselves in terms of grooming.
Lars Brownworth
Yeah. In the Danelaw, like you get invaded by these people- … they’re kicking your rear end militarily. Now they’re stealing your women just to insult you as well. You know?
Yeah. In the Danelaw, like you get invaded by these people- … they’re kicking your rear end militarily. Now they’re stealing your women just to insult you as well. You know?
Lex Fridman
Yeah. They’re… They wash themselves daily. They’ve got good teeth. What is this bullshit?
Yeah. They’re… They wash themselves daily. They’ve got good teeth. What is this bullshit?
Lars Brownworth
Whether they need it or not, I know. It really is…
Whether they need it or not, I know. It really is…
Lex Fridman
Why you guys can’t have everything? What are you doing? Anyway, so yeah, so one of the lessons I think we need to draw is shower daily.
Why you guys can’t have everything? What are you doing? Anyway, so yeah, so one of the lessons I think we need to draw is shower daily.
Lars Brownworth
Shower daily.
Shower daily.
Lex Fridman
Yeah, there you go.
Yeah, there you go.
Lars Brownworth
That’s right.
That’s right.
Lex Fridman
That’s the one thing I forgot.
That’s the one thing I forgot.
Lars Brownworth
That’s the lesson, that’s the takeaway, that’s the big profound takeaway.
That’s the lesson, that’s the takeaway, that’s the big profound takeaway.
Lex Fridman
Is there something bigger about the exploration about the leaps into the unknown?
Is there something bigger about the exploration about the leaps into the unknown?
Lars Brownworth
Yeah, I think a couple of years ago, there were all these debates about statues. Make sure we pull these statues down. This person did a bad thing, let’s pull these statues. You know, and I always thought they were kind of silly, too. I mean, I understand the point, but… Like, we don’t… When you have a statue of Christopher Columbus, for example, you’re not glorifying every single thing the man ever did and all the bad stuff that comes from this or that. You’re honoring something about him, like the spirit it takes to cross an ocean not knowing what’s on the other side, and that’s that spirit of exploration. I think with the Vikings, it’s the same. There’s this way you approach the world, this fearless, pragmatic approach.
Yeah, I think a couple of years ago, there were all these debates about statues. Make sure we pull these statues down. This person did a bad thing, let’s pull these statues. You know, and I always thought they were kind of silly, too. I mean, I understand the point, but… Like, we don’t… When you have a statue of Christopher Columbus, for example, you’re not glorifying every single thing the man ever did and all the bad stuff that comes from this or that. You’re honoring something about him, like the spirit it takes to cross an ocean not knowing what’s on the other side, and that’s that spirit of exploration. I think with the Vikings, it’s the same. There’s this way you approach the world, this fearless, pragmatic approach.
Lars Brownworth
I think as an American too, it’s the ultimate rags to riches. It’s the myth we tell ourself. You know, the man who starts with nothing and ends up as a sea king, well respected and sung about by poets. I mean, that’s it right there. You know, this is… And when you’re a society and you stop doing this, you run into trouble as well.
I think as an American too, it’s the ultimate rags to riches. It’s the myth we tell ourself. You know, the man who starts with nothing and ends up as a sea king, well respected and sung about by poets. I mean, that’s it right there. You know, this is… And when you’re a society and you stop doing this, you run into trouble as well.
Lex Fridman
What about the, the Byzantine Empire? What lessons do you draw from them?
What about the, the Byzantine Empire? What lessons do you draw from them?
Lars Brownworth
This is a much… That’s a much bigger one.
This is a much… That’s a much bigger one.
Lex Fridman
1000-year history.
1000-year history.
Lars Brownworth
1000-year history, and it’s also what I think is so cool about the Byzantines is that in the ways that they are like us and the way that they are unlike us. In some ways, they’re very analogous to the United States. The kind of the polyglot nature of their inhabitants, you know, the, their roots, the Greco-Roman, Judeo-Christian roots. And yet it was a place of incredible alien things as well. Men sitting on top of pillars. You know, a king, an incredibly hierarchical system which abhorred democracy. So I think it’s a way, it’s a way we… It’s a route we could have taken…. And it’s the way they handled things. Immigration, inflation, war, peace, diplomacy. I think there are, there are lessons there for us.
1000-year history, and it’s also what I think is so cool about the Byzantines is that in the ways that they are like us and the way that they are unlike us. In some ways, they’re very analogous to the United States. The kind of the polyglot nature of their inhabitants, you know, the, their roots, the Greco-Roman, Judeo-Christian roots. And yet it was a place of incredible alien things as well. Men sitting on top of pillars. You know, a king, an incredibly hierarchical system which abhorred democracy. So I think it’s a way, it’s a way we… It’s a route we could have taken…. And it’s the way they handled things. Immigration, inflation, war, peace, diplomacy. I think there are, there are lessons there for us.
Lex Fridman
Yeah. Yeah, I think from the Vikings, the lessons are a bit more poetic. The lessons from the Byzantine Empire is like quite literal, like how to run a government, how to run the law, how to-
Yeah. Yeah, I think from the Vikings, the lessons are a bit more poetic. The lessons from the Byzantine Empire is like quite literal, like how to run a government, how to run the law, how to-
Lars Brownworth
Yeah. How to build a stable society. And honestly, like you can count on the fingers of one hand, states that have lasted a thousand years, right? Byzantium and Venice, I think. And Venice was an offshoot of the Byzantines. Like that’s a gov— for a government to last a thousand years is a rare thing. Like, we should be taking a look at this. Like how? And how much of that is due to Augustus? Can we give him any credit for this? He built the system.
Yeah. How to build a stable society. And honestly, like you can count on the fingers of one hand, states that have lasted a thousand years, right? Byzantium and Venice, I think. And Venice was an offshoot of the Byzantines. Like that’s a gov— for a government to last a thousand years is a rare thing. Like, we should be taking a look at this. Like how? And how much of that is due to Augustus? Can we give him any credit for this? He built the system.
Lex Fridman
Yeah. But there was a lot, like you mentioned a lot of people along the way from Constantine to Justinian, the Basils. There’s so many emperors along the way that revolutionized and then restabilized the empire after it was almost falling apart.
Yeah. But there was a lot, like you mentioned a lot of people along the way from Constantine to Justinian, the Basils. There’s so many emperors along the way that revolutionized and then restabilized the empire after it was almost falling apart.
Lars Brownworth
Oh, yeah. You know what else too though? Like what happens to a human when you give that human essentially absolute power? ‘Cause the Byzantine emperor stood halfway there. I mean, he was more autocratic than anything other than, I don’t know, the pope that we have in the modern world. What happens when you give someone that level of power? Like, I love Justinian, but I wouldn’t have liked to know him. You know, I wouldn’t like to be one of his subjects. I love Basil I, but the man was a bloodthirsty tyrant. Like, I think it shows you what happened. What is it? Lord Acton: “Absolute power corrupts, and absolute power corrupts absolutely.” Like that’s quite clear throughout Byzantine histories, and it’s a long, a long list.
Oh, yeah. You know what else too though? Like what happens to a human when you give that human essentially absolute power? ‘Cause the Byzantine emperor stood halfway there. I mean, he was more autocratic than anything other than, I don’t know, the pope that we have in the modern world. What happens when you give someone that level of power? Like, I love Justinian, but I wouldn’t have liked to know him. You know, I wouldn’t like to be one of his subjects. I love Basil I, but the man was a bloodthirsty tyrant. Like, I think it shows you what happened. What is it? Lord Acton: “Absolute power corrupts, and absolute power corrupts absolutely.” Like that’s quite clear throughout Byzantine histories, and it’s a long, a long list.
Lex Fridman
And as technologies become more powerful, absolute power becomes potentially more destructive, so-
And as technologies become more powerful, absolute power becomes potentially more destructive, so-
Lars Brownworth
Yeah, it’s more absolute.
Yeah, it’s more absolute.
Lex Fridman
It’s more absolute.
It’s more absolute.
Lars Brownworth
You know?
You know?
Lex Fridman
And it’s— I mean, this is the project for the 21st century, the 20th and the 21st centuries post-Industrial Revolution, post the computer technological revolution, post nuclear weapons discovery. How do we construct societies that last like the Byzantine Empire did a thousand years? It’s just like a new challenge for us. There’s gonna be history books written about us ’cause like nuclear weapons, you know, 80 years ago, it’s like Greek fire that you can apply to the entirety of human civilization. And, and so- … that, there’s gonna be good history books, and I hope there’s gonna be these stories about the American empire, about the rest that sound similar to Byzantium Empire versus the Viking age. It only lasted-
And it’s— I mean, this is the project for the 21st century, the 20th and the 21st centuries post-Industrial Revolution, post the computer technological revolution, post nuclear weapons discovery. How do we construct societies that last like the Byzantine Empire did a thousand years? It’s just like a new challenge for us. There’s gonna be history books written about us ’cause like nuclear weapons, you know, 80 years ago, it’s like Greek fire that you can apply to the entirety of human civilization. And, and so- … that, there’s gonna be good history books, and I hope there’s gonna be these stories about the American empire, about the rest that sound similar to Byzantium Empire versus the Viking age. It only lasted-
Lars Brownworth
Yeah. Yeah.
Yeah. Yeah.
Lex Fridman
… three centuries.
… three centuries.
Lars Brownworth
I mean, I suppose the good news is it can be done, right? Or it has been done.
I mean, I suppose the good news is it can be done, right? Or it has been done.
Lex Fridman
It has been done. What gives you hope about the future, having looked at the deep history of us? What gives you hope?
It has been done. What gives you hope about the future, having looked at the deep history of us? What gives you hope?
Lars Brownworth
During grad school, I was reading Frederick Douglass’ autobiography, and he said, “I could sit with Plato and Cicero, and they would not flinch.” You know, by which he meant that the great conversation was for everyone, no matter what your skin color, no matter what your level of income, and even no matter your intelligence, you know? And I think that’s actually what, that’s why history comes alive for me, is because these are not alien people. You, you had asked how similar are ancient people to us psychologically.
During grad school, I was reading Frederick Douglass’ autobiography, and he said, “I could sit with Plato and Cicero, and they would not flinch.” You know, by which he meant that the great conversation was for everyone, no matter what your skin color, no matter what your level of income, and even no matter your intelligence, you know? And I think that’s actually what, that’s why history comes alive for me, is because these are not alien people. You, you had asked how similar are ancient people to us psychologically.
Lars Brownworth
You know, and what their goals were for li- And I think the short answer is they were identical to us, which is why we can understand them. It’s why you should read things. It’s why you should read the Meditations because this is not just some dry whatever talking to himself in a culture that you cannot understand and can never recreate. It’s a human talking about being human, you know? And I think human nature has not changed, and I don’t think human nature will change. So we are flawed and broken, and we’re… that’s, that’s the human condition. We’re gonna be flawed and broken. So I don’t think… I actually think that’s the great, that’s the great question of history. If you wanna understand history, you have to know about human nature. What is our human nature?
You know, and what their goals were for li- And I think the short answer is they were identical to us, which is why we can understand them. It’s why you should read things. It’s why you should read the Meditations because this is not just some dry whatever talking to himself in a culture that you cannot understand and can never recreate. It’s a human talking about being human, you know? And I think human nature has not changed, and I don’t think human nature will change. So we are flawed and broken, and we’re… that’s, that’s the human condition. We’re gonna be flawed and broken. So I don’t think… I actually think that’s the great, that’s the great question of history. If you wanna understand history, you have to know about human nature. What is our human nature?
Lars Brownworth
If you think it’s a blank slate and we can kind of educate ourselves to a utopia or, you know, like the Marxists said, then okay. Hasn’t really worked out, but okay. If you believe we’re basically bad, there’s a whole set of things that come with that. If you believe we’re basically good, there’s a whole set of… right? So, you won’t learn the appropriate lesson if you misdiagnose human nature.
If you think it’s a blank slate and we can kind of educate ourselves to a utopia or, you know, like the Marxists said, then okay. Hasn’t really worked out, but okay. If you believe we’re basically bad, there’s a whole set of things that come with that. If you believe we’re basically good, there’s a whole set of… right? So, you won’t learn the appropriate lesson if you misdiagnose human nature.
Lex Fridman
Yeah. I think the diagnosis that you’re kinda hinting at is, is seemingly the most accurate one, which is we’re flawed. A mix of good, a mix of evil, capacity for both.
Yeah. I think the diagnosis that you’re kinda hinting at is, is seemingly the most accurate one, which is we’re flawed. A mix of good, a mix of evil, capacity for both.
Lars Brownworth
That’s right. That’s right. I mean, I have to teach my kids to be kind. I don’t have to teach my kids to be unkind. I mean, one of those is natural, and one is not. I think my kids can become kind, you know?
That’s right. That’s right. I mean, I have to teach my kids to be kind. I don’t have to teach my kids to be unkind. I mean, one of those is natural, and one is not. I think my kids can become kind, you know?
Lex Fridman
The capacity.
The capacity.
Lars Brownworth
The capacity.
The capacity.
Lex Fridman
Is there.
Is there.
Lars Brownworth
Humans have the capacity for much greater things. But not perfection. It has to come outside of us.
Humans have the capacity for much greater things. But not perfection. It has to come outside of us.
Lex Fridman
Well, what is it? That line of, “All of us are in the gutter, but some of us are looking up at the stars.” And so you gotta teach as many of us to look up at the stars and dream. Because once you allow yourself to dream of a better world, you try, you try. Like the Vikings did, go out there. Don’t… try to not to murder your neighbor, but if you do, all of us have, of course.
Well, what is it? That line of, “All of us are in the gutter, but some of us are looking up at the stars.” And so you gotta teach as many of us to look up at the stars and dream. Because once you allow yourself to dream of a better world, you try, you try. Like the Vikings did, go out there. Don’t… try to not to murder your neighbor, but if you do, all of us have, of course.
Lars Brownworth
If you do, there’s Greenland. There’s Greenland.
If you do, there’s Greenland. There’s Greenland.
Lex Fridman
There’s Greenland. Thank you for everything you’ve done for the world. Thank you for the podcast you put out there. Thank you for your incredible books, and thank you for the conversation today.
There’s Greenland. Thank you for everything you’ve done for the world. Thank you for the podcast you put out there. Thank you for your incredible books, and thank you for the conversation today.
Lars Brownworth
Thank you. I really appreciate the opportunity. Thanks a lot.
Thank you. I really appreciate the opportunity. Thanks a lot.
Lex Fridman
Thanks for listening to this conversation with Lars Brownworth. To support this podcast, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from the Volsunga Saga, a 13th century Icelandic prose epic that tells the story of the Volsunga clan, a legendary Norse dynasty of heroes and dragon slayers. “Fear not death, for the hour of your doom is set, and none may escape it.” And another powerful quote from this saga is, “Better to fight and fall than to live without hope.” Thank you for listening, and hope to see you next time.
Thanks for listening to this conversation with Lars Brownworth. To support this podcast, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from the Volsunga Saga, a 13th century Icelandic prose epic that tells the story of the Volsunga clan, a legendary Norse dynasty of heroes and dragon slayers. “Fear not death, for the hour of your doom is set, and none may escape it.” And another powerful quote from this saga is, “Better to fight and fall than to live without hope.” Thank you for listening, and hope to see you next time.
Transcript for Jensen Huang: NVIDIA – The $4 Trillion Company & the AI Revolution | Lex Fridman Podcast #494
This is a transcript of Lex Fridman Podcast #494 with Jensen Huang.
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
- Go back to this episode’s main page
- Watch the full YouTube version of the podcast
Table of Contents
Here are the loose “chapters” in the conversation.
Click link to jump approximately to that part in the transcript:
Click link to jump approximately to that part in the transcript:
- 0:00 – Introduction
- 0:33 – Extreme co-design and rack-scale engineering
- 3:18 – How Jensen runs NVIDIA
- 22:40 – AI scaling laws
- 37:40 – Biggest blockers to AI scaling laws
- 39:23 – Supply chain
- 41:18 – Memory
- 47:24 – Power
- 52:43 – Elon and Colossus
- 56:11 – Jensen’s approach to engineering and leadership
- 1:01:37 – China
- 1:09:50 – TSMC and Taiwan
- 1:15:04 – NVIDIA’s moat
- 1:20:41 – AI data centers in space
- 1:24:30 – Will NVIDIA be worth $10 trillion?
- 1:34:39 – Leadership under pressure
- 1:48:25 – Video games
- 1:55:16 – AGI timeline
- 1:57:29 – Future of programming
- 2:11:01 – Consciousness
- 2:17:22 – Mortality
Introduction
Lex Fridman
The following is a conversation with Jensen Huang, CEO of NVIDIA, one of the most important and influential companies in the history of human civilization. NVIDIA is the engine powering the AI revolution, and a lot of its success can be directly attributed to Jensen’s sheer force of will and his many brilliant bets and decisions as a leader, engineer, and innovator. This is Lex Fridman Podcast. And now dear friends, here’s Jensen Huang.
The following is a conversation with Jensen Huang, CEO of NVIDIA, one of the most important and influential companies in the history of human civilization. NVIDIA is the engine powering the AI revolution, and a lot of its success can be directly attributed to Jensen’s sheer force of will and his many brilliant bets and decisions as a leader, engineer, and innovator. This is Lex Fridman Podcast. And now dear friends, here’s Jensen Huang.
Extreme co-design and rack-scale engineering
Lex Fridman
You’ve propelled NVIDIA into a new era in AI, moving beyond his focus on chip scale design to now rack scale design.
You’ve propelled NVIDIA into a new era in AI, moving beyond his focus on chip scale design to now rack scale design.
Lex Fridman
And I think it’s fair to say that winning for NVIDIA for a long time used to be about building the best GPU possible, and you still do, but now you’ve expanded that to extreme co-design of GPU, CPU memory, networking, storage, power cooling, software, the rack itself, the pod that you’ve announced, and even the data center. So let’s talk about extreme co-design. What is the hardest part of co-designing a system with that many complex components and design variables?
And I think it’s fair to say that winning for NVIDIA for a long time used to be about building the best GPU possible, and you still do, but now you’ve expanded that to extreme co-design of GPU, CPU memory, networking, storage, power cooling, software, the rack itself, the pod that you’ve announced, and even the data center. So let’s talk about extreme co-design. What is the hardest part of co-designing a system with that many complex components and design variables?
Jensen Huang
Yeah, thanks for that question. So first of all, the reason why extreme co-design is necessary is because the problem no longer fits inside one computer to be accelerated by one GPU. The problem that you’re trying to solve is you would like to go faster than the number of computers that you add. So you added 10,000 computers, but you would like it to go a million times faster. Then all of a sudden you have to take the algorithm, you have to break up the algorithm, you have to refactor it, you have to shard the pipeline, you have to shard the data, you have to shard the model. Now all of a sudden when you distribute the problem this way, not just scaling up the problem, but you’re distributing the problem, then everything gets in the way.
Yeah, thanks for that question. So first of all, the reason why extreme co-design is necessary is because the problem no longer fits inside one computer to be accelerated by one GPU. The problem that you’re trying to solve is you would like to go faster than the number of computers that you add. So you added 10,000 computers, but you would like it to go a million times faster. Then all of a sudden you have to take the algorithm, you have to break up the algorithm, you have to refactor it, you have to shard the pipeline, you have to shard the data, you have to shard the model. Now all of a sudden when you distribute the problem this way, not just scaling up the problem, but you’re distributing the problem, then everything gets in the way.
Jensen Huang
This is the Amdahl’s law problem where the amount of speed up you have for something depends on how much of the total workload it is. And so if computation represents 50% of the problem, and I sped up computation infinitely like a million times, you know, I only sped up the total workload by a factor of two. Now all of a sudden, not only do you have to distribute a computation, you have to shard the pipeline somehow. You also have to solve the networking problem because you’ve got all of these computers are all connected together. And so distributed computing at the scale that we do, the CPU is a problem, the GPU is a problem, the networking is a problem, the switching is a problem. And distributing the workload across all these computers is a problem.
This is the Amdahl’s law problem where the amount of speed up you have for something depends on how much of the total workload it is. And so if computation represents 50% of the problem, and I sped up computation infinitely like a million times, you know, I only sped up the total workload by a factor of two. Now all of a sudden, not only do you have to distribute a computation, you have to shard the pipeline somehow. You also have to solve the networking problem because you’ve got all of these computers are all connected together. And so distributed computing at the scale that we do, the CPU is a problem, the GPU is a problem, the networking is a problem, the switching is a problem. And distributing the workload across all these computers is a problem.
Jensen Huang
It’s just a massively complex computer science problem. And so we just gotta bring every technology to bear. Otherwise, we scale up linearly or we scale up based on the capabilities of Moore’s Law, which has largely slowed because Dennard scaling has slowed.
It’s just a massively complex computer science problem. And so we just gotta bring every technology to bear. Otherwise, we scale up linearly or we scale up based on the capabilities of Moore’s Law, which has largely slowed because Dennard scaling has slowed.
How Jensen runs NVIDIA
Lex Fridman
I’m sure there’s trade-offs there. Plus you have a complete disparate disciplines here. I’m sure you have specialists in each one of these high bandwidth memory, the network and the NVLink, the NICs, the optics and the copper that you’re doing, the power delivery, the cooling, all of that. I mean, there’s like world experts in each of those. How do you get ’em in a room together to figure out-
I’m sure there’s trade-offs there. Plus you have a complete disparate disciplines here. I’m sure you have specialists in each one of these high bandwidth memory, the network and the NVLink, the NICs, the optics and the copper that you’re doing, the power delivery, the cooling, all of that. I mean, there’s like world experts in each of those. How do you get ’em in a room together to figure out-
Jensen Huang
That’s why my staff is so large. Yeah.
That’s why my staff is so large. Yeah.
Lex Fridman
What’s the pro- can you take me through the process of the specialists and the generalists? Like how do you put together the rack when you know the s- the set of things you have to shove into a rack together? Like what does that process look like of designing it all together?
What’s the pro- can you take me through the process of the specialists and the generalists? Like how do you put together the rack when you know the s- the set of things you have to shove into a rack together? Like what does that process look like of designing it all together?
Jensen Huang
Yeah. There’s the first question, which is: what is extreme co-design? We’re optimizing across the entire stack of software from architectures to chips, to systems, to system software, to the algorithms, to the applications. That’s one layer. The second thing that you and I just talked about goes beyond CPUs and GPUs and networking chips and scale up switches and scale out switches. And then of course, you gotta include power and cooling and all of that because all these computers are extremely power hungry. They do a lot of work and they’re very energy efficient, but they in aggregate still consume a lot of power. And so that’s one. The first question is, what is it?
Yeah. There’s the first question, which is: what is extreme co-design? We’re optimizing across the entire stack of software from architectures to chips, to systems, to system software, to the algorithms, to the applications. That’s one layer. The second thing that you and I just talked about goes beyond CPUs and GPUs and networking chips and scale up switches and scale out switches. And then of course, you gotta include power and cooling and all of that because all these computers are extremely power hungry. They do a lot of work and they’re very energy efficient, but they in aggregate still consume a lot of power. And so that’s one. The first question is, what is it?
Jensen Huang
The second question is, why is it, and we just spoke about the reason, you know you want to distribute the workload so that you can exceed the benefit of just increasing the number of computers. And the, and then the third question is, how is it, how do you do it?
The second question is, why is it, and we just spoke about the reason, you know you want to distribute the workload so that you can exceed the benefit of just increasing the number of computers. And the, and then the third question is, how is it, how do you do it?
Jensen Huang
And, and that’s the, that’s kind of the miracle of this company. You know, when you’re designing a computer, you have to have an operating system of computers. When you’re designing a company, you should first think about what is it that you want the company to produce. You know, I see a lot of companies’ organization charts, and they all look the same. Hamburger organization charts, soft organization charts, and car company organization charts. They all look the same. And it doesn’t make any sense to me. You know, the goal of a comp- of a company is to be the machinery, the mechanism, the system that produces the output. And that output is the product that we like to create. It is also designed, the architecture of the company should reflect the environment by which it exists.
And, and that’s the, that’s kind of the miracle of this company. You know, when you’re designing a computer, you have to have an operating system of computers. When you’re designing a company, you should first think about what is it that you want the company to produce. You know, I see a lot of companies’ organization charts, and they all look the same. Hamburger organization charts, soft organization charts, and car company organization charts. They all look the same. And it doesn’t make any sense to me. You know, the goal of a comp- of a company is to be the machinery, the mechanism, the system that produces the output. And that output is the product that we like to create. It is also designed, the architecture of the company should reflect the environment by which it exists.
Jensen Huang
It almost directly says what you should do with the organization. My direct staff is 60 people. You know, I don’t have one-on-ones with ’em because it’s impossible. You can’t have 60 people on your staff if you’re, you know, gonna get work done and-
It almost directly says what you should do with the organization. My direct staff is 60 people. You know, I don’t have one-on-ones with ’em because it’s impossible. You can’t have 60 people on your staff if you’re, you know, gonna get work done and-
Lex Fridman
So you still have 60 reports. You still have across-
So you still have 60 reports. You still have across-
Jensen Huang
More, yeah.
More, yeah.
Lex Fridman
More. And most stars at least have a foot in engineering.
More. And most stars at least have a foot in engineering.
Jensen Huang
Almost all of them. There’s experts in memory, there’s experts in CPUs, there’s experts in optical. All-
Almost all of them. There’s experts in memory, there’s experts in CPUs, there’s experts in optical. All-
Lex Fridman
That’s incredible.
That’s incredible.
Jensen Huang
Yeah, GPUs and- Architecture, algorithms, design-
Yeah, GPUs and- Architecture, algorithms, design-
Lex Fridman
So, you constantly have an eye on the entire stack, and you’re having to have, like, intense discussions about the design of the entire stack?
So, you constantly have an eye on the entire stack, and you’re having to have, like, intense discussions about the design of the entire stack?
Jensen Huang
And no conversation is ever one person. That’s why I don’t do one-on-ones. We present a problem and all of us attack it. You know, because we’re doing extreme co-design. And literally, the company is doing extreme co-design all the time.
And no conversation is ever one person. That’s why I don’t do one-on-ones. We present a problem and all of us attack it. You know, because we’re doing extreme co-design. And literally, the company is doing extreme co-design all the time.
Lex Fridman
So, even if you’re talking about a particular component, like cooling, networking, everybody’s listening in?
So, even if you’re talking about a particular component, like cooling, networking, everybody’s listening in?
Jensen Huang
Yeah, exactly.
Yeah, exactly.
Lex Fridman
And they can contribute, “Well, this doesn’t work for the power distribution. This doesn’t-“
And they can contribute, “Well, this doesn’t work for the power distribution. This doesn’t-“
Jensen Huang
Exactly.
Exactly.
Lex Fridman
“… This doesn’t work for the memory. This doesn’t work for this.”
“… This doesn’t work for the memory. This doesn’t work for this.”
Jensen Huang
Exactly. And whoever wants to tune out, tune out. You know what I’m saying? And the reason for that is because the people who are on the staff, they know when to pay attention. There’s supposed… You know, it’s something they could have contributed to, they didn’t contribute to, “I’m going to call them out.” You know? And so, “Hey, come on, let’s get in here.”
Exactly. And whoever wants to tune out, tune out. You know what I’m saying? And the reason for that is because the people who are on the staff, they know when to pay attention. There’s supposed… You know, it’s something they could have contributed to, they didn’t contribute to, “I’m going to call them out.” You know? And so, “Hey, come on, let’s get in here.”
Lex Fridman
So, as you mentioned, NVIDIA is this company that’s adapting to the environment. So, which point can you say, did the environment change and began adapting sort of secretly- … in the early days from GPU for gaming, maybe the early deep learning revolution to we’re now going to start thinking of it as an AI factory? What does NVIDIA do? It produces AI; let’s build a factory that makes AI.
So, as you mentioned, NVIDIA is this company that’s adapting to the environment. So, which point can you say, did the environment change and began adapting sort of secretly- … in the early days from GPU for gaming, maybe the early deep learning revolution to we’re now going to start thinking of it as an AI factory? What does NVIDIA do? It produces AI; let’s build a factory that makes AI.
Jensen Huang
I could reason through that systematically. We started out as an accelerator company. But the problem with accelerators is that the application domain’s too narrow. It has the benefit of being incredibly optimized for the job. You know, any specialist has that benefit. The problem with intense specialization is that, of course, your market reach is narrower, but that’s even fine. The problem is, the market size also dictates your R&D capacity. And your R&D capacity ultimately dictates the influence and impact that you can possibly have in computing. And so, when we first started out as an accelerator, very specific accelerator, we always knew that was going to be our first step.
I could reason through that systematically. We started out as an accelerator company. But the problem with accelerators is that the application domain’s too narrow. It has the benefit of being incredibly optimized for the job. You know, any specialist has that benefit. The problem with intense specialization is that, of course, your market reach is narrower, but that’s even fine. The problem is, the market size also dictates your R&D capacity. And your R&D capacity ultimately dictates the influence and impact that you can possibly have in computing. And so, when we first started out as an accelerator, very specific accelerator, we always knew that was going to be our first step.
Jensen Huang
We had to find a way to become accelerated computing. But the problem is, when you become a computing company, it’s too general purpose and it takes away from your specialization. The tur- I connected two words that actually have fundamental tension. The better computing company we become, the worse we became as a specialist. The more of a specialist, the less capacity we have to do overall computing. And so, that… And I connected those two words together on purpose, that the company has to find that really narrow path, step by step by step, to expand our aperture of computing, but not give up on the most important specialization that we had. Okay, so the first step that we took beyond acceleration was we invented a programmable pixel shader.
We had to find a way to become accelerated computing. But the problem is, when you become a computing company, it’s too general purpose and it takes away from your specialization. The tur- I connected two words that actually have fundamental tension. The better computing company we become, the worse we became as a specialist. The more of a specialist, the less capacity we have to do overall computing. And so, that… And I connected those two words together on purpose, that the company has to find that really narrow path, step by step by step, to expand our aperture of computing, but not give up on the most important specialization that we had. Okay, so the first step that we took beyond acceleration was we invented a programmable pixel shader.
Jensen Huang
So, that was the first step towards programmability. It was our first journey towards moving into the world of computing. The second thing that we did was we created, we put FP32 into our shaders. That FP32 step, IEEE-compatible FP32, was a huge step in the direction of computing. It was the reason why all of the people who were working on stream processors and, you know, other types of data flow processors discovered us. And they said, “Hey, all of a sudden, you know, we might be able to use this GPU that’s incredibly computationally intensive, and it’s now, you know, compliant with IEEE.”
So, that was the first step towards programmability. It was our first journey towards moving into the world of computing. The second thing that we did was we created, we put FP32 into our shaders. That FP32 step, IEEE-compatible FP32, was a huge step in the direction of computing. It was the reason why all of the people who were working on stream processors and, you know, other types of data flow processors discovered us. And they said, “Hey, all of a sudden, you know, we might be able to use this GPU that’s incredibly computationally intensive, and it’s now, you know, compliant with IEEE.”
Jensen Huang
I can take my software that I was writing, you know, previously on CPUs, and I can see about using the GPU for that. And which led us to create, put C on top of FP32, what’s called, we call Cg. The Cg path took us to eventually CUDA. CUDA, step by step by step we… Well, the putting CUDA on GeForce, that was a strategic decision that was very, very hard to do, because it cost the company enormous amounts of our profits, and we couldn’t afford it at the time. But we did it anyway because we wanted to be a computing company. A computing company has a computing architecture. A computing architecture has to be compatible across all of the chips that we build.
I can take my software that I was writing, you know, previously on CPUs, and I can see about using the GPU for that. And which led us to create, put C on top of FP32, what’s called, we call Cg. The Cg path took us to eventually CUDA. CUDA, step by step by step we… Well, the putting CUDA on GeForce, that was a strategic decision that was very, very hard to do, because it cost the company enormous amounts of our profits, and we couldn’t afford it at the time. But we did it anyway because we wanted to be a computing company. A computing company has a computing architecture. A computing architecture has to be compatible across all of the chips that we build.
Lex Fridman
Can you take me through that decision? So, putting CUDA on GeForce, could not afford to do? Can you explain that decision? Why boldly choose to do that anyway? Can you explain that decision?
Can you take me through that decision? So, putting CUDA on GeForce, could not afford to do? Can you explain that decision? Why boldly choose to do that anyway? Can you explain that decision?
Jensen Huang
Yeah, excellent. That was… I would say that that was the first strategic decision that is as close to an existential threat.
Yeah, excellent. That was… I would say that that was the first strategic decision that is as close to an existential threat.
Lex Fridman
For people who don’t know, it turned out to be, spoiler alert, one of the most incredibly brilliant decisions ever made by a company. So, CUDA turned out to be an incredible foundation for computation in this AI infrastructure world. So, so- … just setting the context. It turned out to be a good decision.
For people who don’t know, it turned out to be, spoiler alert, one of the most incredibly brilliant decisions ever made by a company. So, CUDA turned out to be an incredible foundation for computation in this AI infrastructure world. So, so- … just setting the context. It turned out to be a good decision.
Jensen Huang
Yeah, it turned out to have been a good decision. I think the… So, here’s the way it went. So, we invented this thing called CUDA, and it expanded the aperture of applications that we can accelerate with our accelerator. The question is, how do we attract developers to CUDA? Because a computing platform is all about developers. And developers don’t come to a computing platform just because, you know, it could perform something interesting. They come to a computing platform because the install base is large. Because a developer, like anybody else, wants to develop software that reaches a lot of people. So, the install base is, in fact, the single most important part of an architecture. The architecture could attract enormous amounts of criticism.
Yeah, it turned out to have been a good decision. I think the… So, here’s the way it went. So, we invented this thing called CUDA, and it expanded the aperture of applications that we can accelerate with our accelerator. The question is, how do we attract developers to CUDA? Because a computing platform is all about developers. And developers don’t come to a computing platform just because, you know, it could perform something interesting. They come to a computing platform because the install base is large. Because a developer, like anybody else, wants to develop software that reaches a lot of people. So, the install base is, in fact, the single most important part of an architecture. The architecture could attract enormous amounts of criticism.
Jensen Huang
For example, no architecture has ever attracted more criticism than the x86… you know, as a less than elegant architecture, but yet it is the defining architecture of today. It gives you an example that in fact so many RISC architectures which were beautifully architected, incredibly well-designed by some of the brightest computer scientists in the world, largely failed. And so I’ve given you two examples where one is, you know, one is elegant, the other one’s barely aesthetic, and so yet x86 survived and the reason for-
For example, no architecture has ever attracted more criticism than the x86… you know, as a less than elegant architecture, but yet it is the defining architecture of today. It gives you an example that in fact so many RISC architectures which were beautifully architected, incredibly well-designed by some of the brightest computer scientists in the world, largely failed. And so I’ve given you two examples where one is, you know, one is elegant, the other one’s barely aesthetic, and so yet x86 survived and the reason for-
Lex Fridman
Install base is everything.
Install base is everything.
Jensen Huang
Install base defines an architecture. Not… Everything else is secondary, okay? And so there were other architectures at the time. CUDA came out, OpenCL was here. There were… You know, there’s several other competing architectures. But the thing that… The decision that we made that was good was we said, “Hey, look, ultimately it’s about install base and what is the best way we could get a new computing architecture into the world?” By that timeframe, GeForce had become successful.
Install base defines an architecture. Not… Everything else is secondary, okay? And so there were other architectures at the time. CUDA came out, OpenCL was here. There were… You know, there’s several other competing architectures. But the thing that… The decision that we made that was good was we said, “Hey, look, ultimately it’s about install base and what is the best way we could get a new computing architecture into the world?” By that timeframe, GeForce had become successful.
Jensen Huang
We were already selling millions and millions of GeForce GPUs a year, and we said, “You know, we, we ought to put CUDA on GeForce and put it into every single PC whether customers use it or not, and use it as a starting point of cultivating our install base.” Meanwhile, we’ll go and attract developers, and we went to universities and wrote books and taught classes and put CUDA everywhere. And eventually people discover… And at the time, the PC was the primary computing vehicle. There was no cloud, and we could put a supercomputer in the hands of every researcher in school, every scientist, you know, every engineering school, every… or every student in school, and eventually something amazing will happen.
We were already selling millions and millions of GeForce GPUs a year, and we said, “You know, we, we ought to put CUDA on GeForce and put it into every single PC whether customers use it or not, and use it as a starting point of cultivating our install base.” Meanwhile, we’ll go and attract developers, and we went to universities and wrote books and taught classes and put CUDA everywhere. And eventually people discover… And at the time, the PC was the primary computing vehicle. There was no cloud, and we could put a supercomputer in the hands of every researcher in school, every scientist, you know, every engineering school, every… or every student in school, and eventually something amazing will happen.
Jensen Huang
Well, the problem was CUDA increased our cost of that GPU, which is a consumer product, so tremendously, it completely consumed all of the company’s gross profit dollars. And so at the time, the company was probably, you know, worth, I don’t know, at the time, eight… Was it like $8 billion or something? Like six, $7 billion or something like that. After we launched CUDA, I recognized that it was going to add so much cost, but it was something we believed in. You know, our market cap went down to like one and a half billion dollars. And so we were down there for a while and we clawed our way back slowly, but we carried CUDA on GeForce. I always say that NVIDIA is the house that GeForce built, because it was GeForce that took CUDA out to everybody.
Well, the problem was CUDA increased our cost of that GPU, which is a consumer product, so tremendously, it completely consumed all of the company’s gross profit dollars. And so at the time, the company was probably, you know, worth, I don’t know, at the time, eight… Was it like $8 billion or something? Like six, $7 billion or something like that. After we launched CUDA, I recognized that it was going to add so much cost, but it was something we believed in. You know, our market cap went down to like one and a half billion dollars. And so we were down there for a while and we clawed our way back slowly, but we carried CUDA on GeForce. I always say that NVIDIA is the house that GeForce built, because it was GeForce that took CUDA out to everybody.
Jensen Huang
Researchers, scientists, they discovered CUDA on GeForce because they were all, you know… Many of ’em were gamers. Many of them built their own PCs anyways. In a university lab, many of them built clusters themselves, you know, using PC components. And, and so that, you know, that’s kind of how we got going.
Researchers, scientists, they discovered CUDA on GeForce because they were all, you know… Many of ’em were gamers. Many of them built their own PCs anyways. In a university lab, many of them built clusters themselves, you know, using PC components. And, and so that, you know, that’s kind of how we got going.
Lex Fridman
And then that became the platform and the foundation for the deep learning revolution.
And then that became the platform and the foundation for the deep learning revolution.
Jensen Huang
That was also another great, great observation. Yeah.
That was also another great, great observation. Yeah.
Lex Fridman
That existential moment, do you remember… Like, what were those meetings like? What were those discussions like, deciding as a company, risking everything?
That existential moment, do you remember… Like, what were those meetings like? What were those discussions like, deciding as a company, risking everything?
Jensen Huang
Well I had to make it clear to the board what we’re trying to do, and the management team knew our gross margins were gonna get crushed. So you could imagine a world where GeForce would carry the burden of CUDA and none of the gamers would appreciate it and none of the gamers would pay for it. You know, they only pay certain price and it doesn’t matter what your cost is. And so the… You know, we increased our cost by 50% and that consumed… And we were a 35% gross margin company, and so it was a… It was quite a difficult decision to make. But you could imagine that someday this would go into workstations and it would go into supercomputers and in those segments, maybe we can capture more margin.
Well I had to make it clear to the board what we’re trying to do, and the management team knew our gross margins were gonna get crushed. So you could imagine a world where GeForce would carry the burden of CUDA and none of the gamers would appreciate it and none of the gamers would pay for it. You know, they only pay certain price and it doesn’t matter what your cost is. And so the… You know, we increased our cost by 50% and that consumed… And we were a 35% gross margin company, and so it was a… It was quite a difficult decision to make. But you could imagine that someday this would go into workstations and it would go into supercomputers and in those segments, maybe we can capture more margin.
Jensen Huang
So you could reason your way into being able to afford this, but it still took… It took a decade.
So you could reason your way into being able to afford this, but it still took… It took a decade.
Lex Fridman
But that, but that’s more of, like, conversation with the board convincing them, but you psychologically- … as NVIDIA’s continued to make bold bets that predict the future, and in part, especially now, define the future. So I’m almost looking for wisdom about how you’re able to make those decisions, to make leaps- … like that as a company.
But that, but that’s more of, like, conversation with the board convincing them, but you psychologically- … as NVIDIA’s continued to make bold bets that predict the future, and in part, especially now, define the future. So I’m almost looking for wisdom about how you’re able to make those decisions, to make leaps- … like that as a company.
Jensen Huang
Well, first of all, I’m informed by a lot of curiosity. At some point, there’s a reasoning system that convinces me so clearly this outcome will happen. That this will happen. And so I believe it in my mind, and when I believe it in my mind, you know how it is. You manifest a future and that future is so convincing, there’s no way it won’t happen. There’s a lot of suffering in between, but you’ve gotta believe what you believe.
Well, first of all, I’m informed by a lot of curiosity. At some point, there’s a reasoning system that convinces me so clearly this outcome will happen. That this will happen. And so I believe it in my mind, and when I believe it in my mind, you know how it is. You manifest a future and that future is so convincing, there’s no way it won’t happen. There’s a lot of suffering in between, but you’ve gotta believe what you believe.
Lex Fridman
So you, you, you envision the future- … and you essentially, from a sort of engineering perspective, manifest it?
So you, you, you envision the future- … and you essentially, from a sort of engineering perspective, manifest it?
Jensen Huang
Yeah. And you reason about how to get there. You reason about why it must exist. And you know, I reason… We all reason it here. The management team would reason about it. All the people that I… We spend a lot of time reasoning about it. The thing that… The next part of it is probably a skill thing, which is, you know, oftentimes in leadership the leadership stays quiet or they learn about something, and then they do some manifesto, and it’s a brand-new year, and somehow at the end of the year, next year, we’re gonna have a brand-new plan. Big huge layoff this way, big huge organization change this way, new mission statement… brand new logos, you know, that kind of stuff.
Yeah. And you reason about how to get there. You reason about why it must exist. And you know, I reason… We all reason it here. The management team would reason about it. All the people that I… We spend a lot of time reasoning about it. The thing that… The next part of it is probably a skill thing, which is, you know, oftentimes in leadership the leadership stays quiet or they learn about something, and then they do some manifesto, and it’s a brand-new year, and somehow at the end of the year, next year, we’re gonna have a brand-new plan. Big huge layoff this way, big huge organization change this way, new mission statement… brand new logos, you know, that kind of stuff.
Jensen Huang
We’ve just never, I never do things that way. When I learn about something and it’s starting to influence how I think, I’ll make it very clear to everybody near me that, you know, this is interesting. This is going to make a difference. This is going to impact that. And I reason about things step by step by step. Oftentimes, I’ve already made up my mind, but I’ll take every possible opportunity—external information, new insights, new discoveries, new engineering revelations, new milestones developed—I’ll take those opportunities and I’ll use it to shape everybody else’s belief system. And I’m doing that literally every single day. I’m doing that with my board, I’m doing that with my management team, I’m doing that with my employees.
We’ve just never, I never do things that way. When I learn about something and it’s starting to influence how I think, I’ll make it very clear to everybody near me that, you know, this is interesting. This is going to make a difference. This is going to impact that. And I reason about things step by step by step. Oftentimes, I’ve already made up my mind, but I’ll take every possible opportunity—external information, new insights, new discoveries, new engineering revelations, new milestones developed—I’ll take those opportunities and I’ll use it to shape everybody else’s belief system. And I’m doing that literally every single day. I’m doing that with my board, I’m doing that with my management team, I’m doing that with my employees.
Jensen Huang
I’m trying to shape their belief systems such that when I come the day I say, “Hey, let’s buy Mellanox,” it’s completely obvious to everybody that we absolutely should. On the day that I said, “Hey guys, let’s go all in on deep learning,” and let me tell you why. I’ve already been laying down the bricks to different organizations inside the company. Every organization and everybody, many of the people might have heard everything. Most of the company hears, of course, pieces of it. And on the day that I announce it, everybody’s kind of bought in to many pieces of it.
I’m trying to shape their belief systems such that when I come the day I say, “Hey, let’s buy Mellanox,” it’s completely obvious to everybody that we absolutely should. On the day that I said, “Hey guys, let’s go all in on deep learning,” and let me tell you why. I’ve already been laying down the bricks to different organizations inside the company. Every organization and everybody, many of the people might have heard everything. Most of the company hears, of course, pieces of it. And on the day that I announce it, everybody’s kind of bought in to many pieces of it.
Jensen Huang
And in a lot of ways, I like to announce these things, and I imagine that the employees are kind of saying, “You know, Jensen, what took you so long?” And in fact, I’ve been shaping their belief system for some time, and therefore leadership. Sometimes it looks like you’re leading from behind, but you’ve been shaping their, you know, to the point where on the day that I declared it, 100% buy-in. But that’s what you want. You want to bring everybody along. Otherwise, we announce something about deep learning and everybody goes, “What are you talking about?” You know, you announce something about let’s go all in on this thing, and your management team, your board, your employees, your customers, they’re kind of like, “Where’s this coming from?”
And in a lot of ways, I like to announce these things, and I imagine that the employees are kind of saying, “You know, Jensen, what took you so long?” And in fact, I’ve been shaping their belief system for some time, and therefore leadership. Sometimes it looks like you’re leading from behind, but you’ve been shaping their, you know, to the point where on the day that I declared it, 100% buy-in. But that’s what you want. You want to bring everybody along. Otherwise, we announce something about deep learning and everybody goes, “What are you talking about?” You know, you announce something about let’s go all in on this thing, and your management team, your board, your employees, your customers, they’re kind of like, “Where’s this coming from?”
Jensen Huang
You know, this is insane.” And so, so GTC effect, if you go back in time, you look at, look at the keynotes, I’m also shaping the belief system of my partners in the industry and, and I’m using that to shape, you know, the belief system of my own employees. And, and, and so by the time that I announce something, like for example, we just announ- we just announced Grok. We’ve been late… I’ve been talking about the stepping stones for two and a half years. You just go back and go, “Oh my gosh, they’ve been talking about it for two and a half years.” And so I’ve been laying the foundation step by step by step, so when the time comes you announce it, everybody’s saying, “You know, what took you so long?”
You know, this is insane.” And so, so GTC effect, if you go back in time, you look at, look at the keynotes, I’m also shaping the belief system of my partners in the industry and, and I’m using that to shape, you know, the belief system of my own employees. And, and, and so by the time that I announce something, like for example, we just announ- we just announced Grok. We’ve been late… I’ve been talking about the stepping stones for two and a half years. You just go back and go, “Oh my gosh, they’ve been talking about it for two and a half years.” And so I’ve been laying the foundation step by step by step, so when the time comes you announce it, everybody’s saying, “You know, what took you so long?”
Lex Fridman
But it’s not just inside the company. You’re shaping the landscape, the broader global landscape of innovation. Like, putting those ideas out there, you really are manifesting reality.
But it’s not just inside the company. You’re shaping the landscape, the broader global landscape of innovation. Like, putting those ideas out there, you really are manifesting reality.
Jensen Huang
We don’t build computers. We actually don’t build clouds. We don’t… As it turns out, we’re a computing platform company. And so nobody can buy anything from us. That’s the weird thing. You know, we vertically design, vertically integrate to design and optimize, but then we open up the entire platform at every single layer to be integrated into other companies’ products and services and clouds and supercomputers and OEM computers, and so the amazing thing is, I can’t do what I do without having convinced them first. And so most of GTC is about manifesting a future that by the time that we… My product is ready, they’re going, “What took you so long?”
We don’t build computers. We actually don’t build clouds. We don’t… As it turns out, we’re a computing platform company. And so nobody can buy anything from us. That’s the weird thing. You know, we vertically design, vertically integrate to design and optimize, but then we open up the entire platform at every single layer to be integrated into other companies’ products and services and clouds and supercomputers and OEM computers, and so the amazing thing is, I can’t do what I do without having convinced them first. And so most of GTC is about manifesting a future that by the time that we… My product is ready, they’re going, “What took you so long?”
AI scaling laws
Lex Fridman
Yeah. So one of the things you’ve been a believer for a long time is scaling laws, broadly defined. So are you still a believer in the scaling laws?
Yeah. So one of the things you’ve been a believer for a long time is scaling laws, broadly defined. So are you still a believer in the scaling laws?
Jensen Huang
Yeah, yeah. Yeah, we have more scaling laws now.
Yeah, yeah. Yeah, we have more scaling laws now.
Lex Fridman
So I think you’ve outlined four of them with pre-training, post-training, test time, and agentic scaling. What do you think, when you think about the future, deep future and the near-term future, what are the blockers that you’re most concerned about that keep you up at night that you have to overcome in order to keep scaling?
So I think you’ve outlined four of them with pre-training, post-training, test time, and agentic scaling. What do you think, when you think about the future, deep future and the near-term future, what are the blockers that you’re most concerned about that keep you up at night that you have to overcome in order to keep scaling?
Jensen Huang
Well, we can go back and reflect on what people thought were blockers. So in the beginning, we were… The pre-training scaling law. You know, people thought, rightfully so, that the amount of data that we have, high-quality data that we have, will limit the intelligence that we achieve. And that scaling law was an important, very important scaling law. The larger the model, the correspondingly more data results in a smarter AI. And so that was pre-training. And Ilya Sutskever said, “We’re out of data,” or something like that. “Pre-training is over,” or something like that. The industry panicked, you know, that this is the end of AI. And of course, that’s obviously not true.
Well, we can go back and reflect on what people thought were blockers. So in the beginning, we were… The pre-training scaling law. You know, people thought, rightfully so, that the amount of data that we have, high-quality data that we have, will limit the intelligence that we achieve. And that scaling law was an important, very important scaling law. The larger the model, the correspondingly more data results in a smarter AI. And so that was pre-training. And Ilya Sutskever said, “We’re out of data,” or something like that. “Pre-training is over,” or something like that. The industry panicked, you know, that this is the end of AI. And of course, that’s obviously not true.
Jensen Huang
We’re gonna keep on scaling the amount of data that we have to train with. A lot of that data is probably gonna be synthetic, and that also confused people, you know? And what people don’t realize is they’ve kind of forgotten that most of the data that we are training, that we teach each other with, inform each other with, is synthetic. You know, it’s synthetic because it didn’t come out of nature. You created it. I’m consuming it. I modify it, augment it, I regenerate it, somebody else consumes it. And so we’ve now reached a level where AI is able to take ground truth, augment it… Enhance it, synthetically generate an enormous amount of data.
We’re gonna keep on scaling the amount of data that we have to train with. A lot of that data is probably gonna be synthetic, and that also confused people, you know? And what people don’t realize is they’ve kind of forgotten that most of the data that we are training, that we teach each other with, inform each other with, is synthetic. You know, it’s synthetic because it didn’t come out of nature. You created it. I’m consuming it. I modify it, augment it, I regenerate it, somebody else consumes it. And so we’ve now reached a level where AI is able to take ground truth, augment it… Enhance it, synthetically generate an enormous amount of data.
Jensen Huang
And that part of post-training continues to scale, and so the amount of data that we could use that is human generated will be smaller, and smaller, and smaller. The amount of data that we use to train models is going to continue to scale to the point where we’re no longer limited… Training is no longer limited by… Data is now limited by compute. And the reason for that is most of the data is synthetic. Then the next phase is test time, and I still remember people telling me that, “Inference? Oh, yeah, that’s easy. Pre-training, that’s hard.” These are giant systems that people are talking about. Inference must be easy. And so inference chips are gonna be little tiny chips, and-
And that part of post-training continues to scale, and so the amount of data that we could use that is human generated will be smaller, and smaller, and smaller. The amount of data that we use to train models is going to continue to scale to the point where we’re no longer limited… Training is no longer limited by… Data is now limited by compute. And the reason for that is most of the data is synthetic. Then the next phase is test time, and I still remember people telling me that, “Inference? Oh, yeah, that’s easy. Pre-training, that’s hard.” These are giant systems that people are talking about. Inference must be easy. And so inference chips are gonna be little tiny chips, and-
Jensen Huang
… you know, they’re not, they’re not like NVIDIA’s chips. Oh, those are gonna be complicated and expensive, and, you know, we could make… And this is- … in, in the future, inference is gonna be the biggest market, and it’s gonna be easy, and we’re gonna commoditize it. You know, everybody can build their own chips. And, and that was always illogical to me because inference is thinking, and I think thinking is hard. Thinking is way harder than reading.
… you know, they’re not, they’re not like NVIDIA’s chips. Oh, those are gonna be complicated and expensive, and, you know, we could make… And this is- … in, in the future, inference is gonna be the biggest market, and it’s gonna be easy, and we’re gonna commoditize it. You know, everybody can build their own chips. And, and that was always illogical to me because inference is thinking, and I think thinking is hard. Thinking is way harder than reading.
Jensen Huang
You know, pre-training is just memorization and generalization, you know, and looking for patterns in relationships. You’re reading and reading, versus thinking, reasoning, solving problems, taking unexplored experiences, new experiences, and breaking it down into… Decomposing it into, you know, solvable pieces that we then go off, either through first principle reasoning, or, you know, through previous examples, prior experiences. You know, or just exploration and search and, you know, trying different things. And that whole process of test time scaling inference, is really about thinking. And it’s about reasoning, it’s about planning, it’s about search, it’s about…
You know, pre-training is just memorization and generalization, you know, and looking for patterns in relationships. You’re reading and reading, versus thinking, reasoning, solving problems, taking unexplored experiences, new experiences, and breaking it down into… Decomposing it into, you know, solvable pieces that we then go off, either through first principle reasoning, or, you know, through previous examples, prior experiences. You know, or just exploration and search and, you know, trying different things. And that whole process of test time scaling inference, is really about thinking. And it’s about reasoning, it’s about planning, it’s about search, it’s about…
Jensen Huang
And so how could that possibly be compute light? And we were absolutely right about that. You know, so test time scaling is intensely compute intensive. Then the question is, okay, now we’re at inference and we’re at test time scaling, what’s beyond that? Well, obviously we have now created, you know, one agentic person, and that one agentic person has a large language model that we’ve now developed. But during test time, that agentic system goes off and does research and bangs on databases, and it goes out and, you know, uses tools, and one of the most important things it does is spins off and spawns off a whole bunch of sub-agents. Which means we’re now creating large teams. It’s so much easier to scale NVIDIA by hiring more employees than it is to scale myself.
And so how could that possibly be compute light? And we were absolutely right about that. You know, so test time scaling is intensely compute intensive. Then the question is, okay, now we’re at inference and we’re at test time scaling, what’s beyond that? Well, obviously we have now created, you know, one agentic person, and that one agentic person has a large language model that we’ve now developed. But during test time, that agentic system goes off and does research and bangs on databases, and it goes out and, you know, uses tools, and one of the most important things it does is spins off and spawns off a whole bunch of sub-agents. Which means we’re now creating large teams. It’s so much easier to scale NVIDIA by hiring more employees than it is to scale myself.
Jensen Huang
And so the next scaling law is the agentic scaling law. It’s kind of like multiplying AI. Multiplying AI, we could spin off agents as fast as you want to spin off agents. And so, you know, I… You know, I have four scaling laws. And as we use the agentic systems, they’re gonna create a lot more data, they’re gonna create a lot of experiences. Some of it we’re gonna say, “Wow, this is really good. We ought to memorize this.”
And so the next scaling law is the agentic scaling law. It’s kind of like multiplying AI. Multiplying AI, we could spin off agents as fast as you want to spin off agents. And so, you know, I… You know, I have four scaling laws. And as we use the agentic systems, they’re gonna create a lot more data, they’re gonna create a lot of experiences. Some of it we’re gonna say, “Wow, this is really good. We ought to memorize this.”
Jensen Huang
That data set then comes all the way back to pre-training. We memorize and generalize it. We then refine it and fine-tune it back into post-training. Then we enhance it even more with test time, you know, and the agentic systems, you know, put it out to the industry. And so this loop, this cycle, is gonna go on and on and on. It kinda comes down to basically intelligence is gonna scale by one thing, and that’s compute.
That data set then comes all the way back to pre-training. We memorize and generalize it. We then refine it and fine-tune it back into post-training. Then we enhance it even more with test time, you know, and the agentic systems, you know, put it out to the industry. And so this loop, this cycle, is gonna go on and on and on. It kinda comes down to basically intelligence is gonna scale by one thing, and that’s compute.
Lex Fridman
But there’s a tricky thing there that you have to anticipate and predict, which is some of these components, it requires different kind of hardware to really do it optimally. So you have to anticipate where the AI innovation’s going to lead. For example, a mixture of-
But there’s a tricky thing there that you have to anticipate and predict, which is some of these components, it requires different kind of hardware to really do it optimally. So you have to anticipate where the AI innovation’s going to lead. For example, a mixture of-
Jensen Huang
Perfect.
Perfect.
Lex Fridman
… experts with sparsity.
… experts with sparsity.
Jensen Huang
Perfect.
Perfect.
Lex Fridman
With hardware, you can’t just pivot on a week’s notice. You have to anticipate what that’s going to look like. It has some-
With hardware, you can’t just pivot on a week’s notice. You have to anticipate what that’s going to look like. It has some-
Jensen Huang
So good.
So good.
Lex Fridman
… that’s so scary and difficult to do, right?
… that’s so scary and difficult to do, right?
Jensen Huang
For example, these AI model architectures are being invented about once every six months. Right? And system architectures and hardware architectures kind of every three years. And so you need to anticipate what likely is going to happen, you know, two, three years from now. And there’s a couple ways that you could do that. First of all, we could do research internally ourselves, and that’s one of the reasons why we have basic research, we have applied research.
For example, these AI model architectures are being invented about once every six months. Right? And system architectures and hardware architectures kind of every three years. And so you need to anticipate what likely is going to happen, you know, two, three years from now. And there’s a couple ways that you could do that. First of all, we could do research internally ourselves, and that’s one of the reasons why we have basic research, we have applied research.
Jensen Huang
We create our own models. And so we have hands-on life experience right here. This is part of the co-design that I’m talking about. We’re also the only AI company in the world that works with literally every AI company in the world. And to the extent that we can, we try to get a sense of what are the challenges that people are experiencing.
We create our own models. And so we have hands-on life experience right here. This is part of the co-design that I’m talking about. We’re also the only AI company in the world that works with literally every AI company in the world. And to the extent that we can, we try to get a sense of what are the challenges that people are experiencing.
Lex Fridman
So you’re listening to the whispers across the industry, the AI labs.
So you’re listening to the whispers across the industry, the AI labs.
Jensen Huang
That’s right. You got to listen and learn from everybody. And have a… And then the last part is to have an architecture that’s flexible, that can adapt and move with the wind. And one of the benefits of CUDA is that it’s, you know, on the one hand, an incredible accelerator. On the other hand, it’s really flexible. And so that balance, incredible balance between specialization, otherwise we can’t accelerate the CPU, versus generalization, so that we can adapt with changing algorithms, that’s really, really important. That’s the reason why CUDA has been so resilient on the one hand, and yet we continue to enhance it.
That’s right. You got to listen and learn from everybody. And have a… And then the last part is to have an architecture that’s flexible, that can adapt and move with the wind. And one of the benefits of CUDA is that it’s, you know, on the one hand, an incredible accelerator. On the other hand, it’s really flexible. And so that balance, incredible balance between specialization, otherwise we can’t accelerate the CPU, versus generalization, so that we can adapt with changing algorithms, that’s really, really important. That’s the reason why CUDA has been so resilient on the one hand, and yet we continue to enhance it.
Jensen Huang
We’re at CUDA 13.2, and so we’re evolving the architecture so fast that we can stay with the modern algorithms. For example… When mixtures of experts came out, that’s the reason why we had NVLink 72 instead of NVLink 8. We could now take an entire 4 trillion, 10 trillion parameter model and put it in one computing domain as if it’s running on one GPU. People probably didn’t notice, I said it, but if you look at the architecture of the Grace Blackwell racks, it was completely focused on doing one thing, processing the LLM. All of a sudden, one year later, you’re looking at a Vera Rubin rack. It has storage accelerators. It has this incredible new CPU called Vera. It has Vera Rubin and NVLink 72 to run the LLMs.
We’re at CUDA 13.2, and so we’re evolving the architecture so fast that we can stay with the modern algorithms. For example… When mixtures of experts came out, that’s the reason why we had NVLink 72 instead of NVLink 8. We could now take an entire 4 trillion, 10 trillion parameter model and put it in one computing domain as if it’s running on one GPU. People probably didn’t notice, I said it, but if you look at the architecture of the Grace Blackwell racks, it was completely focused on doing one thing, processing the LLM. All of a sudden, one year later, you’re looking at a Vera Rubin rack. It has storage accelerators. It has this incredible new CPU called Vera. It has Vera Rubin and NVLink 72 to run the LLMs.
Jensen Huang
It also has this new additional rack called Rock. And so this entire rack system is completely different than the previous one, and it’s got all these new components in it. And the reason for that is because the last one was designed to run MoE large language models, inference. And this one is to run agents and agents bang on tools, and-
It also has this new additional rack called Rock. And so this entire rack system is completely different than the previous one, and it’s got all these new components in it. And the reason for that is because the last one was designed to run MoE large language models, inference. And this one is to run agents and agents bang on tools, and-
Lex Fridman
Obviously, the design of the system had to have been done before Claude Code, Codex, OpenClaw. So you were anticipating the future, essentially. And that, and that comes from what? From the whispers, from the understanding what all the state-
Obviously, the design of the system had to have been done before Claude Code, Codex, OpenClaw. So you were anticipating the future, essentially. And that, and that comes from what? From the whispers, from the understanding what all the state-
Jensen Huang
No
No
Lex Fridman
… of the art is about?
… of the art is about?
Jensen Huang
No, it’s easier than that. You just reason about it. First of all, you just reason. No matter, no matter what happens, at some point in order for that large language model to be a digital worker… Let’s just use that metaphor. Let’s say that we want the LLM to be a digital worker. What does that have to do? It has to access ground truth. That’s our file system. It has to be able to do research. It doesn’t know everything. We don’t have… And I don’t wanna wait until this AI becomes, you know, universally smart about everything, past, present, and future before I make it useful. And so therefore, I might as well let it go do research. It’s obvious; if it wants to help me, it’s gotta use my tools.
No, it’s easier than that. You just reason about it. First of all, you just reason. No matter, no matter what happens, at some point in order for that large language model to be a digital worker… Let’s just use that metaphor. Let’s say that we want the LLM to be a digital worker. What does that have to do? It has to access ground truth. That’s our file system. It has to be able to do research. It doesn’t know everything. We don’t have… And I don’t wanna wait until this AI becomes, you know, universally smart about everything, past, present, and future before I make it useful. And so therefore, I might as well let it go do research. It’s obvious; if it wants to help me, it’s gotta use my tools.
Jensen Huang
You know, a lot of people would say, “You know AI is gonna completely destroy software. We don’t need software anymore. We don’t even need tools anymore.” That’s ridiculous. Let’s use the… Let’s use a thought experiment. And you could just sit there, enjoy a glass of whiskey, and think about all these things, and it would become completely obvious. Like, if I were to create the most amazing agent that we can imagine in the next 10 years. Let’s say it’d be a humanoid robot. If that humanoid robot were to be created, is it more likely that the humanoid robot comes into my house and uses the tools that I have to do the work that it needs to do?
You know, a lot of people would say, “You know AI is gonna completely destroy software. We don’t need software anymore. We don’t even need tools anymore.” That’s ridiculous. Let’s use the… Let’s use a thought experiment. And you could just sit there, enjoy a glass of whiskey, and think about all these things, and it would become completely obvious. Like, if I were to create the most amazing agent that we can imagine in the next 10 years. Let’s say it’d be a humanoid robot. If that humanoid robot were to be created, is it more likely that the humanoid robot comes into my house and uses the tools that I have to do the work that it needs to do?
Jensen Huang
Or does this hand turn into a 10-pound hammer in one instance, turn into a scalpel in another instance, and in order to boil water, it beams, you know, microwaves out of its fingers? You know, or is it more likely just to use a microwave, you know? And the first time it goes up to the microwave, it probably doesn’t know how to use it. But that’s okay. It’s connected to the internet. It reads the manual of this microwave, reads it, instantly becomes an expert. And so it uses it. And so I think the… I just described, in fact, almost all of the properties of OpenClaw.
Or does this hand turn into a 10-pound hammer in one instance, turn into a scalpel in another instance, and in order to boil water, it beams, you know, microwaves out of its fingers? You know, or is it more likely just to use a microwave, you know? And the first time it goes up to the microwave, it probably doesn’t know how to use it. But that’s okay. It’s connected to the internet. It reads the manual of this microwave, reads it, instantly becomes an expert. And so it uses it. And so I think the… I just described, in fact, almost all of the properties of OpenClaw.
Jensen Huang
You know, that it’s gonna use tools, that it’s gonna access files, it’s gonna be able to do research. It has an IO subsystem. And when you’re done reasoning through it, reasoning about it in that way, then you say, “Oh, my gosh, the impact to the future of computing is deeply profound.” And the reason for that is, I think we’ve just reinvented the computer. And then now you say, “Okay, when did we reason about that? When did we reason about OpenClaw?” If you take the OpenClaw schematic that I used at GTC, you’ll find it two years ago. Literally, two years ago at GTC, I was talking about agentic systems that exactly reflect OpenClaw today. And, of course, the confluence of many things had to happen.
You know, that it’s gonna use tools, that it’s gonna access files, it’s gonna be able to do research. It has an IO subsystem. And when you’re done reasoning through it, reasoning about it in that way, then you say, “Oh, my gosh, the impact to the future of computing is deeply profound.” And the reason for that is, I think we’ve just reinvented the computer. And then now you say, “Okay, when did we reason about that? When did we reason about OpenClaw?” If you take the OpenClaw schematic that I used at GTC, you’ll find it two years ago. Literally, two years ago at GTC, I was talking about agentic systems that exactly reflect OpenClaw today. And, of course, the confluence of many things had to happen.
Jensen Huang
First of all, we needed Claude and GPT and, you know, all of these models to reach a level of capability. So their innovation and their breakthroughs and their continued advances was really important. And then, of course, somebody had to create an open source project that was sufficiently robust and sufficiently complete and that we can all put to work. And I think OpenClaw did for agentic systems what ChatGPT did for generative systems. And I just think it’s a very big deal.
First of all, we needed Claude and GPT and, you know, all of these models to reach a level of capability. So their innovation and their breakthroughs and their continued advances was really important. And then, of course, somebody had to create an open source project that was sufficiently robust and sufficiently complete and that we can all put to work. And I think OpenClaw did for agentic systems what ChatGPT did for generative systems. And I just think it’s a very big deal.
Lex Fridman
Yeah, it’s a really special moment. I’m not exactly sure why it captured so much of the world’s attention, but it did, more than Claude Code and Codex and so on.
Yeah, it’s a really special moment. I’m not exactly sure why it captured so much of the world’s attention, but it did, more than Claude Code and Codex and so on.
Jensen Huang
Because consumers could reach it.
Because consumers could reach it.
Lex Fridman
Sure, yeah. But there’s also so much of this is vibes. And Peter, I had a podcast with him, he’s a wonderful human being. So part of it is also the humans that represent the thing.
Sure, yeah. But there’s also so much of this is vibes. And Peter, I had a podcast with him, he’s a wonderful human being. So part of it is also the humans that represent the thing.
Jensen Huang
Yeah, no doubt.
Yeah, no doubt.
Lex Fridman
Part of it is memes and the— ‘Cause we’re all trying to figure it out. There’s really serious and complicated security concerns about when you have such powerful technology, how do you hand over your data so they can do useful stuff? But then there’s scary things associated with that. And we, as a civilization, as individual people and as a civilization, are figuring out how to find that right balance.
Part of it is memes and the— ‘Cause we’re all trying to figure it out. There’s really serious and complicated security concerns about when you have such powerful technology, how do you hand over your data so they can do useful stuff? But then there’s scary things associated with that. And we, as a civilization, as individual people and as a civilization, are figuring out how to find that right balance.
Jensen Huang
Yeah, we jumped on it right away and we sent a bunch of security experts this way. And we did this thing called OpenShell. It’s already been integrated into OpenClaw.
Yeah, we jumped on it right away and we sent a bunch of security experts this way. And we did this thing called OpenShell. It’s already been integrated into OpenClaw.
Lex Fridman
And NVIDIA put forward NemoClaw.
And NVIDIA put forward NemoClaw.
Jensen Huang
Yep, exactly.
Yep, exactly.
Lex Fridman
They install super easy. It makes sure that it’s secure.
They install super easy. It makes sure that it’s secure.
Jensen Huang
We give you two out of three rights. Agentic systems can access sensitive information, it can execute code, and it can communicate externally. We could keep things safe if we gave you two out of those three capabilities at any time, but not all three. And out of those two out of three capabilities, we also give you access control based on whatever rights that you’re given by enterprise. And then we connect it to a policy engine that all these enterprises already have. And so we’re going to try to do our best to help OpenClaw become a better claw.
We give you two out of three rights. Agentic systems can access sensitive information, it can execute code, and it can communicate externally. We could keep things safe if we gave you two out of those three capabilities at any time, but not all three. And out of those two out of three capabilities, we also give you access control based on whatever rights that you’re given by enterprise. And then we connect it to a policy engine that all these enterprises already have. And so we’re going to try to do our best to help OpenClaw become a better claw.
Biggest blockers to AI scaling laws
Lex Fridman
So you eloquently explained how we have a long history of blockers that we thought were going to be blockers, and we overcame them. But now looking into the future, what do you think might be the blockers now that it’s clear that agents will be everywhere? So it’s obviously we’re going to need compute. So what is going to be the blocker for that scaling?
So you eloquently explained how we have a long history of blockers that we thought were going to be blockers, and we overcame them. But now looking into the future, what do you think might be the blockers now that it’s clear that agents will be everywhere? So it’s obviously we’re going to need compute. So what is going to be the blocker for that scaling?
Jensen Huang
Power is a concern, but it’s not the only concern. But that’s the reason why we’re pushing so hard on extreme co-design, so that we can improve the tokens per second per watt orders of magnitude every single year. And so in the last 10 years, Moore’s Law would have progressed computing about 100 times in the last 10 years. We progressed and scaled up computing by a million times in the last 10 years. And so we’re gonna keep on doing that through extreme co-design. So energy efficiency, perf per watt, completely affects the revenues of a company. It affects the revenues of a factory. And we’re just going to push that to the limit so that we can keep on driving token costs down as fast as we can.
Power is a concern, but it’s not the only concern. But that’s the reason why we’re pushing so hard on extreme co-design, so that we can improve the tokens per second per watt orders of magnitude every single year. And so in the last 10 years, Moore’s Law would have progressed computing about 100 times in the last 10 years. We progressed and scaled up computing by a million times in the last 10 years. And so we’re gonna keep on doing that through extreme co-design. So energy efficiency, perf per watt, completely affects the revenues of a company. It affects the revenues of a factory. And we’re just going to push that to the limit so that we can keep on driving token costs down as fast as we can.
Jensen Huang
You know, our computer price is going up, but our token generation effectiveness is going up so much faster that token cost is coming down. It’s just coming down an order of magnitude every year.
You know, our computer price is going up, but our token generation effectiveness is going up so much faster that token cost is coming down. It’s just coming down an order of magnitude every year.
Lex Fridman
So power, that’s an interesting one. So the way to try to get around the power blocker is to try to, with the tokens per second per watt, try to make it more and more efficient. Of course, there’s the question of how do we get more power.
So power, that’s an interesting one. So the way to try to get around the power blocker is to try to, with the tokens per second per watt, try to make it more and more efficient. Of course, there’s the question of how do we get more power.
Jensen Huang
We should also get more power.
We should also get more power.
Supply chain
Lex Fridman
That’s a really complicated one. You’ve talked about small modular nuclear power plants. There’s all kinds of ideas for energy. How much does it keep you up at night? The bottlenecks in the supply chain of AI, like ASML with EUV lithography machines, TSMC with advanced packaging like CoWoS, and SK Hynix with the high bandwidth memory?
That’s a really complicated one. You’ve talked about small modular nuclear power plants. There’s all kinds of ideas for energy. How much does it keep you up at night? The bottlenecks in the supply chain of AI, like ASML with EUV lithography machines, TSMC with advanced packaging like CoWoS, and SK Hynix with the high bandwidth memory?
Jensen Huang
All the time, and we’re working on it all the time. No company in history has ever grown at a scale that we’re growing while accelerating that growth. It’s incredible. And it’s hard for people to even understand this. In the overall world of AI computing, we’re increasing share. And so supply chain, upstream and downstream, are really important to us. I spend a lot of time informing all the CEOs that I work with: what are the dynamics that’s going to cause the growth to continue or even accelerate? It’s part of the reasons why to the entire right-hand side of me were CEOs of practically the entire IT industry upstream and practically the entire infrastructure industry downstream.
All the time, and we’re working on it all the time. No company in history has ever grown at a scale that we’re growing while accelerating that growth. It’s incredible. And it’s hard for people to even understand this. In the overall world of AI computing, we’re increasing share. And so supply chain, upstream and downstream, are really important to us. I spend a lot of time informing all the CEOs that I work with: what are the dynamics that’s going to cause the growth to continue or even accelerate? It’s part of the reasons why to the entire right-hand side of me were CEOs of practically the entire IT industry upstream and practically the entire infrastructure industry downstream.
Jensen Huang
And they were all… There were several hundred CEOs. And I don’t think there’s ever been keynotes where several hundred CEOs show up. And part of it is, I’m telling them about our business condition now. I’m telling them about the growth drivers in the very near future and what’s happening. And I’m also describing where are we going to go next so that they could use all of this information and all of the dynamics that are here to inform how they want to invest. And so I inform them that way like I inform my own employees.
And they were all… There were several hundred CEOs. And I don’t think there’s ever been keynotes where several hundred CEOs show up. And part of it is, I’m telling them about our business condition now. I’m telling them about the growth drivers in the very near future and what’s happening. And I’m also describing where are we going to go next so that they could use all of this information and all of the dynamics that are here to inform how they want to invest. And so I inform them that way like I inform my own employees.
Memory
Jensen Huang
And then of course, then I make trips out to them and make sure that, “Hey, listen, I want you to know this quarter, this coming year, this next year, these things are going to happen.” And if you look at the CEOs of the DRAM industry—the number one DRAM in the world was DDR memory for CPUs in data centers. About three years ago, I was able to convince several of the CEOs that even though at the time HBM memory was used quite scarcely, and barely by supercomputers, that this was going to be a mainstream memory for data centers in the future. At first it sounded ridiculous, but several of the CEOs believed me and decided to invest in building HBM memories.
And then of course, then I make trips out to them and make sure that, “Hey, listen, I want you to know this quarter, this coming year, this next year, these things are going to happen.” And if you look at the CEOs of the DRAM industry—the number one DRAM in the world was DDR memory for CPUs in data centers. About three years ago, I was able to convince several of the CEOs that even though at the time HBM memory was used quite scarcely, and barely by supercomputers, that this was going to be a mainstream memory for data centers in the future. At first it sounded ridiculous, but several of the CEOs believed me and decided to invest in building HBM memories.
Jensen Huang
Another memory was rather odd to put into a data center: the low power memories that we use for cell phones. And we wanted them to adapt them for supercomputers in the data center. And they go, “Cell phone memory for supercomputers?” And I explained to them why. Well, look at these two memories, LPDDR5, HBM4. The volumes are so incredible. All three of them had record years in history, and these are 45-year companies. And so, you know, I… That’s part of my job, is to inform and shape, inspire, you know.
Another memory was rather odd to put into a data center: the low power memories that we use for cell phones. And we wanted them to adapt them for supercomputers in the data center. And they go, “Cell phone memory for supercomputers?” And I explained to them why. Well, look at these two memories, LPDDR5, HBM4. The volumes are so incredible. All three of them had record years in history, and these are 45-year companies. And so, you know, I… That’s part of my job, is to inform and shape, inspire, you know.
Lex Fridman
So you’re not just manifesting the future and maybe inspiring NVIDIA, the different engineers of the company, you’re manifesting the supply chain of the future. So you’re having conversations with TSMC, with ASML.
So you’re not just manifesting the future and maybe inspiring NVIDIA, the different engineers of the company, you’re manifesting the supply chain of the future. So you’re having conversations with TSMC, with ASML.
Jensen Huang
Upstream, downstream.
Upstream, downstream.
Lex Fridman
Upstream, downstream. So that’s the thing.
Upstream, downstream. So that’s the thing.
Jensen Huang
GEV, Caterpillar. Yeah, that’s downstream from us. Yeah, yeah, there you go.
GEV, Caterpillar. Yeah, that’s downstream from us. Yeah, yeah, there you go.
Lex Fridman
Yeah, the whole thing. I mean, but that’s so… There’s so much incredibly difficult engineering that happens in the entire semiconductor industry, and it just feels scary how intricate the supply chain is, how many components there are, but it works somehow.
Yeah, the whole thing. I mean, but that’s so… There’s so much incredibly difficult engineering that happens in the entire semiconductor industry, and it just feels scary how intricate the supply chain is, how many components there are, but it works somehow.
Jensen Huang
Exactly, the deep science. The deep engineering, the incredible manufacturing, and so much of the manufacturing is already robotics, but we have a couple of hundred suppliers that contribute the technology that goes into our 1.3 million component rack. Each rack is 1.3, one and a half million components. There are 200 suppliers across the Vera Rubin rack.
Exactly, the deep science. The deep engineering, the incredible manufacturing, and so much of the manufacturing is already robotics, but we have a couple of hundred suppliers that contribute the technology that goes into our 1.3 million component rack. Each rack is 1.3, one and a half million components. There are 200 suppliers across the Vera Rubin rack.
Lex Fridman
So it’s interesting that you don’t list that as the thing that keeps you up at night in the list of blockers.
So it’s interesting that you don’t list that as the thing that keeps you up at night in the list of blockers.
Jensen Huang
But I’m doing, I’m doing all the things necessary to-
But I’m doing, I’m doing all the things necessary to-
Lex Fridman
Okay.
Okay.
Jensen Huang
… yeah, see? I can go to sleep because I checked it off. I said, “Okay,” you know, I go, I can go to sleep and I go, “Well, let’s see, let’s reason about this. What’s important for us?” Because let’s reason about this. Because we changed the system architecture from the original DGX-1 that you remembered to NVLink-72 rack scale computing- … what’s gonna… What does that, what does that mean? What does that mean to software? What does that mean to engineering? What does that mean to how we design and test? And what does that mean to the supply chain? Well, one of the things that it meant was we moved supercomputer integration at the data center into supercomputer manufacturing in the supply chain.
… yeah, see? I can go to sleep because I checked it off. I said, “Okay,” you know, I go, I can go to sleep and I go, “Well, let’s see, let’s reason about this. What’s important for us?” Because let’s reason about this. Because we changed the system architecture from the original DGX-1 that you remembered to NVLink-72 rack scale computing- … what’s gonna… What does that, what does that mean? What does that mean to software? What does that mean to engineering? What does that mean to how we design and test? And what does that mean to the supply chain? Well, one of the things that it meant was we moved supercomputer integration at the data center into supercomputer manufacturing in the supply chain.
Jensen Huang
If you’re doing that, you also have to recognize you’re gonna move one… And if your total footprint of whatever data center you’re gonna build, let’s say you would like to have, you know, 50 gigawatts of supercomputers that are running simultaneously, and it takes one week to manufacture that 50 gigawatts of supercomputers, then each week in the supply chain, the supercomputers are gonna need a gigawatt of power. And so we’re gonna need the supply chain to increase the amount of power it has to build and test the supercomputers in the supply chain before I ship it.
If you’re doing that, you also have to recognize you’re gonna move one… And if your total footprint of whatever data center you’re gonna build, let’s say you would like to have, you know, 50 gigawatts of supercomputers that are running simultaneously, and it takes one week to manufacture that 50 gigawatts of supercomputers, then each week in the supply chain, the supercomputers are gonna need a gigawatt of power. And so we’re gonna need the supply chain to increase the amount of power it has to build and test the supercomputers in the supply chain before I ship it.
Lex Fridman
Oh.
Oh.
Jensen Huang
Well, NVLink-72 literally builds supercomputers in the supply chain and ships ’em two, three tons at a time per rack. It used to be they used to come in parts and we used to assemble ’em inside the data center. But that’s impossible now because NVLink-72 is so dense. And so that’s an example. And I would have to go and fly into the supply chain, go meet my partners saying, “Hey,” I said, “guess what? So here’s what I’m going to do with… This is the way we used to build our DGXs. We’re gonna build them this way. This is gonna be so much better because we’re going to need ’em for inference.” The market for inference is, you know, coming. The inflection point for inference is coming. It’s gonna be a big market.
Well, NVLink-72 literally builds supercomputers in the supply chain and ships ’em two, three tons at a time per rack. It used to be they used to come in parts and we used to assemble ’em inside the data center. But that’s impossible now because NVLink-72 is so dense. And so that’s an example. And I would have to go and fly into the supply chain, go meet my partners saying, “Hey,” I said, “guess what? So here’s what I’m going to do with… This is the way we used to build our DGXs. We’re gonna build them this way. This is gonna be so much better because we’re going to need ’em for inference.” The market for inference is, you know, coming. The inflection point for inference is coming. It’s gonna be a big market.
Jensen Huang
And so I first explain to them what’s going on, why it’s gonna happen, and then I ask ’em to make several billion dollars of capital investments each. And because they trust me and I’m very respectful of ’em, and I give ’em every opportunity to question me and I spend time to explain things to people and I reason about it. I draw pictures and I reason about it in first principles. And by the time I’m done with them, they know what to do.
And so I first explain to them what’s going on, why it’s gonna happen, and then I ask ’em to make several billion dollars of capital investments each. And because they trust me and I’m very respectful of ’em, and I give ’em every opportunity to question me and I spend time to explain things to people and I reason about it. I draw pictures and I reason about it in first principles. And by the time I’m done with them, they know what to do.
Lex Fridman
So a lot of it is about relationships and building a shared view of the future. But do you worry about certain bottlenecks? I mean, what are the biggest bottlenecks in the supply chain? Are you worried about ASML’s EUV tooling? Are you worried about the packaging, CoWoS packaging of TSMC, about how fast it could scale? Like you said, you’re not only growing incredibly fast, you’re accelerating your growth. So it feels like everybody in the supply chain, and those are certainly bottlenecks, would have to scale up. Are you having conversations with them, like, how can you scale up faster?
So a lot of it is about relationships and building a shared view of the future. But do you worry about certain bottlenecks? I mean, what are the biggest bottlenecks in the supply chain? Are you worried about ASML’s EUV tooling? Are you worried about the packaging, CoWoS packaging of TSMC, about how fast it could scale? Like you said, you’re not only growing incredibly fast, you’re accelerating your growth. So it feels like everybody in the supply chain, and those are certainly bottlenecks, would have to scale up. Are you having conversations with them, like, how can you scale up faster?
Jensen Huang
All the time.
All the time.
Lex Fridman
Do you worry about it?
Do you worry about it?
Jensen Huang
No.
No.
Lex Fridman
Okay.
Okay.
Jensen Huang
Because I told ’em what I needed. They understood what I need. They told me what they’re gonna go do, and I believe them what they’re going to do.
Because I told ’em what I needed. They understood what I need. They told me what they’re gonna go do, and I believe them what they’re going to do.
Power
Lex Fridman
Interesting. That’s great to hear. So maybe if we can just linger on the power for a little bit. What are your hopes for how to solve the energy problem?
Interesting. That’s great to hear. So maybe if we can just linger on the power for a little bit. What are your hopes for how to solve the energy problem?
Jensen Huang
One of the areas, Lex, that I would love us to talk about and just get the message out, you know, our power grid is designed for the worst case condition with some margin. Well, 99% of the time we’re nowhere near the worst case condition because the worst case condition is a few days in the winter, a few days in the summer, and extreme weather. Most of the time we’re nowhere near the worst case condition and we’re probably running around, call it 60% of peak.
One of the areas, Lex, that I would love us to talk about and just get the message out, you know, our power grid is designed for the worst case condition with some margin. Well, 99% of the time we’re nowhere near the worst case condition because the worst case condition is a few days in the winter, a few days in the summer, and extreme weather. Most of the time we’re nowhere near the worst case condition and we’re probably running around, call it 60% of peak.
Jensen Huang
And so 99% of the time, our power grid has excess power, and they’re just sitting idle, but they have to be there sitting idle because just in case, when the time comes, hospitals have to be powered and, you know, infrastructure has to be powered and airports have to run and so on and so forth. And so the question that I have is whether we could go and help them understand and create contractual agreements and design computer architecture systems, data centers, such that when they need the maximum power for infrastructure in society, that the data centers would get less.
And so 99% of the time, our power grid has excess power, and they’re just sitting idle, but they have to be there sitting idle because just in case, when the time comes, hospitals have to be powered and, you know, infrastructure has to be powered and airports have to run and so on and so forth. And so the question that I have is whether we could go and help them understand and create contractual agreements and design computer architecture systems, data centers, such that when they need the maximum power for infrastructure in society, that the data centers would get less.
Jensen Huang
But that’s in a very rare instance anyways. And during that time, we either have a backup generator for that little part of it, or we just have our computers shift the workload somewhere else, or we have the computers just run slower. You know, we could degrade our performance, reduce our power consumption and provide for a, you know, slightly longer latency response, you know, when somebody asks for, you know, asks for an answer. And so I think that that, that way of using computers, of building data centers, instead of expecting 100% uptime—and these contracts that are really, really quite rigorous, it’s putting a lot of pressure on the grid to be able to… Now, they’re gonna have to increase from their maximum. I just wanna use their excess. It’s just sitting there.
But that’s in a very rare instance anyways. And during that time, we either have a backup generator for that little part of it, or we just have our computers shift the workload somewhere else, or we have the computers just run slower. You know, we could degrade our performance, reduce our power consumption and provide for a, you know, slightly longer latency response, you know, when somebody asks for, you know, asks for an answer. And so I think that that, that way of using computers, of building data centers, instead of expecting 100% uptime—and these contracts that are really, really quite rigorous, it’s putting a lot of pressure on the grid to be able to… Now, they’re gonna have to increase from their maximum. I just wanna use their excess. It’s just sitting there.
Lex Fridman
Yeah, that’s not talked about enough. So what’s stopping there? Is it regulation? Is it bureaucracy?
Yeah, that’s not talked about enough. So what’s stopping there? Is it regulation? Is it bureaucracy?
Jensen Huang
I think it’s a three-way problem. It starts with the end customer. The end customer puts requirements on the data centers that they can never not be available, okay? So that the end customer expects perfection. Now, in order to deliver that perfection, you need a combination of backup generators and your grid power supplier to deliver on perfection. And so everybody’s gotta have six nines. Well, I think first of all, right now, we ought to have everybody understand that when the customer asks for these things, you have somebody in your data center operations team disconnected from the CEO. I bet the CEO doesn’t know this. I’m gonna talk to all the CEOs.
I think it’s a three-way problem. It starts with the end customer. The end customer puts requirements on the data centers that they can never not be available, okay? So that the end customer expects perfection. Now, in order to deliver that perfection, you need a combination of backup generators and your grid power supplier to deliver on perfection. And so everybody’s gotta have six nines. Well, I think first of all, right now, we ought to have everybody understand that when the customer asks for these things, you have somebody in your data center operations team disconnected from the CEO. I bet the CEO doesn’t know this. I’m gonna talk to all the CEOs.
Jensen Huang
The CEOs are probably not paying any attention to the contracts that are being signed, and so everybody wants to sign the best contract, of course. And they go down to cloud service providers, and the two contract negotiators that are… You, I could just see them now. You know, negotiating these multi-year contracts. Both sides want, you know, the best contract. As a result, the CSPs then have to go down to the utilities, and they expect the nine, the six nines. And so I think, I think the first thing is just make sure that, that all of the customers, the CEOs and the customers realize what they’re asking for. Now, the second thing is we have to build data centers that gracefully degrade.
The CEOs are probably not paying any attention to the contracts that are being signed, and so everybody wants to sign the best contract, of course. And they go down to cloud service providers, and the two contract negotiators that are… You, I could just see them now. You know, negotiating these multi-year contracts. Both sides want, you know, the best contract. As a result, the CSPs then have to go down to the utilities, and they expect the nine, the six nines. And so I think, I think the first thing is just make sure that, that all of the customers, the CEOs and the customers realize what they’re asking for. Now, the second thing is we have to build data centers that gracefully degrade.
Jensen Huang
And so if the power, if the utility, if the grid tells us, “Listen, we’re gonna have to back you down to about 80%,” we’re gonna say, “That’s no problem at all.” We’re just gonna move our workload around. We’re gonna make sure that data’s never lost, but we can reduce the computing rate and use less energy. The quality of service degrades a little bit. For the critical workloads, I shift that somewhere else right away so I don’t have that problem, and so, you know, whichever data center still has 100% uptime, and so…
And so if the power, if the utility, if the grid tells us, “Listen, we’re gonna have to back you down to about 80%,” we’re gonna say, “That’s no problem at all.” We’re just gonna move our workload around. We’re gonna make sure that data’s never lost, but we can reduce the computing rate and use less energy. The quality of service degrades a little bit. For the critical workloads, I shift that somewhere else right away so I don’t have that problem, and so, you know, whichever data center still has 100% uptime, and so…
Lex Fridman
How difficult of an engineering problem is that, that smart, dynamic allocation of power in a data center?
How difficult of an engineering problem is that, that smart, dynamic allocation of power in a data center?
Jensen Huang
As soon as you could specify, you could engineer it. Beautifully put. So long as it obeys the laws of physics on first principles, I think we’re good.
As soon as you could specify, you could engineer it. Beautifully put. So long as it obeys the laws of physics on first principles, I think we’re good.
Lex Fridman
What was the third thing you were mentioning?
What was the third thing you were mentioning?
Jensen Huang
So the second thing is the, the data centers. And the third thing is we need the utilities to also recognize that this is an opportunity- … and instead of saying, “Look, it’s gonna take me five years to increase my grid capability,” if you have, if you’re willing to take power of this level of guarantee, I can make them available for you next month and at this price. And so if utilities also offered more segments of power delivery promises, then I think everybody will figure out what to do with it. Yeah, but there’s just way too much waste in the grid right now. We should go after it.
So the second thing is the, the data centers. And the third thing is we need the utilities to also recognize that this is an opportunity- … and instead of saying, “Look, it’s gonna take me five years to increase my grid capability,” if you have, if you’re willing to take power of this level of guarantee, I can make them available for you next month and at this price. And so if utilities also offered more segments of power delivery promises, then I think everybody will figure out what to do with it. Yeah, but there’s just way too much waste in the grid right now. We should go after it.
Elon and Colossus
Lex Fridman
You’ve highly lauded Elon and xAI’s accomplishment in Memphis, in building Colossus supercomputer, probably in record time in just four months. It’s now at 200,000 GPUs and growing very quickly. Is there something that you could speak to the understanding about his approach that’s instructive to, broadly to all the data center creators that enabled that kind of accomplishment? His approach to engineering, his approach to the whole management of construction, everything?
You’ve highly lauded Elon and xAI’s accomplishment in Memphis, in building Colossus supercomputer, probably in record time in just four months. It’s now at 200,000 GPUs and growing very quickly. Is there something that you could speak to the understanding about his approach that’s instructive to, broadly to all the data center creators that enabled that kind of accomplishment? His approach to engineering, his approach to the whole management of construction, everything?
Jensen Huang
First of all, Elon is deep in so many different topics. Yet he’s also a really good systems thinker. And so he’s able to think through multiple disciplines, and he obviously pushes things, questions everything, where they’re, number one, is it necessary? Number two, does it have to be done this way? And then number three, you know, does it have to take this long? And so he has the ability to question everything to the point where everything is down to its minimal amount that’s necessary, you can’t take anything else out. And yet the necessary capabilities of the product remains, you know? And so he is as minimalist as you could possibly imagine, and he does it at a system scale. I think… I also love the fact that he is represented. He is present at the point of action.
First of all, Elon is deep in so many different topics. Yet he’s also a really good systems thinker. And so he’s able to think through multiple disciplines, and he obviously pushes things, questions everything, where they’re, number one, is it necessary? Number two, does it have to be done this way? And then number three, you know, does it have to take this long? And so he has the ability to question everything to the point where everything is down to its minimal amount that’s necessary, you can’t take anything else out. And yet the necessary capabilities of the product remains, you know? And so he is as minimalist as you could possibly imagine, and he does it at a system scale. I think… I also love the fact that he is represented. He is present at the point of action.
Jensen Huang
You know, he’ll just go there. If there’s a problem, he’ll just go there and then, “Show me the problem.” You know, when you do all of this in combination, you overcome a lot of previous, “This is just the way we do it.” “You know, I’m waiting for them.” You know, I mean, it’s just, everybody has a lot of excuses. And so, and then the last thing is when you act personally with so much urgency, it causes everybody else to act with urgency, you know? And every supplier has a lot of customers going on. Every supplier has a lot of projects going on, and he makes it his business that he’s the top priority of everybody else’s projects. And so he does that by demonstrating it.
You know, he’ll just go there. If there’s a problem, he’ll just go there and then, “Show me the problem.” You know, when you do all of this in combination, you overcome a lot of previous, “This is just the way we do it.” “You know, I’m waiting for them.” You know, I mean, it’s just, everybody has a lot of excuses. And so, and then the last thing is when you act personally with so much urgency, it causes everybody else to act with urgency, you know? And every supplier has a lot of customers going on. Every supplier has a lot of projects going on, and he makes it his business that he’s the top priority of everybody else’s projects. And so he does that by demonstrating it.
Lex Fridman
Yeah, I’ve been in a bunch of those meetings. It’s just, it’s fun to watch, ’cause really, not enough people ask the question like, “Okay, so can this be done a lot faster, and how? Why does it have to take this long?”
Yeah, I’ve been in a bunch of those meetings. It’s just, it’s fun to watch, ’cause really, not enough people ask the question like, “Okay, so can this be done a lot faster, and how? Why does it have to take this long?”
Jensen Huang
Yeah, right.
Yeah, right.
Lex Fridman
And then in the… That becomes an engineering question often. And yes, I think when you get the ground truth of actually… I remember… One of the times I was hanging out with him, he literally is going through the entire process of how to plug in cables into a rack. He’s working with an engineer on the ground that’s doing that task, and he’s just trying to understand what does that process look like so it can be less error-prone. And just building up that intuition from every single task involved in putting together a data center-
And then in the… That becomes an engineering question often. And yes, I think when you get the ground truth of actually… I remember… One of the times I was hanging out with him, he literally is going through the entire process of how to plug in cables into a rack. He’s working with an engineer on the ground that’s doing that task, and he’s just trying to understand what does that process look like so it can be less error-prone. And just building up that intuition from every single task involved in putting together a data center-
Lex Fridman
… you start to immediately get a sense at the detailed scale and at the broad systems scale of where the inefficiencies are, and so you can make it more and more and more efficient. Plus you have the big hammer of being able to say, “Let’s do it totally different-“
… you start to immediately get a sense at the detailed scale and at the broad systems scale of where the inefficiencies are, and so you can make it more and more and more efficient. Plus you have the big hammer of being able to say, “Let’s do it totally different-“
Jensen Huang
Yeah. That’s right.
Yeah. That’s right.
Lex Fridman
“… and remove all possible blockers.”
“… and remove all possible blockers.”
Jensen Huang
That’s right.
That’s right.
Jensen’s approach to engineering and leadership
Lex Fridman
Is there parallels in the NVIDIA Extreme Systems co-design approach that you see in the way Elon approaches systems engineering?
Is there parallels in the NVIDIA Extreme Systems co-design approach that you see in the way Elon approaches systems engineering?
Jensen Huang
Well, first of all, co-design is an ultimate systems engineering problem. And so we approach the work that we do from that first principle. The other thing that we do and this is a philosophy that, a thought, a state of mind, I guess, a method that I started 30 years ago, and it’s called the speed of light. The speed of light is not just about the speed. The speed of light is my shorthand for what’s the limit of what physics can do. And so every single thing that we do is compared against the speed of light. Memory speed, math speed, power, cost, time, effort, number of people, manufacturing cycle time.
Well, first of all, co-design is an ultimate systems engineering problem. And so we approach the work that we do from that first principle. The other thing that we do and this is a philosophy that, a thought, a state of mind, I guess, a method that I started 30 years ago, and it’s called the speed of light. The speed of light is not just about the speed. The speed of light is my shorthand for what’s the limit of what physics can do. And so every single thing that we do is compared against the speed of light. Memory speed, math speed, power, cost, time, effort, number of people, manufacturing cycle time.
Jensen Huang
And when you think about latency versus throughput when you think about cost versus throughput, cost versus capacity, all of these things you test against the speed of light to achieve all of these different constraints separately. And then when you consider it together, you know you have to make compromises because a system that achieves extremely low latency versus a cheap, a system that achieves very high throughput are architected fundamentally differently. But you want to know what’s the speed of light of a system that achieves high throughput, what’s the speed of light of a system that achieves low latency? And then when you think about the total system, you can make trade-offs. And so I force everybody to think about what’s the first principles, the limits-
And when you think about latency versus throughput when you think about cost versus throughput, cost versus capacity, all of these things you test against the speed of light to achieve all of these different constraints separately. And then when you consider it together, you know you have to make compromises because a system that achieves extremely low latency versus a cheap, a system that achieves very high throughput are architected fundamentally differently. But you want to know what’s the speed of light of a system that achieves high throughput, what’s the speed of light of a system that achieves low latency? And then when you think about the total system, you can make trade-offs. And so I force everybody to think about what’s the first principles, the limits-
Jensen Huang
… the physical limits for everything before we do anything. And we test everything against that. And so that’s a good frame of mind. I don’t love the other methods, which is continuous improvement. The problem with continuous improvement, it… First of all, you should engineer something from first principles at the speed, you know, with speed of light thinking. Limit it only by physical limits, and physics limits. And after that, of course you would improve it over time. But I don’t like going into a problem and somebody says, “Hey, you know, it takes 74 days to do this today-” “… Right now. And we can do it for you in 72 days.” You know, I’d rather strip it all back to zero-
… the physical limits for everything before we do anything. And we test everything against that. And so that’s a good frame of mind. I don’t love the other methods, which is continuous improvement. The problem with continuous improvement, it… First of all, you should engineer something from first principles at the speed, you know, with speed of light thinking. Limit it only by physical limits, and physics limits. And after that, of course you would improve it over time. But I don’t like going into a problem and somebody says, “Hey, you know, it takes 74 days to do this today-” “… Right now. And we can do it for you in 72 days.” You know, I’d rather strip it all back to zero-
Jensen Huang
… and say, “First of all, explain to me why 74 days in the first place. And l- let’s note, let’s think about what’s possible today. And if I were to- to build it completely from scratch, you know, how long would it take?” Oftentimes, you’d be surprised. It might come to six days. Now, the rest of the six days, the 74, could be very well-reasoned and compromises, and, you know, cost reductions, and all kinds of different things. But at least you know what they are. And then now that you know that six days is possible, then the conversation from 74 to six, surprisingly much more effective.
… and say, “First of all, explain to me why 74 days in the first place. And l- let’s note, let’s think about what’s possible today. And if I were to- to build it completely from scratch, you know, how long would it take?” Oftentimes, you’d be surprised. It might come to six days. Now, the rest of the six days, the 74, could be very well-reasoned and compromises, and, you know, cost reductions, and all kinds of different things. But at least you know what they are. And then now that you know that six days is possible, then the conversation from 74 to six, surprisingly much more effective.
Lex Fridman
In such incredibly complex systems that you’re working with, is simplicity sometimes a good heuristic to reach for? I mean, if I can just… I mean, the pod, the Vera Rubin pod that you announced is just incredible. We’re talking about seven chips, seven chip types, five purpose-built rack types, 40 racks, 1.2 quadrillion transistors, nearly 20,000 NVIDIA dies, over 1,100 Rubin GPUs, 60 exaflops, 10 petabytes per second of scale bandwidth. That’s all just one…
In such incredibly complex systems that you’re working with, is simplicity sometimes a good heuristic to reach for? I mean, if I can just… I mean, the pod, the Vera Rubin pod that you announced is just incredible. We’re talking about seven chips, seven chip types, five purpose-built rack types, 40 racks, 1.2 quadrillion transistors, nearly 20,000 NVIDIA dies, over 1,100 Rubin GPUs, 60 exaflops, 10 petabytes per second of scale bandwidth. That’s all just one…
Jensen Huang
That’s just one pod.
That’s just one pod.
Lex Fridman
That’s just one pod .
That’s just one pod .
Jensen Huang
Yeah, that’s just one pod.
Yeah, that’s just one pod.
Lex Fridman
I mean, in- … so you have the… And then even the NVL72 rack alone is 1.3 million components, 1300 chips, 4,000 pods crammed into a single 19-inch wide rack.
I mean, in- … so you have the… And then even the NVL72 rack alone is 1.3 million components, 1300 chips, 4,000 pods crammed into a single 19-inch wide rack.
Jensen Huang
And Lex, we’re probably gonna have to crank out about 200 of these pods a week, just to put it in perspective.
And Lex, we’re probably gonna have to crank out about 200 of these pods a week, just to put it in perspective.
Lex Fridman
The amount of different components, I suppose simplicity is impossible, but is that a metric that you kind of reach for in trying to design things?
The amount of different components, I suppose simplicity is impossible, but is that a metric that you kind of reach for in trying to design things?
Jensen Huang
You know, the phrase that I use most often is, we need things to be as complex as necessary, but as simple as possible. And so the question is, is all that complexity there necessary? And we ought to test for that. And we got to challenge that. And then after that, everything else above it, you know, is gratuitous.
You know, the phrase that I use most often is, we need things to be as complex as necessary, but as simple as possible. And so the question is, is all that complexity there necessary? And we ought to test for that. And we got to challenge that. And then after that, everything else above it, you know, is gratuitous.
Lex Fridman
But it’s still almost incredible. Semiconductor industry broadly, but what NVIDIA is doing is some of the greatest engineering in history. So these systems are just truly, truly marvels of engineering.
But it’s still almost incredible. Semiconductor industry broadly, but what NVIDIA is doing is some of the greatest engineering in history. So these systems are just truly, truly marvels of engineering.
Jensen Huang
It is the most complex computer the world has ever made.
It is the most complex computer the world has ever made.
Lex Fridman
Yeah, the engineering teams, I mean- … I don’t, it’s not a competition, but I don’t know. If it was like an Olympics of engineering teams, I mean, TSMC does incredible engineering. Like I said, ASML at every scale, but NVIDIA is gonna give them a run for their money. Just incredible, incredible teams.
Yeah, the engineering teams, I mean- … I don’t, it’s not a competition, but I don’t know. If it was like an Olympics of engineering teams, I mean, TSMC does incredible engineering. Like I said, ASML at every scale, but NVIDIA is gonna give them a run for their money. Just incredible, incredible teams.
Jensen Huang
Well, it’s gold medalists in every single, in every single sport, all assembled right here.
Well, it’s gold medalists in every single, in every single sport, all assembled right here.
China
Lex Fridman
And have to work together. And report directly to you. This is wonderful. You recently traveled to China. So it’s interesting to ask you, China’s been incredibly successful in building up its technology sector. What do you understand about how China’s able to, over the past 10 years, build so many incredible world-class companies, world-class engineering teams, and just this technology ecosystem- … that produces so many incredible products?
And have to work together. And report directly to you. This is wonderful. You recently traveled to China. So it’s interesting to ask you, China’s been incredibly successful in building up its technology sector. What do you understand about how China’s able to, over the past 10 years, build so many incredible world-class companies, world-class engineering teams, and just this technology ecosystem- … that produces so many incredible products?
Jensen Huang
A whole bunch of reasons for… Well, first of all, let’s start, let’s start with some facts. 50% of the world’s AI researchers are Chinese, plus or minus, and they’re mostly in China still. We have many of them here, but there’s amazing researchers still in China. They—their tech industry showed up at precisely the right time. At the time of the mobile cloud era, their way of contributing was software, and so this is a country’s incredible science and math really well-educated kids. Their tech industry was created during the era of software. They’re very comfortable with modern software. China is not one giant economic country. It’s got many provinces and cities with mayors all competing with each other.
A whole bunch of reasons for… Well, first of all, let’s start, let’s start with some facts. 50% of the world’s AI researchers are Chinese, plus or minus, and they’re mostly in China still. We have many of them here, but there’s amazing researchers still in China. They—their tech industry showed up at precisely the right time. At the time of the mobile cloud era, their way of contributing was software, and so this is a country’s incredible science and math really well-educated kids. Their tech industry was created during the era of software. They’re very comfortable with modern software. China is not one giant economic country. It’s got many provinces and cities with mayors all competing with each other.
Jensen Huang
That’s the reason why there’s so many EV companies. That’s the reason why there’s so many AI companies. That’s the reason why there’s so many—every company you could imagine, they all create some of them. And, and as a result, they have insane competition internally. And, you know, what remains is an incredible company. They also have a social culture where, where it’s family first, friends second, and company third. And so the amount of conversation that goes back and forth between… They’re essentially open source all the time.
That’s the reason why there’s so many EV companies. That’s the reason why there’s so many AI companies. That’s the reason why there’s so many—every company you could imagine, they all create some of them. And, and as a result, they have insane competition internally. And, you know, what remains is an incredible company. They also have a social culture where, where it’s family first, friends second, and company third. And so the amount of conversation that goes back and forth between… They’re essentially open source all the time.
Jensen Huang
So the fact that they contribute more to open source is so sensible because they’re probably, “What are we protecting?” You know, my engineers, their brothers are in that company, their friends are in that company, and they’re all schoolmates. You know, the schoolmate concept. There’s a, you know, one schoolmate, you’re brother for life. And and so they, they, they share knowledge very, very quickly. And so there’s no sense keeping technology hidden. You might as well put it on open source. And so the open source community then amplifies, accelerates the, the innovation process. So you get this rapid, incredible great talent, rapid innovation because of open source and just, you know, the nature of friends, and, and insane competition.
So the fact that they contribute more to open source is so sensible because they’re probably, “What are we protecting?” You know, my engineers, their brothers are in that company, their friends are in that company, and they’re all schoolmates. You know, the schoolmate concept. There’s a, you know, one schoolmate, you’re brother for life. And and so they, they, they share knowledge very, very quickly. And so there’s no sense keeping technology hidden. You might as well put it on open source. And so the open source community then amplifies, accelerates the, the innovation process. So you get this rapid, incredible great talent, rapid innovation because of open source and just, you know, the nature of friends, and, and insane competition.
Jensen Huang
Among the company, what emerges is incredible stuff. And so this is the fastest innovating country in the world today, and this is something that has everything that, everything that I’ve just said is fundamental to just how the kids were grown, the fact that they have excellent education, the fact that they, parents want them to do well in school, the fact that they, their culture is that way. These are, you know, these are just the thing about their country, and they showed up at precisely the time when technology is going through that exponential.
Among the company, what emerges is incredible stuff. And so this is the fastest innovating country in the world today, and this is something that has everything that, everything that I’ve just said is fundamental to just how the kids were grown, the fact that they have excellent education, the fact that they, parents want them to do well in school, the fact that they, their culture is that way. These are, you know, these are just the thing about their country, and they showed up at precisely the time when technology is going through that exponential.
Lex Fridman
Plus culturally, it’s pretty cool to be an engineer. It connects to all the components that you’re mentioning…
Plus culturally, it’s pretty cool to be an engineer. It connects to all the components that you’re mentioning…
Jensen Huang
It’s a builder nation.
It’s a builder nation.
Lex Fridman
It’s a builder nation.
It’s a builder nation.
Jensen Huang
Yeah, it’s a builder nation. Our country’s leaders, incredible, but they’re mostly lawyers. Their country’s leaders—and because we’re, they’re trying to keep us safe, rule of law governing—their country was built out of poverty. And so most of their leaders are incredible engineers. Some of the brightest minds.
Yeah, it’s a builder nation. Our country’s leaders, incredible, but they’re mostly lawyers. Their country’s leaders—and because we’re, they’re trying to keep us safe, rule of law governing—their country was built out of poverty. And so most of their leaders are incredible engineers. Some of the brightest minds.
Lex Fridman
To take a small tangent, because you mentioned open source, I have to go to Perplexity here, who you have been a fan of a long time.
To take a small tangent, because you mentioned open source, I have to go to Perplexity here, who you have been a fan of a long time.
Jensen Huang
Love it, yeah.
Love it, yeah.
Lex Fridman
And thank you for releasing open source Nemotron 3 Super, which you can also use inside Perplexity to look stuff up. Now, which is 120 billion parameter open weight MoE model. What’s your vision with open source? So you mentioned China with, with DeepSeek and MiniMax, with all these companies really pushing forward the open source AI movement, and NVIDIA is really leading the way in close to state-of-the-art open source LLMs. What’s your vision there?
And thank you for releasing open source Nemotron 3 Super, which you can also use inside Perplexity to look stuff up. Now, which is 120 billion parameter open weight MoE model. What’s your vision with open source? So you mentioned China with, with DeepSeek and MiniMax, with all these companies really pushing forward the open source AI movement, and NVIDIA is really leading the way in close to state-of-the-art open source LLMs. What’s your vision there?
Jensen Huang
First, if we’re gonna be a great AI computing company, we have to understand how AI models are evolving.
First, if we’re gonna be a great AI computing company, we have to understand how AI models are evolving.
Jensen Huang
One of the things that I love about Nemotron 3 is it’s not just a pure transformer model, it’s transformer and SSMs. And we were early in developing the, the conditional GANs, which, that progressive GANs, which led step-by-step to diffusion. And so the fact that we’re doing basic research in model architecture and in different domains gives us visibility into, you know, what kind of computing systems would do a good job for future models. And so it is part of our extreme co-design strategy. Second, I think we rightfully recognize that on the one hand, we want world-class models as products, and they should be proprietary. On the other hand, we also want AI to diffuse into every industry and every country, every researcher, every student.
One of the things that I love about Nemotron 3 is it’s not just a pure transformer model, it’s transformer and SSMs. And we were early in developing the, the conditional GANs, which, that progressive GANs, which led step-by-step to diffusion. And so the fact that we’re doing basic research in model architecture and in different domains gives us visibility into, you know, what kind of computing systems would do a good job for future models. And so it is part of our extreme co-design strategy. Second, I think we rightfully recognize that on the one hand, we want world-class models as products, and they should be proprietary. On the other hand, we also want AI to diffuse into every industry and every country, every researcher, every student.
Jensen Huang
And if everything is proprietary, it’s hard to do research and it’s hard to innovate on top of, around, with. And so… Open source is fundamentally necessary for many industries to join the AI revolution. NVIDIA has the scale and we have the motives—not only skills, scale, and motivation—to build and continue to build these AI models for as long as we shall live. And so therefore, we ought to do that. We can open up, we can activate every industry, every researcher, you know, every country to be able to join the AI revolution. There’s the third reason, which is from that, to recognizing that AI is not just language. These AIs will likely use tools and models and sub-agents that were trained on other modalities of information.
And if everything is proprietary, it’s hard to do research and it’s hard to innovate on top of, around, with. And so… Open source is fundamentally necessary for many industries to join the AI revolution. NVIDIA has the scale and we have the motives—not only skills, scale, and motivation—to build and continue to build these AI models for as long as we shall live. And so therefore, we ought to do that. We can open up, we can activate every industry, every researcher, you know, every country to be able to join the AI revolution. There’s the third reason, which is from that, to recognizing that AI is not just language. These AIs will likely use tools and models and sub-agents that were trained on other modalities of information.
Jensen Huang
Maybe it’s biology or chemistry or you know, laws of physics, or you know, fluids and thermodynamics, and not all of it is in language structure. And so somebody has to go make sure that weather prediction, biology, AI, AI for biology, physical AI, all of that stuff stays, can be pushed to the limits and pushed to the frontier. We don’t build cars, but we wanna make sure every car company has access to great models. We don’t discover drugs, but I wanna make sure that Lilly has the world’s best biology AI systems, so that they can go use it for discovering drugs. And so these three fundamental reasons, both in recognizing that AI is not just language, that AI is really broad, that we wanna engage everybody into the world of AI, and then also co-design of AI.
Maybe it’s biology or chemistry or you know, laws of physics, or you know, fluids and thermodynamics, and not all of it is in language structure. And so somebody has to go make sure that weather prediction, biology, AI, AI for biology, physical AI, all of that stuff stays, can be pushed to the limits and pushed to the frontier. We don’t build cars, but we wanna make sure every car company has access to great models. We don’t discover drugs, but I wanna make sure that Lilly has the world’s best biology AI systems, so that they can go use it for discovering drugs. And so these three fundamental reasons, both in recognizing that AI is not just language, that AI is really broad, that we wanna engage everybody into the world of AI, and then also co-design of AI.
Lex Fridman
Well, I have to say, once again, thank you for open sourcing, really truly open sourcing Nemotron 3 and …
Well, I have to say, once again, thank you for open sourcing, really truly open sourcing Nemotron 3 and …
Jensen Huang
Yeah, I appreciate you were saying that. We open sourced the models, we open sourced the weights, we open sourced the data, we open sourced how we created it. Yeah, it’s pretty amazing.
Yeah, I appreciate you were saying that. We open sourced the models, we open sourced the weights, we open sourced the data, we open sourced how we created it. Yeah, it’s pretty amazing.
TSMC and Taiwan
Lex Fridman
It’s really incredible. You’re originally from Taiwan and have a close relationship with TSMC. So I have to ask TSMC I think also is a legendary company in terms of the engineering teams, in terms of the incredible engineering work that they do. What do you understand about TSMC culture and their approach that explains how they’re able to achieve this singular unmatched success in everything they’re doing with semiconductors?
It’s really incredible. You’re originally from Taiwan and have a close relationship with TSMC. So I have to ask TSMC I think also is a legendary company in terms of the engineering teams, in terms of the incredible engineering work that they do. What do you understand about TSMC culture and their approach that explains how they’re able to achieve this singular unmatched success in everything they’re doing with semiconductors?
Jensen Huang
You know, first of all, the deepest misunderstanding about TSMC is that their technology is all they have. That somehow they have a really great transistor, and if somebody shows up another transistor, game over. It’s the technology and, of course, you know, I don’t mean just the transistor, the metallization systems, the packaging, the 3D packaging, the silicon photonics, the, you know, all of the technology that they have. That technology is really what makes the company special. Their technology makes the company special.
You know, first of all, the deepest misunderstanding about TSMC is that their technology is all they have. That somehow they have a really great transistor, and if somebody shows up another transistor, game over. It’s the technology and, of course, you know, I don’t mean just the transistor, the metallization systems, the packaging, the 3D packaging, the silicon photonics, the, you know, all of the technology that they have. That technology is really what makes the company special. Their technology makes the company special.
Jensen Huang
But their ability to orchestrate the demands, the dynamic demands of hundreds of companies in the world as they’re moving up, shifting out, you know, increasing, decreasing, pushing out, pulling in, changing from customer to customer, wafer starting, wafer stopping, emergency wafer starts, you know, all of this dynamics of the world’s complexity as the world is shape-shifting all the time, and somehow they’re running a factory with high throughput, high yields, really great costs, excellent customer service. They take their promises seriously.
But their ability to orchestrate the demands, the dynamic demands of hundreds of companies in the world as they’re moving up, shifting out, you know, increasing, decreasing, pushing out, pulling in, changing from customer to customer, wafer starting, wafer stopping, emergency wafer starts, you know, all of this dynamics of the world’s complexity as the world is shape-shifting all the time, and somehow they’re running a factory with high throughput, high yields, really great costs, excellent customer service. They take their promises seriously.
Jensen Huang
They, when your wafer—because they know that they’re helping you run your company—when the wafers were promised to show up, the wafers show up, you know, so that you could run your company appropriately. And so their system, their manufacturing system is completely miraculous, I would say. Then the second thing is their culture. This culture is simultaneously technology focused on one hand, advancing technology; simultaneously customer service oriented on the other hand. A lot of companies are very customer service oriented, but they’re not very technology excellent. They’re not at the bleeding edge of technology.
They, when your wafer—because they know that they’re helping you run your company—when the wafers were promised to show up, the wafers show up, you know, so that you could run your company appropriately. And so their system, their manufacturing system is completely miraculous, I would say. Then the second thing is their culture. This culture is simultaneously technology focused on one hand, advancing technology; simultaneously customer service oriented on the other hand. A lot of companies are very customer service oriented, but they’re not very technology excellent. They’re not at the bleeding edge of technology.
Jensen Huang
There are a lot of companies who are tech, at the bleeding edge of technology, but they’re not the best customer service oriented company. And so it just depends on somehow they’ve, they’ve balanced these two and they’re world-class at both. And then probably the third thing is the technology that I most value in them that they created this, you know, this, this intangible called trust. I trust them to put my company on top of them. That’s a very big deal.
There are a lot of companies who are tech, at the bleeding edge of technology, but they’re not the best customer service oriented company. And so it just depends on somehow they’ve, they’ve balanced these two and they’re world-class at both. And then probably the third thing is the technology that I most value in them that they created this, you know, this, this intangible called trust. I trust them to put my company on top of them. That’s a very big deal.
Lex Fridman
When they trust, I mean, there’s a really close relationship there that you’ve established, and that trust is established based on many years of performance, but there’s human relationships involved there as well.
When they trust, I mean, there’s a really close relationship there that you’ve established, and that trust is established based on many years of performance, but there’s human relationships involved there as well.
Jensen Huang
Three decades, I don’t know how many tens, hundreds of billions of dollars of business we’ve done through them, and we don’t have a contract. That’s pretty great.
Three decades, I don’t know how many tens, hundreds of billions of dollars of business we’ve done through them, and we don’t have a contract. That’s pretty great.
Lex Fridman
Amazing. Okay, there’s this story … … That in 2013, the founders of TSMC, Morris Chang offered you the chance to become TSMC’s chief executive and you said you already had a job. Is this story true?
Amazing. Okay, there’s this story … … That in 2013, the founders of TSMC, Morris Chang offered you the chance to become TSMC’s chief executive and you said you already had a job. Is this story true?
Jensen Huang
Story is true. I didn’t, I didn’t dismiss it. But I was deeply honored and, and of course, I knew then as I know now, TSMC is one of the most consequential companies in history. And Morris is one of the highest regarded executives and business and personal friend that I’ve had in my life. And, for him to ask, I was humbled and really honored. But the work that I’m doing here is really important, and I’ve seen, you know, in my mind’s eye, what NVIDIA was going to be and what the impact that we could have. And it was really important work. And it’s my responsibility, you know, my sole responsibility to make this happen. And so I declined it, not because it wasn’t an incredible offer. It’s an unbelievable offer, but I simply couldn’t take it.
Story is true. I didn’t, I didn’t dismiss it. But I was deeply honored and, and of course, I knew then as I know now, TSMC is one of the most consequential companies in history. And Morris is one of the highest regarded executives and business and personal friend that I’ve had in my life. And, for him to ask, I was humbled and really honored. But the work that I’m doing here is really important, and I’ve seen, you know, in my mind’s eye, what NVIDIA was going to be and what the impact that we could have. And it was really important work. And it’s my responsibility, you know, my sole responsibility to make this happen. And so I declined it, not because it wasn’t an incredible offer. It’s an unbelievable offer, but I simply couldn’t take it.
Lex Fridman
I think NVIDIA, both NVIDIA and TSMC are two of the greatest companies in the history of human civilization. And running either one, I’m sure, is an incredibly complicated effort and takes… You have to truly be all in. Everybody at every scale, not just at the CEO level. Everybody is really truly all in-
I think NVIDIA, both NVIDIA and TSMC are two of the greatest companies in the history of human civilization. And running either one, I’m sure, is an incredibly complicated effort and takes… You have to truly be all in. Everybody at every scale, not just at the CEO level. Everybody is really truly all in-
Jensen Huang
Yeah. Yeah, no doubt.
Yeah. Yeah, no doubt.
Lex Fridman
… To, to accomplish this kind of complexity.
… To, to accomplish this kind of complexity.
Jensen Huang
So now I can help both companies.
So now I can help both companies.
NVIDIA’s moat
Lex Fridman
Exactly. So NVIDIA is now the most valuable company in the world. I have to ask, what is the NVIDIA’s biggest moat, as the folks in the tech sector say? The edge you have that protects you from the competition.
Exactly. So NVIDIA is now the most valuable company in the world. I have to ask, what is the NVIDIA’s biggest moat, as the folks in the tech sector say? The edge you have that protects you from the competition.
Jensen Huang
Our single most important property as a company is the install base of our computing platform. Our single most important thing today is the install base of CUDA. Now, the reason why 20 years ago, of course, there was no install base. But what makes… And if somebody came up with a GUDA or TUDA, it wouldn’t make any difference at all. And the reason for that is because it’s never been just about the technology. The technology, of course, was incredible, visionary. But it’s the fact that the company was dedicated to it, stuck with it, expanded its reach. It wasn’t three people that made CUDA successful. It was 43,000 people that made CUDA successful.
Our single most important property as a company is the install base of our computing platform. Our single most important thing today is the install base of CUDA. Now, the reason why 20 years ago, of course, there was no install base. But what makes… And if somebody came up with a GUDA or TUDA, it wouldn’t make any difference at all. And the reason for that is because it’s never been just about the technology. The technology, of course, was incredible, visionary. But it’s the fact that the company was dedicated to it, stuck with it, expanded its reach. It wasn’t three people that made CUDA successful. It was 43,000 people that made CUDA successful.
Jensen Huang
And the several million developers that believed in us that trusted that we were going to continue to make CUDA 1, 2, 3, 13, that they decided to port and dedicate their software on top of it, their mountain of software on top of it. And so the install base is the number one most important advantage. That install base, when you amplify it with the velocity of our execution at the scale that we’re talking about, no company in history had ever built systems of this complexity, period. And then to build it once a year is impossible. And that velocity combined with the install base, in the developer’s mind, you just go now, take the developer’s mind. From the developer’s perspective, if I support CUDA, tomorrow it’ll be 10 times better. I just have to wait six months on average.
And the several million developers that believed in us that trusted that we were going to continue to make CUDA 1, 2, 3, 13, that they decided to port and dedicate their software on top of it, their mountain of software on top of it. And so the install base is the number one most important advantage. That install base, when you amplify it with the velocity of our execution at the scale that we’re talking about, no company in history had ever built systems of this complexity, period. And then to build it once a year is impossible. And that velocity combined with the install base, in the developer’s mind, you just go now, take the developer’s mind. From the developer’s perspective, if I support CUDA, tomorrow it’ll be 10 times better. I just have to wait six months on average.
Jensen Huang
Not only that, if I develop it on CUDA, I reach a few hundred million people, computers. I’m in every cloud, I’m in every computer company, I’m in every single industry, I’m in every single country. So if I create an open source package and I put it on CUDA first, I get these both attributes simultaneously. And not only that, I trust 100% that NVIDIA is going to keep CUDA around and maintain it and improve it and keep optimizing the libraries for as long as they shall live. You could take that to the bank, and that last part, trust. You put all that stuff together, if I were a developer today, I would target CUDA first. I would target CUDA most. And that’s the reason that I think in the final analysis is our first, that’s even our first-
Not only that, if I develop it on CUDA, I reach a few hundred million people, computers. I’m in every cloud, I’m in every computer company, I’m in every single industry, I’m in every single country. So if I create an open source package and I put it on CUDA first, I get these both attributes simultaneously. And not only that, I trust 100% that NVIDIA is going to keep CUDA around and maintain it and improve it and keep optimizing the libraries for as long as they shall live. You could take that to the bank, and that last part, trust. You put all that stuff together, if I were a developer today, I would target CUDA first. I would target CUDA most. And that’s the reason that I think in the final analysis is our first, that’s even our first-
Jensen Huang
… core advantage. Our second one is our ecosystem. The fact that we vertically integrated this incredibly complex system, but we integrate it horizontally into every single company’s computers. We’re into Google Cloud, we’re into Amazon, we’re in Azure. You know, we’re ramping up AWS like crazy right now. We’re in new companies like CoreWeave and Nscale. We’re in supercomputers at Lilly. We’re in enterprise computers. We’re at the edge in radio base stations. You know, I mean, it’s just crazy. One architecture is in all these different systems. We’re in cars, we’re in robots, we’re in satellites, we’re out in space. And so the fact that you have this one architecture and the ecosystem is so broad, it basically covers every single industry in the world.
… core advantage. Our second one is our ecosystem. The fact that we vertically integrated this incredibly complex system, but we integrate it horizontally into every single company’s computers. We’re into Google Cloud, we’re into Amazon, we’re in Azure. You know, we’re ramping up AWS like crazy right now. We’re in new companies like CoreWeave and Nscale. We’re in supercomputers at Lilly. We’re in enterprise computers. We’re at the edge in radio base stations. You know, I mean, it’s just crazy. One architecture is in all these different systems. We’re in cars, we’re in robots, we’re in satellites, we’re out in space. And so the fact that you have this one architecture and the ecosystem is so broad, it basically covers every single industry in the world.
Lex Fridman
Well, how does the CUDA install base evolve into the future with AI factories as a moat? What do you… Do you think it’s possible that NVIDIA of the future is all about the AI factory?
Well, how does the CUDA install base evolve into the future with AI factories as a moat? What do you… Do you think it’s possible that NVIDIA of the future is all about the AI factory?
Jensen Huang
Well, the unit of computing used to be GPU to us. Then it became a computer, then it became a cluster. Now it’s an entire AI factory. When I see a computer, when I see what NVIDIA builds, in the old days, I would, you know, I visualize the chip. And then when I announced the new product, new generation, like, “Ladies and gentlemen, we’re announcing Ampere today,” I’d pick up the chip. That was my mental model- … of what I was building. Today, I wouldn’t… Picking up the chip is kind of still adorable.
Well, the unit of computing used to be GPU to us. Then it became a computer, then it became a cluster. Now it’s an entire AI factory. When I see a computer, when I see what NVIDIA builds, in the old days, I would, you know, I visualize the chip. And then when I announced the new product, new generation, like, “Ladies and gentlemen, we’re announcing Ampere today,” I’d pick up the chip. That was my mental model- … of what I was building. Today, I wouldn’t… Picking up the chip is kind of still adorable.
Jensen Huang
But it’s adorable. It’s not my mental model of what I’m doing. My mental model is this giant gigawatt thing that has power generations connected to the grid. It’s got cooling systems and networking of incredible monstrosity, you know. 10,000 people are in there trying to install it, hundreds of networking engineers in there, thousands of engineers behind it trying to power it up. You know, powering up one of those factories, as you know, it’s not somebody going, “It’s on now.” It takes thousands of people to bring it up.
But it’s adorable. It’s not my mental model of what I’m doing. My mental model is this giant gigawatt thing that has power generations connected to the grid. It’s got cooling systems and networking of incredible monstrosity, you know. 10,000 people are in there trying to install it, hundreds of networking engineers in there, thousands of engineers behind it trying to power it up. You know, powering up one of those factories, as you know, it’s not somebody going, “It’s on now.” It takes thousands of people to bring it up.
Lex Fridman
So mentally, you’re actually… When you’re thinking about a single unit of compute, you’re like literally, when you go to bed at night, you’re thinking now about a collection of racks, so pods, not individual chips.
So mentally, you’re actually… When you’re thinking about a single unit of compute, you’re like literally, when you go to bed at night, you’re thinking now about a collection of racks, so pods, not individual chips.
Jensen Huang
Entire infrastructure. And I’m hoping my next click is when I’m thinking about building computers, it’s planetary scale. That’ll be the next click.
Entire infrastructure. And I’m hoping my next click is when I’m thinking about building computers, it’s planetary scale. That’ll be the next click.
AI data centers in space
Lex Fridman
Well, what do you think about the space angle that Elon has talked about, doing compute in space for solving some of the… It makes some of the energy issues in terms of scaling energy easier.
Well, what do you think about the space angle that Elon has talked about, doing compute in space for solving some of the… It makes some of the energy issues in terms of scaling energy easier.
Jensen Huang
Cooling issues is not easy. Yeah.
Cooling issues is not easy. Yeah.
Lex Fridman
Cooling. Well, there’s a large number of engineering complexities involved with that. So what… You know, NVIDIA has also announced that you’re already thinking about that.
Cooling. Well, there’s a large number of engineering complexities involved with that. So what… You know, NVIDIA has also announced that you’re already thinking about that.
Jensen Huang
Yeah, we’re already there. NVIDIA GPUs are the first GPUs in space. And I didn’t realize it, it was so interesting to… I would have declared it maybe. We’re in space. You know, little, little astronaut suit on one of our GPUs. But we’ve been in space. It’s the right place to do a lot of imaging.
Yeah, we’re already there. NVIDIA GPUs are the first GPUs in space. And I didn’t realize it, it was so interesting to… I would have declared it maybe. We’re in space. You know, little, little astronaut suit on one of our GPUs. But we’ve been in space. It’s the right place to do a lot of imaging.
Jensen Huang
You know, because those satellites have really high resolution imaging systems, and they’re sweeping the Earth, you know, continuously now. And you want, you know, centimeter scale imaging that is done continuously for the world, so that, you know, you’ll basically have real time telemetry of everything. You don’t wanna beam that back down to Earth. It’s just, you know, petabytes and petabytes of data. You gotta just do AI right there at the edge, throw away everything you don’t need, you’ve seen before, didn’t change, and then just keep the stuff that you need. And so AI had to be done at the edge. Obviously we have 24/7 solar, if we put it at the polars. And but, you know, there’s no conduction, no convection.
You know, because those satellites have really high resolution imaging systems, and they’re sweeping the Earth, you know, continuously now. And you want, you know, centimeter scale imaging that is done continuously for the world, so that, you know, you’ll basically have real time telemetry of everything. You don’t wanna beam that back down to Earth. It’s just, you know, petabytes and petabytes of data. You gotta just do AI right there at the edge, throw away everything you don’t need, you’ve seen before, didn’t change, and then just keep the stuff that you need. And so AI had to be done at the edge. Obviously we have 24/7 solar, if we put it at the polars. And but, you know, there’s no conduction, no convection.
Jensen Huang
And so, you know, you’re pretty much just radiation. And but, you know, space is big. I guess, you know, we’re just gonna put big, giant radiators out there.
And so, you know, you’re pretty much just radiation. And but, you know, space is big. I guess, you know, we’re just gonna put big, giant radiators out there.
Lex Fridman
How crazy of an idea do you think it is? Like is this five years out, 10 years out, 20 years out? So we’re talking about blockers for AI scaling.
How crazy of an idea do you think it is? Like is this five years out, 10 years out, 20 years out? So we’re talking about blockers for AI scaling.
Jensen Huang
You know, I’m just so much more practical. I look for where my next, next bucket of opportunities are first. Meanwhile, I’m cultivating space. And so I send, I send engineers to go work on the problem. We’re starting to… We’re learning a lot about it. How do we deal with radiation? How do we deal with degrading performance? How do we deal with a continuous testing and attestation of defects? And you know, how do we deal with redundancy? And how do we degrade gracefully and things like that? And so we could do a… What about software? How do you think about software and redundancy and performance out in space?
You know, I’m just so much more practical. I look for where my next, next bucket of opportunities are first. Meanwhile, I’m cultivating space. And so I send, I send engineers to go work on the problem. We’re starting to… We’re learning a lot about it. How do we deal with radiation? How do we deal with degrading performance? How do we deal with a continuous testing and attestation of defects? And you know, how do we deal with redundancy? And how do we degrade gracefully and things like that? And so we could do a… What about software? How do you think about software and redundancy and performance out in space?
Jensen Huang
Make it so that the computer never breaks, it just gets slower, you know. And I… So we could start doing a lot of engineering exploration upfront. But in the meantime, my favorite answer is eliminate waste. You know, we’ve got all that idle power, I want to evacuate it as fast as possible.
Make it so that the computer never breaks, it just gets slower, you know. And I… So we could start doing a lot of engineering exploration upfront. But in the meantime, my favorite answer is eliminate waste. You know, we’ve got all that idle power, I want to evacuate it as fast as possible.
Lex Fridman
Yeah. There, there… Yeah, there’s a lot of low-hanging fruit here on Earth- … That we can utilize for the AI scaling. Quick pause. Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Perplexity for curiosity-driven knowledge exploration, Shopify for selling stuff online, LMNT for electrolytes, Fin for customer service AI agents, and Quo for a phone system, like calls, texts, contacts, for your business. Choose wisely, my friends. And now, back to my conversation with Jensen Huang. Do you think NVIDIA may be worth 10 trillion at some point? Let’s, let’s ask it this way. What does the future of the world look like where that’s true?
Yeah. There, there… Yeah, there’s a lot of low-hanging fruit here on Earth- … That we can utilize for the AI scaling. Quick pause. Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Perplexity for curiosity-driven knowledge exploration, Shopify for selling stuff online, LMNT for electrolytes, Fin for customer service AI agents, and Quo for a phone system, like calls, texts, contacts, for your business. Choose wisely, my friends. And now, back to my conversation with Jensen Huang. Do you think NVIDIA may be worth 10 trillion at some point? Let’s, let’s ask it this way. What does the future of the world look like where that’s true?
Will NVIDIA be worth $10 trillion?
Jensen Huang
I think that NVIDIA’s growth is extremely likely, and in my mind, inevitable. And let me explain why. We’re the largest computer company in history. That alone should beg the question, why? And the reason of course… Two reasons. First, two foundational technical reasons. The first reason is that computing went from being a retrieval-based, file retrieval system. Almost everything is a file… We pre-write something, we pre-record something. You know, we draw something, we put it on the web, we put it in a file. And we use a recommender system, some smart filter, to figure out what to retrieve for you. And so we were a pre-recording, human pre-recording, and file retrieving system. That’s what a computer is, largely.
I think that NVIDIA’s growth is extremely likely, and in my mind, inevitable. And let me explain why. We’re the largest computer company in history. That alone should beg the question, why? And the reason of course… Two reasons. First, two foundational technical reasons. The first reason is that computing went from being a retrieval-based, file retrieval system. Almost everything is a file… We pre-write something, we pre-record something. You know, we draw something, we put it on the web, we put it in a file. And we use a recommender system, some smart filter, to figure out what to retrieve for you. And so we were a pre-recording, human pre-recording, and file retrieving system. That’s what a computer is, largely.
Jensen Huang
To now, AI computers are contextually aware, which means that it has to process and generate tokens in real time. So we went from a retrieval-based computing system to a generative-based computing system. We’re gonna need a lot more processing in this new world than in the old world. We need a lot of storage in the old world. We need a lot of computation in this new world. And so that’s the first part of it. We fundamentally changed computing and the way how computing is done. The only thing that would cause it to go back……
To now, AI computers are contextually aware, which means that it has to process and generate tokens in real time. So we went from a retrieval-based computing system to a generative-based computing system. We’re gonna need a lot more processing in this new world than in the old world. We need a lot of storage in the old world. We need a lot of computation in this new world. And so that’s the first part of it. We fundamentally changed computing and the way how computing is done. The only thing that would cause it to go back……
Jensen Huang
is if this way of computation, this way of computing generating information that’s contextually relevant, situationally aware, that is grounded on new insight before it generates information, this computation-intensive way of doing computing would only go back if it’s not effective. So if… For the last 10, 15 years while working on deep learning, if at any single moment I would have come to the conclusion that, “You know what? This is not gonna work out. I think this is a dead end.” Or, “It’s not gonna scale, it’s not gonna solve this modality, not gonna be used in this application.” Then, of course, I would feel very differently about it, but I think the last five years has given me more confidence than the previous ten years.
is if this way of computation, this way of computing generating information that’s contextually relevant, situationally aware, that is grounded on new insight before it generates information, this computation-intensive way of doing computing would only go back if it’s not effective. So if… For the last 10, 15 years while working on deep learning, if at any single moment I would have come to the conclusion that, “You know what? This is not gonna work out. I think this is a dead end.” Or, “It’s not gonna scale, it’s not gonna solve this modality, not gonna be used in this application.” Then, of course, I would feel very differently about it, but I think the last five years has given me more confidence than the previous ten years.
Jensen Huang
The second idea is computers, because it was a storage system, it was largely a warehouse. We’re now building factories. Warehouses don’t make much money. Factories directly correlates with the company’s revenues. And so, the computer did two things. Not only did it change the way it did it, its purpose in the world changed. It’s no longer a computer, it’s a factory. It’s a factory, it’s used for generation of revenues. We’re now seeing not only is this factory generating products, commodities that people want to consume, we’re seeing that the commodities are so interesting, so valuable to so many different audiences that the tokens are starting to segment, like iPhones. You have free tokens, you have premium tokens, and you have several tokens in the middle.
The second idea is computers, because it was a storage system, it was largely a warehouse. We’re now building factories. Warehouses don’t make much money. Factories directly correlates with the company’s revenues. And so, the computer did two things. Not only did it change the way it did it, its purpose in the world changed. It’s no longer a computer, it’s a factory. It’s a factory, it’s used for generation of revenues. We’re now seeing not only is this factory generating products, commodities that people want to consume, we’re seeing that the commodities are so interesting, so valuable to so many different audiences that the tokens are starting to segment, like iPhones. You have free tokens, you have premium tokens, and you have several tokens in the middle.
Jensen Huang
And so intelligence, as it turns out, you know, it’s a scalable product. There’s extremely high intelligence products, tokens that you could… that are used for specialized things, people be willing to pay. You know, the idea that somebody’s willing to pay $1000 per million tokens is just around the corner. It’s not if, it’s only when. And so, so now we’re seeing that the commodity that this factory makes is actually valuable, and is revenue generating and profit generating. Now the question is how many of these factories does the world need? How many tokens does the world need? And how much is society willing to pay for these tokens? And what would happen to the world’s economy if the productivity were to improve so substantially? What would happen…
And so intelligence, as it turns out, you know, it’s a scalable product. There’s extremely high intelligence products, tokens that you could… that are used for specialized things, people be willing to pay. You know, the idea that somebody’s willing to pay $1000 per million tokens is just around the corner. It’s not if, it’s only when. And so, so now we’re seeing that the commodity that this factory makes is actually valuable, and is revenue generating and profit generating. Now the question is how many of these factories does the world need? How many tokens does the world need? And how much is society willing to pay for these tokens? And what would happen to the world’s economy if the productivity were to improve so substantially? What would happen…
Jensen Huang
Are we, are we gonna discover new drugs, new products, new services? And so when you take these things in combination, I am absolutely certain that the world’s GDP is going to accelerate in growth. I’m absolutely certain the percentage of that GDP that will be used for computation will be 100 times more than the past—mm-hmm—because it’s no longer a storage unit. It’s a product generation unit. And so when you look at it in that context and then you back into what is NVIDIA’s, what does NVIDIA sh—what does NVIDIA do and how much of that new economics, new industry would we have to benefit t—to address, I think we’re gonna be a lot, lot bigger.
Are we, are we gonna discover new drugs, new products, new services? And so when you take these things in combination, I am absolutely certain that the world’s GDP is going to accelerate in growth. I’m absolutely certain the percentage of that GDP that will be used for computation will be 100 times more than the past—mm-hmm—because it’s no longer a storage unit. It’s a product generation unit. And so when you look at it in that context and then you back into what is NVIDIA’s, what does NVIDIA sh—what does NVIDIA do and how much of that new economics, new industry would we have to benefit t—to address, I think we’re gonna be a lot, lot bigger.
Jensen Huang
And then the rest of it, to me, is: is it possible for NVIDIA to be a, you know, $3 trillion revenue company in the near future? The answer is, of course, yes. And the reason for that is because it’s not limited by any physical limits. There’s nothing that I see that says, you know, gosh $3 trillion is not possible. And as it turns out, NVIDIA’s supply chain is—the burden is shared by 200 companies. And the fact that we scale out on the backs of, with the partnership of this ecosystem, the question is: do we have the energy to do so? And surely we will have the energy to do so. And so all of these things combined, that number is just a number, you know?
And then the rest of it, to me, is: is it possible for NVIDIA to be a, you know, $3 trillion revenue company in the near future? The answer is, of course, yes. And the reason for that is because it’s not limited by any physical limits. There’s nothing that I see that says, you know, gosh $3 trillion is not possible. And as it turns out, NVIDIA’s supply chain is—the burden is shared by 200 companies. And the fact that we scale out on the backs of, with the partnership of this ecosystem, the question is: do we have the energy to do so? And surely we will have the energy to do so. And so all of these things combined, that number is just a number, you know?
Jensen Huang
And I still remember, NVIDIA was a… the first time we crossed a billion dollars, I was reminded of a CEO who told me, “You know, Jensen, it’s theoretically impossible for a fabless semiconductor company to exceed a billion dollars.” And I won’t bore you with why, but of course it’s illogical and there’s a lot of evidence we’re not. And then somebody told me, “You know, Jensen, you’ll never be more than $25 billion because of some other company.” Somebody told me that, “You’ll never be, you know, because…” And so those aren’t principled, first principled reason thinking. And the simple way to think about that is what is it that we make and how large is the opportunity that we can create?
And I still remember, NVIDIA was a… the first time we crossed a billion dollars, I was reminded of a CEO who told me, “You know, Jensen, it’s theoretically impossible for a fabless semiconductor company to exceed a billion dollars.” And I won’t bore you with why, but of course it’s illogical and there’s a lot of evidence we’re not. And then somebody told me, “You know, Jensen, you’ll never be more than $25 billion because of some other company.” Somebody told me that, “You’ll never be, you know, because…” And so those aren’t principled, first principled reason thinking. And the simple way to think about that is what is it that we make and how large is the opportunity that we can create?
Jensen Huang
Now, NVIDIA is not in the market share business. Almost everything that I just talked about don’t exist. That’s the part that’s hard. You know, if NVIDIA was a $10 billion company trying to take NVIDIA’s share, then it’s easy to see for shareholders that, oh, yeah, if they could just take 10% share, they could be this much larger. But it’s hard for people to imagine how large we could be because there’s nobody I could take share from. You know? And so I think that that’s one of the challenges for the world is the imagination of the future. But I got plenty of time, and I’ll keep reasoning about it, and I’ll keep talking about it, and every single GTC will become more and more real.
Now, NVIDIA is not in the market share business. Almost everything that I just talked about don’t exist. That’s the part that’s hard. You know, if NVIDIA was a $10 billion company trying to take NVIDIA’s share, then it’s easy to see for shareholders that, oh, yeah, if they could just take 10% share, they could be this much larger. But it’s hard for people to imagine how large we could be because there’s nobody I could take share from. You know? And so I think that that’s one of the challenges for the world is the imagination of the future. But I got plenty of time, and I’ll keep reasoning about it, and I’ll keep talking about it, and every single GTC will become more and more real.
Jensen Huang
You know, and then more and more people will talk about it, and one of these days, you know, we’ll get there. But I’m 100% we’ll get there.
You know, and then more and more people will talk about it, and one of these days, you know, we’ll get there. But I’m 100% we’ll get there.
Lex Fridman
Yeah, this view of you know, token factories essentially, this token per second per watt, and every token having value. Like it’s an actual thing that brings value, and it brings different kinds of value, different amounts of value to different people with value. That’s the actual product—it really could be loosely thought of as the token. And so you have a bunch of token factories. And then it’s very easy, first principles, to imagine a future, given all the potential things that AI can solve, that you’re going to need an exponential number more of token factories.
Yeah, this view of you know, token factories essentially, this token per second per watt, and every token having value. Like it’s an actual thing that brings value, and it brings different kinds of value, different amounts of value to different people with value. That’s the actual product—it really could be loosely thought of as the token. And so you have a bunch of token factories. And then it’s very easy, first principles, to imagine a future, given all the potential things that AI can solve, that you’re going to need an exponential number more of token factories.
Jensen Huang
Yeah. And what’s really interesting, the reason why I was so excited about it, the iPhone of tokens arrived.
Yeah. And what’s really interesting, the reason why I was so excited about it, the iPhone of tokens arrived.
Lex Fridman
What do you call it? Wait, are you saying OpenClaw’s iPhone?
What do you call it? Wait, are you saying OpenClaw’s iPhone?
Jensen Huang
Yeah.
Yeah.
Lex Fridman
That’s interesting.
That’s interesting.
Jensen Huang
Agents.
Agents.
Lex Fridman
Yeah, agents. True.
Yeah, agents. True.
Jensen Huang
Agents in general. The iPhone of tokens arrived. It is the fastest-growing application in history. It went straight up. Went straight up.
Agents in general. The iPhone of tokens arrived. It is the fastest-growing application in history. It went straight up. Went straight up.
Lex Fridman
That says something.
That says something.
Jensen Huang
Yep, there’s no question OpenClaw is the iPhone of tokens.
Yep, there’s no question OpenClaw is the iPhone of tokens.
Lex Fridman
Is there something truly, as you know, something truly special happening from about December, where people have really woke up to the power of Claude Code of Codex, of OpenClaw? I mean, I’m embarrassed to admit that on the way here in the airport, I’ve… It’s the first time I’ve done this in public. I was programming, quote unquote, by talking to my laptop.
Is there something truly, as you know, something truly special happening from about December, where people have really woke up to the power of Claude Code of Codex, of OpenClaw? I mean, I’m embarrassed to admit that on the way here in the airport, I’ve… It’s the first time I’ve done this in public. I was programming, quote unquote, by talking to my laptop.
Jensen Huang
Yeah, exactly.
Yeah, exactly.
Lex Fridman
And I was embarrassed because I was pretending like I’m talking to a human colleague. I’m not sure how I feel about the future where everybody- … is walking around talking to their AI, but it’s such an efficient way to get stuff done.
And I was embarrassed because I was pretending like I’m talking to a human colleague. I’m not sure how I feel about the future where everybody- … is walking around talking to their AI, but it’s such an efficient way to get stuff done.
Jensen Huang
And it’s more likely that your AI is bothering you all the time. And the reason for that is because it’s getting stuff done so fast. It’s reporting back to you, “I got that done.” “You know, what do you want me to do next?” You know, it… That’s the part that I think most people don’t realize is the person who’s gonna be chatting with them, texting them most, is their, is their claws or lobster.
And it’s more likely that your AI is bothering you all the time. And the reason for that is because it’s getting stuff done so fast. It’s reporting back to you, “I got that done.” “You know, what do you want me to do next?” You know, it… That’s the part that I think most people don’t realize is the person who’s gonna be chatting with them, texting them most, is their, is their claws or lobster.
Leadership under pressure
Lex Fridman
What an incredible future. I read that you attribute a lot of your success to your ability to work harder than anyone and withstand more suffering than anyone. So we can list many of the things that entails. I mean, dealing with failure, the cost and engineering problems we’ve talked about. The human problems, uncertainty, responsibility, exhaustion, embarrassment, the near-death company moments that you’ve mentioned but also the pressure. Now, as the CEO of this company that economies and nations strategize around, plan their financial allocations around, plan their AI infrastructure around, how do you deal with this much pressure? What gives you strength, given how many nations and peoples depend on you?
What an incredible future. I read that you attribute a lot of your success to your ability to work harder than anyone and withstand more suffering than anyone. So we can list many of the things that entails. I mean, dealing with failure, the cost and engineering problems we’ve talked about. The human problems, uncertainty, responsibility, exhaustion, embarrassment, the near-death company moments that you’ve mentioned but also the pressure. Now, as the CEO of this company that economies and nations strategize around, plan their financial allocations around, plan their AI infrastructure around, how do you deal with this much pressure? What gives you strength, given how many nations and peoples depend on you?
Jensen Huang
I’m conscious about the fact that NVIDIA’s success is very important to the United States. We generate enormous amounts of tax revenues. We established technology leadership for our nation. Technology leadership is important for national security. National security not just in one aspect of national security, all aspects of national security. When our country’s more prosperous, we could do a better job with domestic policies and helping social benefits. Because we’re generating so much re-industrialization in the United States, we’re creating mountains of jobs. We’re helping shift how we build things back to the United States in so many different plants, chips, computers, and of course, these AI factories. I’m completely aware that, that…
I’m conscious about the fact that NVIDIA’s success is very important to the United States. We generate enormous amounts of tax revenues. We established technology leadership for our nation. Technology leadership is important for national security. National security not just in one aspect of national security, all aspects of national security. When our country’s more prosperous, we could do a better job with domestic policies and helping social benefits. Because we’re generating so much re-industrialization in the United States, we’re creating mountains of jobs. We’re helping shift how we build things back to the United States in so many different plants, chips, computers, and of course, these AI factories. I’m completely aware that, that…
Jensen Huang
And I have the benefit, and this is a real gift with mainstream investors, teachers, policemen who have somehow, for whatever reason, invested in NVIDIA or because they watched Jim Cramer, bought some stock and now are millionaires.
And I have the benefit, and this is a real gift with mainstream investors, teachers, policemen who have somehow, for whatever reason, invested in NVIDIA or because they watched Jim Cramer, bought some stock and now are millionaires.
Jensen Huang
And I am completely aware of that circumstance. I’m aware of the circumstance that NVIDIA is central to a very large network of ecosystem partners behind us and downstream from us. And so the way I deal with that is exactly what I just did. I reason about what is… what is it that we’re doing? What is it causing? What’s the impact that has on other people benefit, you know, positively or even through great burden, for example, to supply chain? And the question is therefore, what are you gonna do about it? In almost everything that I feel, I break it down, I reason about, “Okay, what’s the circumstance? What has changed? What’s hard? And what am I gonna do about it?” And I’m…
And I am completely aware of that circumstance. I’m aware of the circumstance that NVIDIA is central to a very large network of ecosystem partners behind us and downstream from us. And so the way I deal with that is exactly what I just did. I reason about what is… what is it that we’re doing? What is it causing? What’s the impact that has on other people benefit, you know, positively or even through great burden, for example, to supply chain? And the question is therefore, what are you gonna do about it? In almost everything that I feel, I break it down, I reason about, “Okay, what’s the circumstance? What has changed? What’s hard? And what am I gonna do about it?” And I’m…
Jensen Huang
I break it down, decompose the problem, and the decomposition of these circumstances turns it into manageable things that I can do. And the only thing that after that I could do is, “Did you do it? Did you either do it or did you get somebody else to do it? And if you didn’t do it, you reasoned that you need to do it, and you didn’t do it, and you didn’t get anybody else to do it, then stop crying about it.”… you know? And so, and so-
I break it down, decompose the problem, and the decomposition of these circumstances turns it into manageable things that I can do. And the only thing that after that I could do is, “Did you do it? Did you either do it or did you get somebody else to do it? And if you didn’t do it, you reasoned that you need to do it, and you didn’t do it, and you didn’t get anybody else to do it, then stop crying about it.”… you know? And so, and so-
Jensen Huang
so I’m fairly tough on myself. And, but I also break things down so that I don’t panic. I can go to sleep because I’ve made the list of things that needed to be done, and I’ve made sure that everything that could put our company in harm’s way, could put my partners in harm’s way, put our industry in harm’s way, I’ve told somebody. Everything that I feel could put anybody in harm’s way, I’ve told someone. And I’ve told that someone who could do something about it. And so I’ve gotten it off my chest or I’m doing something about it. And so after that, Lex, what else can you do?
so I’m fairly tough on myself. And, but I also break things down so that I don’t panic. I can go to sleep because I’ve made the list of things that needed to be done, and I’ve made sure that everything that could put our company in harm’s way, could put my partners in harm’s way, put our industry in harm’s way, I’ve told somebody. Everything that I feel could put anybody in harm’s way, I’ve told someone. And I’ve told that someone who could do something about it. And so I’ve gotten it off my chest or I’m doing something about it. And so after that, Lex, what else can you do?
Lex Fridman
So given all the insane, intense amount of suffering on the journey of building up NVIDIA, have you hit low points psychologically?
So given all the insane, intense amount of suffering on the journey of building up NVIDIA, have you hit low points psychologically?
Jensen Huang
Oh, yeah. Oh, yeah. Sure. All the time. All the time.
Oh, yeah. Oh, yeah. Sure. All the time. All the time.
Lex Fridman
And there-
And there-
Jensen Huang
All the time
All the time
Lex Fridman
… you just break down the problem into pieces? See what you could do about it?
… you just break down the problem into pieces? See what you could do about it?
Jensen Huang
And part of it, Lex, part of it is forgetting. One of the most important attributes of AI learning, as you know, is, right? Systematic forgetting. You need to know when to forget some things. You can’t memorize everything. You can’t keep everything and, you know, you don’t want to carry everything. One of the things that I do very quickly is decompose the problem, I reason about the problem, and I share the load with it. When I say I tell everybody, I’m essentially sharing that burden.
And part of it, Lex, part of it is forgetting. One of the most important attributes of AI learning, as you know, is, right? Systematic forgetting. You need to know when to forget some things. You can’t memorize everything. You can’t keep everything and, you know, you don’t want to carry everything. One of the things that I do very quickly is decompose the problem, I reason about the problem, and I share the load with it. When I say I tell everybody, I’m essentially sharing that burden.
Jensen Huang
As quickly as possible. Whatever worries me, tell somebody else. Don’t just keep it. You know, don’t freak them out. Decompose the problem into smaller parts and get people to, and inspire them to be able to go do something about it. But part of it is just forgetting. You know, like, a lot of it is you gotta be tough on yourself. You know, just come on, stop crying about it. Let’s get going. You know? And then you get out of bed. And then the other part is you’re attracted to the next shiny light, the next future, the next opportunity, the next, “Okay, that’s behind us. What’s next?” It’s a lot, I think, you know, you watch this with great athletes. They just worry about the next point. The last point is behind them. The embarrassment, the, you know- … the setback.
As quickly as possible. Whatever worries me, tell somebody else. Don’t just keep it. You know, don’t freak them out. Decompose the problem into smaller parts and get people to, and inspire them to be able to go do something about it. But part of it is just forgetting. You know, like, a lot of it is you gotta be tough on yourself. You know, just come on, stop crying about it. Let’s get going. You know? And then you get out of bed. And then the other part is you’re attracted to the next shiny light, the next future, the next opportunity, the next, “Okay, that’s behind us. What’s next?” It’s a lot, I think, you know, you watch this with great athletes. They just worry about the next point. The last point is behind them. The embarrassment, the, you know- … the setback.
Jensen Huang
You know, and because I do so much of my job publicly, you know? Lex, you do a fair amount of your job publicly too. And so I do a lot of my job publicly. And so you know, I say a lot of things that seem sensible at the time or funny at the time, mostly it’s just because it’s funny to me at the time. And then, you know, you reflect on it, it’s less funny, but…
You know, and because I do so much of my job publicly, you know? Lex, you do a fair amount of your job publicly too. And so I do a lot of my job publicly. And so you know, I say a lot of things that seem sensible at the time or funny at the time, mostly it’s just because it’s funny to me at the time. And then, you know, you reflect on it, it’s less funny, but…
Lex Fridman
Yeah. No, trust me, I know. But you basically allow yourself to be pulled by the light of the future. Forget the past and just keep-
Yeah. No, trust me, I know. But you basically allow yourself to be pulled by the light of the future. Forget the past and just keep-
Jensen Huang
That’s right.
That’s right.
Lex Fridman
… keep working towards that. I mean, you did say, there’s this kind of famous thing you said that if you knew how hard it would be to build NVIDIA it turned out to be—what is it? A million times more hard than you anticipated—that you wouldn’t do it.
… keep working towards that. I mean, you did say, there’s this kind of famous thing you said that if you knew how hard it would be to build NVIDIA it turned out to be—what is it? A million times more hard than you anticipated—that you wouldn’t do it.
Jensen Huang
Yeah, right.
Yeah, right.
Lex Fridman
But isn’t… You know, when I hear that, that’s probably true about everything worth doing, right?
But isn’t… You know, when I hear that, that’s probably true about everything worth doing, right?
Jensen Huang
Exactly. That is, by the way, what I was trying to explain, is that there’s an incredible superpower of having the mind of a child. You know? And I say to myself oftentimes when I look at something, and almost everything my first thought is, “How hard can it be?” You know? And so you get yourself into that mode, how hard could it be? And nobody’s ever done it. It looks gigantic. It’s gonna cost hundreds of billions of dollars. It’s gonna take, you know, all this… And you just go, “Yeah, but how hard could it be?” You know? How hard could it be?
Exactly. That is, by the way, what I was trying to explain, is that there’s an incredible superpower of having the mind of a child. You know? And I say to myself oftentimes when I look at something, and almost everything my first thought is, “How hard can it be?” You know? And so you get yourself into that mode, how hard could it be? And nobody’s ever done it. It looks gigantic. It’s gonna cost hundreds of billions of dollars. It’s gonna take, you know, all this… And you just go, “Yeah, but how hard could it be?” You know? How hard could it be?
Jensen Huang
And so, you gotta get yourself into that state of mind. You don’t wanna actually over-simulate everything and all the setbacks and all the trials and tribulations and all the disappointments. You don’t wanna simulate all that in advance. You don’t wanna know that. You wanna go into a new experience thinking it’s gonna be perfect, it’s gonna be great, it’s gonna be incredibly fun. And then while you’re there, you know, you need to have endurance, you need to have grit, so that when the setbacks actually happen, and those setbacks are gonna surprise you, the disappointments are gonna surprise you, the embarrassments are gonna surprise you, the humiliations are gonna surprise you.
And so, you gotta get yourself into that state of mind. You don’t wanna actually over-simulate everything and all the setbacks and all the trials and tribulations and all the disappointments. You don’t wanna simulate all that in advance. You don’t wanna know that. You wanna go into a new experience thinking it’s gonna be perfect, it’s gonna be great, it’s gonna be incredibly fun. And then while you’re there, you know, you need to have endurance, you need to have grit, so that when the setbacks actually happen, and those setbacks are gonna surprise you, the disappointments are gonna surprise you, the embarrassments are gonna surprise you, the humiliations are gonna surprise you.
Jensen Huang
You just can’t let… Now you just gotta turn on the other bit, which is just forget about it. Move on, keep moving. And to the extent that my assumptions about the future and why the future is gonna manifest, so long as those assumptions and that input doesn’t change or didn’t change materially, then I should expect that the output won’t change. And so my simulated output of the future is still gonna happen. And if it’s still gonna happen, I’m still gonna go after it.
You just can’t let… Now you just gotta turn on the other bit, which is just forget about it. Move on, keep moving. And to the extent that my assumptions about the future and why the future is gonna manifest, so long as those assumptions and that input doesn’t change or didn’t change materially, then I should expect that the output won’t change. And so my simulated output of the future is still gonna happen. And if it’s still gonna happen, I’m still gonna go after it.
Jensen Huang
I believe it’s gonna, you know, and so there’s a combination of two or three human characteristics: the ability to go into an experience fresh-minded, the ability to forget the setbacks, the ability to believe in yourself, you know, to believe what you believe and stay true to that belief. But you’re constantly reevaluating.
I believe it’s gonna, you know, and so there’s a combination of two or three human characteristics: the ability to go into an experience fresh-minded, the ability to forget the setbacks, the ability to believe in yourself, you know, to believe what you believe and stay true to that belief. But you’re constantly reevaluating.
Jensen Huang
This combination of three, four, five things I think is really important for resilience. And, you know, I’m fortunate that whatever life experiences led to this, I’ve got kind of those four, five things. You know, I’m always curious, always learning. I’m always learning from everybody, you know? I’m always asking my… And because I’m humble about everything, I’m always thinking, “Gosh, they did that so nicely. They did that so wonderfully.” You know, I wonder what they’re thinking through. How do they… So I’m simulating everybody. In a lot of ways, you know, I’m emulating almost everybody I watch, right? You’re empathetic towards everything that they do that you’re observing and respect. And so you’re constantly learning and, you know.
This combination of three, four, five things I think is really important for resilience. And, you know, I’m fortunate that whatever life experiences led to this, I’ve got kind of those four, five things. You know, I’m always curious, always learning. I’m always learning from everybody, you know? I’m always asking my… And because I’m humble about everything, I’m always thinking, “Gosh, they did that so nicely. They did that so wonderfully.” You know, I wonder what they’re thinking through. How do they… So I’m simulating everybody. In a lot of ways, you know, I’m emulating almost everybody I watch, right? You’re empathetic towards everything that they do that you’re observing and respect. And so you’re constantly learning and, you know.
Lex Fridman
You’re now one of the wealthiest people on Earth. One of the most successful humans on Earth. Is it harder to be humble and to be able to… Do you feel the effect of money and power and fame in making it harder for you to sort of be wrong in your own head? Enough to hear out an opinion of somebody else when they disagree with you and learn from them? Those kinds of things.
You’re now one of the wealthiest people on Earth. One of the most successful humans on Earth. Is it harder to be humble and to be able to… Do you feel the effect of money and power and fame in making it harder for you to sort of be wrong in your own head? Enough to hear out an opinion of somebody else when they disagree with you and learn from them? Those kinds of things.
Jensen Huang
Surprisingly, no. And I would actually go the other way. Because I do so much of my work publicly, when I’m wrong, pretty much everybody sees it.
Surprisingly, no. And I would actually go the other way. Because I do so much of my work publicly, when I’m wrong, pretty much everybody sees it.
Lex Fridman
You get humbled. Fair enough.
You get humbled. Fair enough.
Jensen Huang
And when I’m wrong—when I’m wrong or it didn’t turn out that way or, you know, I mean, most of the things that I say outside I’m fairly certain about. And the reason for that is because it’s gonna impact somebody else and I want to be quite concerned about that and quite circumspect about that. For stuff that I’m reasoning about inside a meeting, you know, a lot of things could turn out differently. And so, but it doesn’t ever stop me from reasoning. The way that I manage and lead, I’m constantly reasoning in front of people. And even when I’m talking to you, you can kind of see me reasoning through things. And I want to make sure that you understand what I’m saying not because I told you-
And when I’m wrong—when I’m wrong or it didn’t turn out that way or, you know, I mean, most of the things that I say outside I’m fairly certain about. And the reason for that is because it’s gonna impact somebody else and I want to be quite concerned about that and quite circumspect about that. For stuff that I’m reasoning about inside a meeting, you know, a lot of things could turn out differently. And so, but it doesn’t ever stop me from reasoning. The way that I manage and lead, I’m constantly reasoning in front of people. And even when I’m talking to you, you can kind of see me reasoning through things. And I want to make sure that you understand what I’m saying not because I told you-
Jensen Huang
… because I’m so humble about what I’m about to tell you. I kind of show you the steps that I got there. And then you can decide whether you believe what I said in the end. And so I’m doing that all day long in meetings. With all of my employees, I’m constantly reasoning through, “Let me tell you how I see it.” And then I reason through it. It gives everybody the opportunity to intercept and say, “I disagree with that part.” The nice thing about reasoning through things and letting people interact with it is that they don’t have to disagree with your outcome. They can disagree with your reasoning steps. And they could pull me in different directions, and then we can reason forward. And so we’re kind of, you know, a collective path searching method. And it’s really fantastic.
… because I’m so humble about what I’m about to tell you. I kind of show you the steps that I got there. And then you can decide whether you believe what I said in the end. And so I’m doing that all day long in meetings. With all of my employees, I’m constantly reasoning through, “Let me tell you how I see it.” And then I reason through it. It gives everybody the opportunity to intercept and say, “I disagree with that part.” The nice thing about reasoning through things and letting people interact with it is that they don’t have to disagree with your outcome. They can disagree with your reasoning steps. And they could pull me in different directions, and then we can reason forward. And so we’re kind of, you know, a collective path searching method. And it’s really fantastic.
Lex Fridman
Yeah, you have this way about you of … When you’re explaining stuff, I can feel you actually reasoning on the spot about it with a constant open-mindedness where you could … I could feel like I could steer your thinking. And that’s a—that’s really beautiful that you’ve been able to maintain that after so many years of success, and pain. I think sometimes pain closes you down a bit. And I think to maintain-
Yeah, you have this way about you of … When you’re explaining stuff, I can feel you actually reasoning on the spot about it with a constant open-mindedness where you could … I could feel like I could steer your thinking. And that’s a—that’s really beautiful that you’ve been able to maintain that after so many years of success, and pain. I think sometimes pain closes you down a bit. And I think to maintain-
Jensen Huang
Yeah. Tolerance for embarrassment, I think is…
Yeah. Tolerance for embarrassment, I think is…
Lex Fridman
Yes, that’s… The tolerance… I mean, that’s a real thing. Is many years of embarrassing yourself. Even those meetings knowing that there’s people around you where you declared one idea and it was shown that that idea was wrong- … and be able to admit that and to grow from that. That’s not—that’s very difficult on a human level.
Yes, that’s… The tolerance… I mean, that’s a real thing. Is many years of embarrassing yourself. Even those meetings knowing that there’s people around you where you declared one idea and it was shown that that idea was wrong- … and be able to admit that and to grow from that. That’s not—that’s very difficult on a human level.
Jensen Huang
Yeah. Well, you know. They knew I was—they knew that recently my first job was cleaning toilets, so.
Yeah. Well, you know. They knew I was—they knew that recently my first job was cleaning toilets, so.
Video games
Lex Fridman
I’m glad you maintained that same spirit of Denny’s, the work. I mean, that was beautiful. Your whole journey starting from Denny’s is a beautiful one. Let me ask you about video games. So I’m a big gaming fan. So I have to say thank you to NVIDIA for many years of incredible graphics.
I’m glad you maintained that same spirit of Denny’s, the work. I mean, that was beautiful. Your whole journey starting from Denny’s is a beautiful one. Let me ask you about video games. So I’m a big gaming fan. So I have to say thank you to NVIDIA for many years of incredible graphics.
Jensen Huang
By the way, GeForce is our still, to this day- … our number one marketing strategy. Right. People learn about NVIDIA while they’re in their teenage years. And then they go to college and they know who NVIDIA is and in the beginning it’s just, you know, playing Call of Duty, Fortnite. And then later they’re using CUDA, and then later they’re using NVIDIA and, you know, Blender and Dassault and Autodesk.
By the way, GeForce is our still, to this day- … our number one marketing strategy. Right. People learn about NVIDIA while they’re in their teenage years. And then they go to college and they know who NVIDIA is and in the beginning it’s just, you know, playing Call of Duty, Fortnite. And then later they’re using CUDA, and then later they’re using NVIDIA and, you know, Blender and Dassault and Autodesk.
Lex Fridman
Yeah. I mean, I should say I mentioned to a friend that I’m talking with you. He said, “Oh, they make great gaming GPUs.”
Yeah. I mean, I should say I mentioned to a friend that I’m talking with you. He said, “Oh, they make great gaming GPUs.”
Jensen Huang
Yeah, exactly.
Yeah, exactly.
Lex Fridman
It’s like-
It’s like-
Jensen Huang
Exactly.
Exactly.
Lex Fridman
You know, there’s more to it, but, yeah, people really love it. It really brought a lot of joy to a lot of people. The hardware really brings these worlds to life. There was some controversy around this with DLSS 5. Can you explain to me the drama around this? I guess people, the gamers online were concerned that it makes games look like AI slop. What do you think of this drama?
You know, there’s more to it, but, yeah, people really love it. It really brought a lot of joy to a lot of people. The hardware really brings these worlds to life. There was some controversy around this with DLSS 5. Can you explain to me the drama around this? I guess people, the gamers online were concerned that it makes games look like AI slop. What do you think of this drama?
Jensen Huang
Yeah. I think their perspective makes sense and I could see where they’re coming from, because I don’t love AI slop myself. You know, all of the AI-generated content increasingly looks similar and they’re all beautiful, and so I’m empathetic towards what they’re thinking. That’s just not what DLSS 5 is trying to do. I showed several examples of it. But DLSS 5 is 3D-conditioned, 3D-guided. It’s ground truth structure data guided. And so the artist determined the geometry. We are completely truthful to the geometry maintained in every single frame. It’s conditioned by the textures, the artistry of the artist. And so every single frame, it enhances but it doesn’t change anything.
Yeah. I think their perspective makes sense and I could see where they’re coming from, because I don’t love AI slop myself. You know, all of the AI-generated content increasingly looks similar and they’re all beautiful, and so I’m empathetic towards what they’re thinking. That’s just not what DLSS 5 is trying to do. I showed several examples of it. But DLSS 5 is 3D-conditioned, 3D-guided. It’s ground truth structure data guided. And so the artist determined the geometry. We are completely truthful to the geometry maintained in every single frame. It’s conditioned by the textures, the artistry of the artist. And so every single frame, it enhances but it doesn’t change anything.
Jensen Huang
Now, the question is about enhancing. DLSS 5 also lets, because the system is open, you could train your own models to determine, and you could even in the future prompt it. You know, I want it to be a toon shader. I want it to look like this kind of, so you can give it even an example. And it would generate in the style of that, all consistent with the artistry, the style, the intent of the artist. And so all of that is done for the artist, so that they can create something that is more beautiful but still in the style that they want. I think that they got the impression that the games are gonna come out the way the games are, shipped the way they do, and then we’re gonna post-process it. That’s not what DLSS is intended to do.
Now, the question is about enhancing. DLSS 5 also lets, because the system is open, you could train your own models to determine, and you could even in the future prompt it. You know, I want it to be a toon shader. I want it to look like this kind of, so you can give it even an example. And it would generate in the style of that, all consistent with the artistry, the style, the intent of the artist. And so all of that is done for the artist, so that they can create something that is more beautiful but still in the style that they want. I think that they got the impression that the games are gonna come out the way the games are, shipped the way they do, and then we’re gonna post-process it. That’s not what DLSS is intended to do.
Jensen Huang
DLSS is integrated with the artist, and so it’s about giving the artist the tool of AI, the tool of generative AI. They could decide not to use it, you know?
DLSS is integrated with the artist, and so it’s about giving the artist the tool of AI, the tool of generative AI. They could decide not to use it, you know?
Lex Fridman
I think people are very sensitive to human faces. And we’re now living in this moment, which I think is a, is a beautiful one, which is people are sensitive to AI slop. It puts a mirror to ourselves to help us realize that what we seek is imperfections. What we seek is sometimes not perfect graphics. It helps us understand what we find compelling in the worlds we create. And that’s beautiful. And as long as it’s tools that help us create those worlds-
I think people are very sensitive to human faces. And we’re now living in this moment, which I think is a, is a beautiful one, which is people are sensitive to AI slop. It puts a mirror to ourselves to help us realize that what we seek is imperfections. What we seek is sometimes not perfect graphics. It helps us understand what we find compelling in the worlds we create. And that’s beautiful. And as long as it’s tools that help us create those worlds-
Jensen Huang
Yeah, that’s right.
Yeah, that’s right.
Lex Fridman
… it’s wonderful.
… it’s wonderful.
Jensen Huang
That’s right. Yet, yet another tool, and they want the generative models to generate the opposite of photo real. Yeah, it’ll do that too. And so it’s just yet another tool. I think the gamers might also appreciate that in the last couple of years, we introduced skin shaders to the game developers. And many of those games have skin shaders that include subsurface scattering that make skin look more skin-like. And so the industries, you know, game developers are looking for more and more tools to express their art. And so this is just yet one more tool, and they get to decide what to use.
That’s right. Yet, yet another tool, and they want the generative models to generate the opposite of photo real. Yeah, it’ll do that too. And so it’s just yet another tool. I think the gamers might also appreciate that in the last couple of years, we introduced skin shaders to the game developers. And many of those games have skin shaders that include subsurface scattering that make skin look more skin-like. And so the industries, you know, game developers are looking for more and more tools to express their art. And so this is just yet one more tool, and they get to decide what to use.
Lex Fridman
Ridiculous question. What do you think is the greatest or most influential game ever made? Maybe from NVIDIA’s perspective?
Ridiculous question. What do you think is the greatest or most influential game ever made? Maybe from NVIDIA’s perspective?
Jensen Huang
Doom.
Doom.
Lex Fridman
Doom, unquestionably. That was the start of the 3D.
Doom, unquestionably. That was the start of the 3D.
Jensen Huang
I would say Doom, from an art, the intersection of the cultural implication as well as the industry, turning a PC into a gaming device. That was a very important moment. Now, of course, flight simulation companies were before it. And but they just didn’t have the popularity that Doom did to have made the industry turn the PC from an office automation tool into a personal computer for families and gamers and things like that. And so Doom was really impactful there. From an actual game technology perspective, I would say Virtua Fighter. And so we’re great friends with both of them, you know?
I would say Doom, from an art, the intersection of the cultural implication as well as the industry, turning a PC into a gaming device. That was a very important moment. Now, of course, flight simulation companies were before it. And but they just didn’t have the popularity that Doom did to have made the industry turn the PC from an office automation tool into a personal computer for families and gamers and things like that. And so Doom was really impactful there. From an actual game technology perspective, I would say Virtua Fighter. And so we’re great friends with both of them, you know?
Lex Fridman
And then there’s games more recently—I mean, Cyberpunk 2077, really nice GPU-accelerated graphics. Like-
And then there’s games more recently—I mean, Cyberpunk 2077, really nice GPU-accelerated graphics. Like-
Jensen Huang
Fully ray traced.
Fully ray traced.
Lex Fridman
Fully ray traced. Also, I like, I personally, I’m a huge fan of Skyrim, Elder Scrolls, and the, you know, it’s, it’s been released a long, long time ago, but people release mods and-
Fully ray traced. Also, I like, I personally, I’m a huge fan of Skyrim, Elder Scrolls, and the, you know, it’s, it’s been released a long, long time ago, but people release mods and-
Jensen Huang
We love mods.
We love mods.
Lex Fridman
… they create these inc- I mean, it’s like a different game and it just allows me to replay the game over and over. It makes you realize that you can re-experience in a totally new way the world you already love. So-
… they create these inc- I mean, it’s like a different game and it just allows me to replay the game over and over. It makes you realize that you can re-experience in a totally new way the world you already love. So-
Jensen Huang
That’s right.
That’s right.
Lex Fridman
… I do that all the time. One of my favorite things is just walk across Skyrim.
… I do that all the time. One of my favorite things is just walk across Skyrim.
Jensen Huang
We created this thing called RTX Mod. Yeah, it’s a modding tool.
We created this thing called RTX Mod. Yeah, it’s a modding tool.
Lex Fridman
Awesome.
Awesome.
Jensen Huang
It allows the community to inject the latest technology into an old game.
It allows the community to inject the latest technology into an old game.
Lex Fridman
Of course, like what makes a great video game is not just graphics, it’s also story and character development, but-
Of course, like what makes a great video game is not just graphics, it’s also story and character development, but-
AGI timeline
Jensen Huang
That’s right
That’s right
Lex Fridman
… beautiful graphics can add to the immersion. The feeling like it’s another place you’re transported to. Ah, what you said, I think accurately, that the AGI timeline question rests on your definition of AGI. So let’s, let me ask you about possible timelines here. Let’s, this ridiculous definition perhaps of what AGI is, but an AI system that’s able to essentially do your job. So, run, no, start, grow, and run a successful technology company that’s worth-
… beautiful graphics can add to the immersion. The feeling like it’s another place you’re transported to. Ah, what you said, I think accurately, that the AGI timeline question rests on your definition of AGI. So let’s, let me ask you about possible timelines here. Let’s, this ridiculous definition perhaps of what AGI is, but an AI system that’s able to essentially do your job. So, run, no, start, grow, and run a successful technology company that’s worth-
Jensen Huang
A good one or a one?
A good one or a one?
Lex Fridman
No. It has to be worth more than a billion, more than a billion dollars. So, you know, you know how hard it is to do all those components. So, how far are we away from that? So, we’re talking about Open-Claude that does all the incredibly complex stuff that are required to, first of all, innovate, to find customers, to sell to them, to manage, to build a team of some agents, some humans, all that kind of stuff. Is this five, 10, 15, 20 years away?
No. It has to be worth more than a billion, more than a billion dollars. So, you know, you know how hard it is to do all those components. So, how far are we away from that? So, we’re talking about Open-Claude that does all the incredibly complex stuff that are required to, first of all, innovate, to find customers, to sell to them, to manage, to build a team of some agents, some humans, all that kind of stuff. Is this five, 10, 15, 20 years away?
Jensen Huang
I think it’s now. I think we’ve achieved AGI.
I think it’s now. I think we’ve achieved AGI.
Lex Fridman
Do you think you could have a company run by an AI system like this?
Do you think you could have a company run by an AI system like this?
Jensen Huang
Possible, and the reason for that is this. You said a billion, and you didn’t say forever. And so for example… It is not out of the question that a Claude was able to create a web service, some interesting little app that all of a sudden, you know, a few billion people used for 50 cents, and then it went out of business again shortly after. Now, we saw a whole bunch of those type of companies during the internet era, and most of those websites were not anything more sophisticated than what Open-Claude could generate today.
Possible, and the reason for that is this. You said a billion, and you didn’t say forever. And so for example… It is not out of the question that a Claude was able to create a web service, some interesting little app that all of a sudden, you know, a few billion people used for 50 cents, and then it went out of business again shortly after. Now, we saw a whole bunch of those type of companies during the internet era, and most of those websites were not anything more sophisticated than what Open-Claude could generate today.
Lex Fridman
Interesting. Achieve virality and monetize that virality.
Interesting. Achieve virality and monetize that virality.
Jensen Huang
Yeah. It’s just that I don’t know what it is, but I couldn’t have predicted any of those companies at the time either, you know? And –
Yeah. It’s just that I don’t know what it is, but I couldn’t have predicted any of those companies at the time either, you know? And –
Future of programming
Lex Fridman
You’re gonna get a lot of people excited with that statement.
You’re gonna get a lot of people excited with that statement.
Jensen Huang
Yeah, no. Yeah.
Yeah, no. Yeah.
Lex Fridman
It’s like, what do you mean? I can just launch an agent and make a lot of money.
It’s like, what do you mean? I can just launch an agent and make a lot of money.
Jensen Huang
Well, by the way, it’s happening right now, right? You know that when you go to China you’re gonna see, you’re gonna see a whole bunch of people teaching their, getting their Claudes to try to go out and look for jobs and, you know, do work, make money. And I’m not, I’m not actually… I wouldn’t be surprised if some social thing happened or somebody created a digital influencer, super, super cute, or some social application that, you know, feeds your little Tamagotchi or something like that, and it become out of the blue an instant success. A lot of people use it for a couple of months and it kind of dies away. Now, the odds of 100,000 of those agents building NVIDIA is zero percent.
Well, by the way, it’s happening right now, right? You know that when you go to China you’re gonna see, you’re gonna see a whole bunch of people teaching their, getting their Claudes to try to go out and look for jobs and, you know, do work, make money. And I’m not, I’m not actually… I wouldn’t be surprised if some social thing happened or somebody created a digital influencer, super, super cute, or some social application that, you know, feeds your little Tamagotchi or something like that, and it become out of the blue an instant success. A lot of people use it for a couple of months and it kind of dies away. Now, the odds of 100,000 of those agents building NVIDIA is zero percent.
Jensen Huang
And then, and then the one part that I will, I won’t do and I wanna make sure we all do, is to recognize that people are really worried about their jobs. And I just want to remind them that the purpose of your job and the tasks and tools that you use to do your job are related, not the same. I’ve been doing my job for 33 years. I’m the longest running tech CEO in the world, 34 years. And the tools that I’ve used to do my job has changed continuously in the last 34 years, and sometimes quite dramatically, you know, over the course of a couple, two, three years. And the one story that I really wanna make sure that everybody hears is the story that the first job that computer scientists said, AI researchers said was gonna go away was radiology.
And then, and then the one part that I will, I won’t do and I wanna make sure we all do, is to recognize that people are really worried about their jobs. And I just want to remind them that the purpose of your job and the tasks and tools that you use to do your job are related, not the same. I’ve been doing my job for 33 years. I’m the longest running tech CEO in the world, 34 years. And the tools that I’ve used to do my job has changed continuously in the last 34 years, and sometimes quite dramatically, you know, over the course of a couple, two, three years. And the one story that I really wanna make sure that everybody hears is the story that the first job that computer scientists said, AI researchers said was gonna go away was radiology.
Jensen Huang
Because computer vision was going to achieve superhuman levels, and it did. CV… Computer vision was superhuman in 2019, 20, maybe maybe a little bit later, 2020?
Because computer vision was going to achieve superhuman levels, and it did. CV… Computer vision was superhuman in 2019, 20, maybe maybe a little bit later, 2020?
Jensen Huang
Okay? And so it’s been a long time since computer vision has been superhuman. And so the prediction was radiologists would go away because studying radiology scans was a thing of the past. AI will do that. Well, they were absolutely right. Computer vision is completely superhuman. Every radiology platform and package today is driven by AI, and yet the number of radiologists grew. And so the question is why? And we now have a shortage of radiologists in the world. And so, one, the alarmist warning went too far and it scared people from doing this profession that is so important to society. And so it did harm. Now, why was it wrong? The reason why is because the purpose of a radiologist, the purpose is to diagnose disease and help patients and doctors diagnose disease.
Okay? And so it’s been a long time since computer vision has been superhuman. And so the prediction was radiologists would go away because studying radiology scans was a thing of the past. AI will do that. Well, they were absolutely right. Computer vision is completely superhuman. Every radiology platform and package today is driven by AI, and yet the number of radiologists grew. And so the question is why? And we now have a shortage of radiologists in the world. And so, one, the alarmist warning went too far and it scared people from doing this profession that is so important to society. And so it did harm. Now, why was it wrong? The reason why is because the purpose of a radiologist, the purpose is to diagnose disease and help patients and doctors diagnose disease.
Jensen Huang
And because we’re able to study scans so much faster now, you could study more scans, you could diagnose better, you could in-patient faster, you can see people more. The hospitals are making more money. You have more patients in the hospital. You need more radiologists. I mean, the amazing thing is, it’s so obvious this was gonna happen. The number of software engineers at NVIDIA is gonna grow, not decline. And the reason for that is because the purpose of a software engineer and the task of a software engineer coding are related, not the same. I wanted my software engineers to solve problems. I didn’t care how many lines of code they wrote, you know? But their job, their purpose of their job didn’t change.
And because we’re able to study scans so much faster now, you could study more scans, you could diagnose better, you could in-patient faster, you can see people more. The hospitals are making more money. You have more patients in the hospital. You need more radiologists. I mean, the amazing thing is, it’s so obvious this was gonna happen. The number of software engineers at NVIDIA is gonna grow, not decline. And the reason for that is because the purpose of a software engineer and the task of a software engineer coding are related, not the same. I wanted my software engineers to solve problems. I didn’t care how many lines of code they wrote, you know? But their job, their purpose of their job didn’t change.
Jensen Huang
Solving problems, working as a team, diagnosing problems, evaluating the result, looking for new problems to solve, innovation, connecting dots. You know, none of that stuff is gonna go away.
Solving problems, working as a team, diagnosing problems, evaluating the result, looking for new problems to solve, innovation, connecting dots. You know, none of that stuff is gonna go away.
Lex Fridman
Do you think it’s possible that… Let’s even take coding. Do you think the number of programmers in the world might increase, not decrease?
Do you think it’s possible that… Let’s even take coding. Do you think the number of programmers in the world might increase, not decrease?
Jensen Huang
Yes. And the reason for that is this. What is the definition of coding? I believe it is… The definition of coding, as of today, is simply specifying, specification, and maybe if you want to be rather directive, you could even give it an architecture of the software that you wanted to write. So the question is, how many people could do that? Describe a specification for a computer to go… telling the computer what to go build. How many people? I think we just went from 30 million to probably 1 billion. And so every carpenter in the future will be a coder, except a carpenter with AI is also an architect. They’ve just increased the value that they could deliver to the customer. Their artistry just elevated tremendously.
Yes. And the reason for that is this. What is the definition of coding? I believe it is… The definition of coding, as of today, is simply specifying, specification, and maybe if you want to be rather directive, you could even give it an architecture of the software that you wanted to write. So the question is, how many people could do that? Describe a specification for a computer to go… telling the computer what to go build. How many people? I think we just went from 30 million to probably 1 billion. And so every carpenter in the future will be a coder, except a carpenter with AI is also an architect. They’ve just increased the value that they could deliver to the customer. Their artistry just elevated tremendously.
Jensen Huang
I believe that every accountant is, you know, also your financial analyst, also your financial advisor. So, all of these professions have just been elevated… and if I were a carpenter, I see AI, I would just completely go berserk. You know, the services I can bring to my clients if I were a plumber, completely go berserk.
I believe that every accountant is, you know, also your financial analyst, also your financial advisor. So, all of these professions have just been elevated… and if I were a carpenter, I see AI, I would just completely go berserk. You know, the services I can bring to my clients if I were a plumber, completely go berserk.
Lex Fridman
And the, the people that are currently programmers and software engineers, I think they’re at the cutting edge of understanding intuitively how to communicate with the agents using natural language in order to design the best kind of software.
And the, the people that are currently programmers and software engineers, I think they’re at the cutting edge of understanding intuitively how to communicate with the agents using natural language in order to design the best kind of software.
Jensen Huang
That’s right, exactly.
That’s right, exactly.
Lex Fridman
So over time they’ll converge, but I think there’s still value in getting, I think learning how to program, like learning what programming languages are. The old kind of programming, what are good practices for programming languages, what are design principles for programming-
So over time they’ll converge, but I think there’s still value in getting, I think learning how to program, like learning what programming languages are. The old kind of programming, what are good practices for programming languages, what are design principles for programming-
Jensen Huang
That’s right
That’s right
Lex Fridman
… Languages for large software systems?
… Languages for large software systems?
Jensen Huang
And the reason for that, Lex, and you know, as you’re saying for the audience, I think the goal of, the goal of specification, the artistry of specification, the goal and the artistry of it is going to depend on what problem you’re trying to solve. When I’m thinking, when I’m thinking about giving the company strategies and formulating corporate directions and things that we should do, I describe it at a level that is sufficiently specific that people generally understand the direction and it’s actionable. It’s specific enough that they can take action on it, but I under-specify it on purpose, so that enables 43,000 amazing people to make it even better than I imagined.
And the reason for that, Lex, and you know, as you’re saying for the audience, I think the goal of, the goal of specification, the artistry of specification, the goal and the artistry of it is going to depend on what problem you’re trying to solve. When I’m thinking, when I’m thinking about giving the company strategies and formulating corporate directions and things that we should do, I describe it at a level that is sufficiently specific that people generally understand the direction and it’s actionable. It’s specific enough that they can take action on it, but I under-specify it on purpose, so that enables 43,000 amazing people to make it even better than I imagined.
Jensen Huang
And so when I’m working with engineers and when I’m working with people, I think about who, what problem am I trying to solve? Who am I working with? And the level of specification, the level of architecture definition relates to that. And so everybody’s going to have to learn how, where in the spectrum of coding they want to be. Writing a specification is coding. And so you might decide to be quite prescriptive because there’s a very specific outcome you’re looking for. You might decide that, you know, this is an area you want to be much more exploratory, and so you might under-specify and enable you to go back and forth with the AI to even push your own boundaries of creativity. And so this artistry of where you are in the spectrum, this is the future of coding.
And so when I’m working with engineers and when I’m working with people, I think about who, what problem am I trying to solve? Who am I working with? And the level of specification, the level of architecture definition relates to that. And so everybody’s going to have to learn how, where in the spectrum of coding they want to be. Writing a specification is coding. And so you might decide to be quite prescriptive because there’s a very specific outcome you’re looking for. You might decide that, you know, this is an area you want to be much more exploratory, and so you might under-specify and enable you to go back and forth with the AI to even push your own boundaries of creativity. And so this artistry of where you are in the spectrum, this is the future of coding.
Lex Fridman
But just to linger on it outside of coding, I think a lot of people, rightfully so, are worried about their jobs, have a lot of anxiety about their jobs, especially in the white-collar sector. I don’t think any of us know what to do with tumultuous times that always come when automations and new technology arrives. And I just… First of all, I think we all need to have compassion and the responsibility to feel sort of the burden of what the actual suffering feels like for individual people and families that lose their job. I think whenever you have transformative technology like that’s coming with artificial intelligence, there’s going to be a lot of pain, and I don’t know what to do about that pain.
But just to linger on it outside of coding, I think a lot of people, rightfully so, are worried about their jobs, have a lot of anxiety about their jobs, especially in the white-collar sector. I don’t think any of us know what to do with tumultuous times that always come when automations and new technology arrives. And I just… First of all, I think we all need to have compassion and the responsibility to feel sort of the burden of what the actual suffering feels like for individual people and families that lose their job. I think whenever you have transformative technology like that’s coming with artificial intelligence, there’s going to be a lot of pain, and I don’t know what to do about that pain.
Lex Fridman
Hopefully, it creates much more opportunities for those same people for the same kind of job as the tooling evolves and makes them more productive and makes them more fun, hopefully, as it does in the programming. I have been having so much fun programming, I have to say. Like, I’ve never had this much fun. So hopefully it makes their job, automates the boring parts and makes the creative parts the ones that the human beings are responsible for. But still there’s going to be a lot of pain and suffering.
Hopefully, it creates much more opportunities for those same people for the same kind of job as the tooling evolves and makes them more productive and makes them more fun, hopefully, as it does in the programming. I have been having so much fun programming, I have to say. Like, I’ve never had this much fun. So hopefully it makes their job, automates the boring parts and makes the creative parts the ones that the human beings are responsible for. But still there’s going to be a lot of pain and suffering.
Jensen Huang
So my first recommendation before… And this is now how I deal with anxiety. In fact, we just talked about it earlier. Enormous anxiety about the future, enormous anxiety about the pressure, enormous anxiety about uncertainty, I first break it down, and then I’m gonna tell myself, “Okay, there are some things you can do something about, there’s some things you can’t do anything about. But for the stuff that you can do something about, let’s reason, reason about it and let’s go do it.”
So my first recommendation before… And this is now how I deal with anxiety. In fact, we just talked about it earlier. Enormous anxiety about the future, enormous anxiety about the pressure, enormous anxiety about uncertainty, I first break it down, and then I’m gonna tell myself, “Okay, there are some things you can do something about, there’s some things you can’t do anything about. But for the stuff that you can do something about, let’s reason, reason about it and let’s go do it.”
Jensen Huang
If we were to hire a new college graduate today, and I have a choice between two, one that has no clue what AI is and one that is expert in using AI, I would hire the one who’s expert in using AI. If I had an accountant, a marketing person, the one that is expert in using AI, supply chain, customer service, a salesperson, business development, a lawyer, I would hire the one who is expert in using AI. And so I would advise that every college student, every teacher should encourage their student to go use AI. Every college student should graduate and be an expert in AI. And everybody, if you’re a carpenter, if you’re an electrician, go use AI. Go see what it can do to transform your current job, elevate yourself.
If we were to hire a new college graduate today, and I have a choice between two, one that has no clue what AI is and one that is expert in using AI, I would hire the one who’s expert in using AI. If I had an accountant, a marketing person, the one that is expert in using AI, supply chain, customer service, a salesperson, business development, a lawyer, I would hire the one who is expert in using AI. And so I would advise that every college student, every teacher should encourage their student to go use AI. Every college student should graduate and be an expert in AI. And everybody, if you’re a carpenter, if you’re an electrician, go use AI. Go see what it can do to transform your current job, elevate yourself.
Jensen Huang
If I were a farmer, I would absolutely use AI. If I were a pharmacist, I would use AI. I wanna see how, what it could do to elevate my job so that I could be the innovator to revolutionize this industry myself. And so that would be the first thing that I would do. And then I would also help them… It is the case that the technology will dislocate and will eliminate many tasks. And because it will automate it, if your job is the task—then you’re very highly going to be disrupted. If your job’s purpose includes you, certain tasks- … then it’s vital that you go learn how to use AI to automate those tasks. And then there’s the world of spectrum in between.
If I were a farmer, I would absolutely use AI. If I were a pharmacist, I would use AI. I wanna see how, what it could do to elevate my job so that I could be the innovator to revolutionize this industry myself. And so that would be the first thing that I would do. And then I would also help them… It is the case that the technology will dislocate and will eliminate many tasks. And because it will automate it, if your job is the task—then you’re very highly going to be disrupted. If your job’s purpose includes you, certain tasks- … then it’s vital that you go learn how to use AI to automate those tasks. And then there’s the world of spectrum in between.
Lex Fridman
And by the way, the beautiful thing about AI, so the chatbot versions, is you can break down… You have anxiety and you can break down the problem by talking to it. Like, I’ve recently… It’s really just incredible how much you can think through your life’s problems, and through… And I don’t mean, like, therapy problems. I mean, like, very practically, “Okay, I’m worried about my…” Literally, “I’m worried about my job. What are the skills? What are the steps I need to take?” How do I get better at AI?” Everything you just said, you could literally ask and it’s going to give you- … a point-by-point plan. I mean, it’s just a great life coach, period. This-
And by the way, the beautiful thing about AI, so the chatbot versions, is you can break down… You have anxiety and you can break down the problem by talking to it. Like, I’ve recently… It’s really just incredible how much you can think through your life’s problems, and through… And I don’t mean, like, therapy problems. I mean, like, very practically, “Okay, I’m worried about my…” Literally, “I’m worried about my job. What are the skills? What are the steps I need to take?” How do I get better at AI?” Everything you just said, you could literally ask and it’s going to give you- … a point-by-point plan. I mean, it’s just a great life coach, period. This-
Jensen Huang
I don’t know how to use AI, and the AI goes, “Well, let me show you.”
I don’t know how to use AI, and the AI goes, “Well, let me show you.”
Lex Fridman
Exactly. It’s very meta, but it’s- It’s kind of incredible. So people definitely should-
Exactly. It’s very meta, but it’s- It’s kind of incredible. So people definitely should-
Jensen Huang
You can’t walk up to Excel and say, “I don’t know how to use Excel.”
You can’t walk up to Excel and say, “I don’t know how to use Excel.”
Lex Fridman
Exactly.
Exactly.
Jensen Huang
You’re done.
You’re done.
Lex Fridman
I mean, that’s really what AI has done for me in all walks of life, is that initial friction of being a beginner of using a thing for the first time. I can literally ask about any single thing, “What are the first steps I need to take?”
I mean, that’s really what AI has done for me in all walks of life, is that initial friction of being a beginner of using a thing for the first time. I can literally ask about any single thing, “What are the first steps I need to take?”
Jensen Huang
That’s right.
That’s right.
Lex Fridman
And that handholding that it does, removing the friction of all the experiences that the world offers is… You know, like I mentioned to you offline, you mentioned, “I’m going to China and Taiwan.”
And that handholding that it does, removing the friction of all the experiences that the world offers is… You know, like I mentioned to you offline, you mentioned, “I’m going to China and Taiwan.”
Jensen Huang
So awesome.
So awesome.
Lex Fridman
Just ask, “Where do I-“
Just ask, “Where do I-“
Jensen Huang
So excited for you.
So excited for you.
Lex Fridman
“Where do I… What do…” “You know, where do I go? How do I…” All of those questions- … immediately answered, and it’s beautiful.
“Where do I… What do…” “You know, where do I go? How do I…” All of those questions- … immediately answered, and it’s beautiful.
Jensen Huang
Well, when you go to Taiwan, just ask AI… “What are Jensen’s favorite restaurants in Taiwan?” And it’ll actually-
Well, when you go to Taiwan, just ask AI… “What are Jensen’s favorite restaurants in Taiwan?” And it’ll actually-
Lex Fridman
You don’t know?
You don’t know?
Jensen Huang
Oh, yeah.
Oh, yeah.
Lex Fridman
Is it accurate? Okay. All right.
Is it accurate? Okay. All right.
Jensen Huang
It’s all over Taiwan.
It’s all over Taiwan.
Lex Fridman
Well, you’re a rockstar over there. And like we also mentioned offline, maybe our paths will cross, which would be really wonderful in computing.
Well, you’re a rockstar over there. And like we also mentioned offline, maybe our paths will cross, which would be really wonderful in computing.
Jensen Huang
COMPUTEX. NVIDIA GTC Taiwan.
COMPUTEX. NVIDIA GTC Taiwan.
Consciousness
Lex Fridman
Do you think there’s some things about human nature, about human consciousness that is fundamentally non-computational? Maybe something a chip, no matter how powerful, can never replicate?
Do you think there’s some things about human nature, about human consciousness that is fundamentally non-computational? Maybe something a chip, no matter how powerful, can never replicate?
Jensen Huang
I don’t know if the chip will ever get nervous. And that’s the, you know, of course, the conditions by which that causes anxiety or nervousness or whatever emotion. I believe that AI will be able to recognize those and understand those. I don’t think my chips will feel those. And therefore, the… How that anxiety, how that feeling, how that excitement, how that, how that, you know… All of those feelings manifest in human performance. For example, extremely amazing human performance, athletic performance, you know, average or lesser than average. That entire spectrum of human performance that comes out of exactly the same circumstances for different people, manifesting a different outcome, manifesting a different performance.
I don’t know if the chip will ever get nervous. And that’s the, you know, of course, the conditions by which that causes anxiety or nervousness or whatever emotion. I believe that AI will be able to recognize those and understand those. I don’t think my chips will feel those. And therefore, the… How that anxiety, how that feeling, how that excitement, how that, how that, you know… All of those feelings manifest in human performance. For example, extremely amazing human performance, athletic performance, you know, average or lesser than average. That entire spectrum of human performance that comes out of exactly the same circumstances for different people, manifesting a different outcome, manifesting a different performance.
Jensen Huang
I don’t think there’s anything about anything that we’re building that would suggest that two different computers being presented with all of exactly the same context would perfo- Of course, it would produce statistically different outcomes, but it’s not because it felt different.
I don’t think there’s anything about anything that we’re building that would suggest that two different computers being presented with all of exactly the same context would perfo- Of course, it would produce statistically different outcomes, but it’s not because it felt different.
Lex Fridman
Yeah, the subjective… Boy, there’s something truly special about the subjective experience that we humans feel. Like I mentioned to you, I was pretty nervous talking to you. Like I mentioned to you, that, the hope, the fear, the anxiety, and just life itself, the richness of life. How amazing everything is. How deeply we fall in love, how deeply our hearts get broken, how afraid we are of death and how much pain we feel when our loved ones pass away. All of that, the whole thing. I know it’s very hard to- … think AI being able to… A computational device being able to do that. But there’s so many mysteries about this whole thing that we’re yet to uncover, that I am open to be surprised. I’ve been surprised a lot over the past-
Yeah, the subjective… Boy, there’s something truly special about the subjective experience that we humans feel. Like I mentioned to you, I was pretty nervous talking to you. Like I mentioned to you, that, the hope, the fear, the anxiety, and just life itself, the richness of life. How amazing everything is. How deeply we fall in love, how deeply our hearts get broken, how afraid we are of death and how much pain we feel when our loved ones pass away. All of that, the whole thing. I know it’s very hard to- … think AI being able to… A computational device being able to do that. But there’s so many mysteries about this whole thing that we’re yet to uncover, that I am open to be surprised. I’ve been surprised a lot over the past-
Lex Fridman
… few months and few years. Scaling can create some incredible miracles in the space of intelligence. It has been truly marvelous to watch, so I’m open to surprise.
… few months and few years. Scaling can create some incredible miracles in the space of intelligence. It has been truly marvelous to watch, so I’m open to surprise.
Jensen Huang
And it’s just really important to break down what is intelligence. You know, the word, that word we use all the time, it’s not a mysterious word. Intelligence has a meaning, you know?
And it’s just really important to break down what is intelligence. You know, the word, that word we use all the time, it’s not a mysterious word. Intelligence has a meaning, you know?
Jensen Huang
And it’s a system that… You know, it’s something that we do that includes perception and understanding and reasoning and the ability to do plan. And, you know, that loop, that loop, is the… Fundamentally what intelligence is. Intelligence is not one word that is exactly equal to humanity. And that’s, I think it’s really important to separate the two. We have two words for that. I’m not… I don’t over-fantasize about, and I don’t over-romanticize about intelligence. Intelligence is… And people have heard me say it before, I actually think intelligence is a commodity. I’m surrounded by intelligent people. And I’m surrounded by intelligent people more intelligent than I am in each one of the spaces that they’re in.
And it’s a system that… You know, it’s something that we do that includes perception and understanding and reasoning and the ability to do plan. And, you know, that loop, that loop, is the… Fundamentally what intelligence is. Intelligence is not one word that is exactly equal to humanity. And that’s, I think it’s really important to separate the two. We have two words for that. I’m not… I don’t over-fantasize about, and I don’t over-romanticize about intelligence. Intelligence is… And people have heard me say it before, I actually think intelligence is a commodity. I’m surrounded by intelligent people. And I’m surrounded by intelligent people more intelligent than I am in each one of the spaces that they’re in.
Jensen Huang
And yet, I have a role in that circle. It’s actually kind of interesting. They’re more educated than I am. They went to better schools than I did. They’re deeper in any of the fields that they’re in. All of them. I have 60 of them. They’re all superhuman to me. And somehow, I’m sitting in the middle orchestrating all 60 of them. And so you gotta ask yourself… What is it about a dishwasher that allows that dishwasher to sit in the middle of superhumans? Does that make sense?
And yet, I have a role in that circle. It’s actually kind of interesting. They’re more educated than I am. They went to better schools than I did. They’re deeper in any of the fields that they’re in. All of them. I have 60 of them. They’re all superhuman to me. And somehow, I’m sitting in the middle orchestrating all 60 of them. And so you gotta ask yourself… What is it about a dishwasher that allows that dishwasher to sit in the middle of superhumans? Does that make sense?
Jensen Huang
And so, but that’s my point. My point is intelligence is a functional thing. Humanity is not specified functionally. It’s a much, much bigger word. And our life experience, our tolerance for pain, our determination, those are different words than intelligence. And so the thing that I wanna help the audience understand, if I could give them one thing, is intelligence is a word that we’ve elevated to a very high form over time.
And so, but that’s my point. My point is intelligence is a functional thing. Humanity is not specified functionally. It’s a much, much bigger word. And our life experience, our tolerance for pain, our determination, those are different words than intelligence. And so the thing that I wanna help the audience understand, if I could give them one thing, is intelligence is a word that we’ve elevated to a very high form over time.
Lex Fridman
The word we should really elevate is humanity.
The word we should really elevate is humanity.
Jensen Huang
Character, humanity.
Character, humanity.
Lex Fridman
All those things.
All those things.
Jensen Huang
All of those things. Compassion, generosity, all of the things that you say just now, I believe those are superhuman powers. And that now intelligence is gonna be commoditized. Because we’ve spoken about it, the most important thing is your education. Now, even when they said the most important thing is your education, when you went to school, there’s more than just knowledge that you gained.
All of those things. Compassion, generosity, all of the things that you say just now, I believe those are superhuman powers. And that now intelligence is gonna be commoditized. Because we’ve spoken about it, the most important thing is your education. Now, even when they said the most important thing is your education, when you went to school, there’s more than just knowledge that you gained.
Jensen Huang
And so, but unfortunately, our society had put everything into one single word, and life is more than one word. And I’m just telling you, my life would suggest that being lower on the intelligence curve than everybody around me doesn’t change the fact I’m the most successful. And so, and I think that kind of is—I’m trying to hopefully to inspire everybody else—that don’t let this democratization of intelligence, this commoditization of intelligence, cause you anxiety. You should be inspired by that.
And so, but unfortunately, our society had put everything into one single word, and life is more than one word. And I’m just telling you, my life would suggest that being lower on the intelligence curve than everybody around me doesn’t change the fact I’m the most successful. And so, and I think that kind of is—I’m trying to hopefully to inspire everybody else—that don’t let this democratization of intelligence, this commoditization of intelligence, cause you anxiety. You should be inspired by that.
Lex Fridman
Yeah. I think AI will help us celebrate humans more. And certainly humanity and human first, and I think what makes this world incredible is humans forever will be so, and just AI is this incredible tool that makes us-
Yeah. I think AI will help us celebrate humans more. And certainly humanity and human first, and I think what makes this world incredible is humans forever will be so, and just AI is this incredible tool that makes us-
Jensen Huang
That’s exactly right.
That’s exactly right.
Lex Fridman
… humans more powerful.
… humans more powerful.
Jensen Huang
That’s exactly right.
That’s exactly right.
Mortality
Lex Fridman
So much of the success of NVIDIA and the lives of millions of people that I mentioned depend on you. But you’re just one human, like we mentioned, a mortal like all of us. Do you think about your mortality? Are you afraid of death?
So much of the success of NVIDIA and the lives of millions of people that I mentioned depend on you. But you’re just one human, like we mentioned, a mortal like all of us. Do you think about your mortality? Are you afraid of death?
Jensen Huang
I really don’t wanna die. I have a great life. I have a great family. I have really important work. This is not a once in a lifetime experience suggests that it has been experienced by many people, just not one person. This is a once in a humanity experience, what I’m going through. NVIDIA is one of the most consequential technology companies in history. We’re doing very important work. I take it very seriously. And so some of the things that of course are practical things, like how do we think about succession planning? And I’m famous in saying that I don’t believe in succession planning.
I really don’t wanna die. I have a great life. I have a great family. I have really important work. This is not a once in a lifetime experience suggests that it has been experienced by many people, just not one person. This is a once in a humanity experience, what I’m going through. NVIDIA is one of the most consequential technology companies in history. We’re doing very important work. I take it very seriously. And so some of the things that of course are practical things, like how do we think about succession planning? And I’m famous in saying that I don’t believe in succession planning.
Lex Fridman
Man.
Man.
Jensen Huang
And the reason for that isn’t because I’m immortal. The reason for that is because if you’re worried about succession planning, if you’re worried all that anxiety of succession planning, then what should you do about it? Then you break it all the way back down. The most important thing you should do today, if you care about the future of your company, post you, is to pass on knowledge, information, insight, skills, experience as often and continuously as you can, which is the reason why I continuously reason about everything in front of my team. Every single meeting is a reasoning meeting. Every moment I spend inside a company, outside a company is about passing on knowledge to people as fast as I can.
And the reason for that isn’t because I’m immortal. The reason for that is because if you’re worried about succession planning, if you’re worried all that anxiety of succession planning, then what should you do about it? Then you break it all the way back down. The most important thing you should do today, if you care about the future of your company, post you, is to pass on knowledge, information, insight, skills, experience as often and continuously as you can, which is the reason why I continuously reason about everything in front of my team. Every single meeting is a reasoning meeting. Every moment I spend inside a company, outside a company is about passing on knowledge to people as fast as I can.
Jensen Huang
Nothing I learn ever sits on my desk longer than, you know, a fraction of a second. I’m passing that information, that knowledge—oh my gosh, this is cool. Before I even finish learning all of it myself, I’m already pointing it to somebody else. “Get on this. This is so cool. You’re gonna wanna learn this.” And so I’m constantly passing knowledge, empowering people, elevating the capability of everybody around me, so that the outcome that I seek, that I hope for, is that I die on the job, you know? And hopefully I die on the job instantaneously, you know? And there’s no long periods of suffering, you know? It’s, uh –
Nothing I learn ever sits on my desk longer than, you know, a fraction of a second. I’m passing that information, that knowledge—oh my gosh, this is cool. Before I even finish learning all of it myself, I’m already pointing it to somebody else. “Get on this. This is so cool. You’re gonna wanna learn this.” And so I’m constantly passing knowledge, empowering people, elevating the capability of everybody around me, so that the outcome that I seek, that I hope for, is that I die on the job, you know? And hopefully I die on the job instantaneously, you know? And there’s no long periods of suffering, you know? It’s, uh –
Lex Fridman
Well, from a fan perspective, given your extremely enormous positive impact on civilization, of course, I hope you keep going. But also it’s just fun to watch what NVIDIA is doing, you know. It’s just the rate of innovation. And I’m a huge fan of engineering. There’s so much incredible engineering continuously being done by NVIDIA. It’s just fun to watch. It’s a celebration of humanity, a celebration of great builders, a celebration of great engineering. So, it represents something special. So I hope you and NVIDIA keep going. What gives you hope about this whole thing we got going on, about humanity, about the future of humanity? When you look out, when you think about the future quite a bit, when you look out 10, 20, 50, 100 years from now, what gives you hope?
Well, from a fan perspective, given your extremely enormous positive impact on civilization, of course, I hope you keep going. But also it’s just fun to watch what NVIDIA is doing, you know. It’s just the rate of innovation. And I’m a huge fan of engineering. There’s so much incredible engineering continuously being done by NVIDIA. It’s just fun to watch. It’s a celebration of humanity, a celebration of great builders, a celebration of great engineering. So, it represents something special. So I hope you and NVIDIA keep going. What gives you hope about this whole thing we got going on, about humanity, about the future of humanity? When you look out, when you think about the future quite a bit, when you look out 10, 20, 50, 100 years from now, what gives you hope?
Jensen Huang
I’ve always had a great confidence in the kindness, the generosity, the compassion, the human capacity. I’ve always been extremely confident of that. Sometimes more so than I should. And I get taken advantage of, but it doesn’t ever cause me not to. I start with always that people want to do good. People want to help others. And vastly, I am proven right. Constantly proven right. And often it exceeds my expectations. And so I have complete confidence in the human capacity. I think the things that give me incredible hope is what I see now as possible, and as I extrapolate based on the things that we’re doing, what will very likely happen.
I’ve always had a great confidence in the kindness, the generosity, the compassion, the human capacity. I’ve always been extremely confident of that. Sometimes more so than I should. And I get taken advantage of, but it doesn’t ever cause me not to. I start with always that people want to do good. People want to help others. And vastly, I am proven right. Constantly proven right. And often it exceeds my expectations. And so I have complete confidence in the human capacity. I think the things that give me incredible hope is what I see now as possible, and as I extrapolate based on the things that we’re doing, what will very likely happen.
Jensen Huang
And that there’s so many things that we wanna solve. There’s so many problems we wanna solve. There’s so many things that we wanna build. There’s so many good things that we wanna do that are now within our reach, and within the reach of my lifetime. You just can’t possibly not be romantic about that. You know what I’m saying? O-
And that there’s so many things that we wanna solve. There’s so many problems we wanna solve. There’s so many things that we wanna build. There’s so many good things that we wanna do that are now within our reach, and within the reach of my lifetime. You just can’t possibly not be romantic about that. You know what I’m saying? O-
Lex Fridman
What an exciting time to be alive. Like, truly-
What an exciting time to be alive. Like, truly-
Jensen Huang
How can-
How can-
Lex Fridman
… truly so.
… truly so.
Jensen Huang
How can you not be romantic about that? The fact that there is a—it’s a reasonable thing to expect the end of disease. It’s a reasonable thing to expect. It’s a reasonable thing to expect that pollution will be drastically reduced. It’s a reasonable thing to expect that traveling at the speed of light is actually in our future. And then, you know, not for long distances, but short distances. You know, and people ask me how. Well, first of all, very soon, I’m gonna put a humanoid on a spaceship, and it’s gonna be, you know, my humanoid, and we’re gonna send it out as soon as possible, and it’s gonna keep improving and enhancing along the flight.
How can you not be romantic about that? The fact that there is a—it’s a reasonable thing to expect the end of disease. It’s a reasonable thing to expect. It’s a reasonable thing to expect that pollution will be drastically reduced. It’s a reasonable thing to expect that traveling at the speed of light is actually in our future. And then, you know, not for long distances, but short distances. You know, and people ask me how. Well, first of all, very soon, I’m gonna put a humanoid on a spaceship, and it’s gonna be, you know, my humanoid, and we’re gonna send it out as soon as possible, and it’s gonna keep improving and enhancing along the flight.
Jensen Huang
And then when it’s time, all of my consciousness has already been—you know, so much of my life has been uploaded in the internet. Take all my inbox, take everything that I’ve done, everything I’ve said. You know, it’s been collected and becoming my AI. And I’m just, when the time comes, we’ll just send that at the speed of light, catch up with my robot.
And then when it’s time, all of my consciousness has already been—you know, so much of my life has been uploaded in the internet. Take all my inbox, take everything that I’ve done, everything I’ve said. You know, it’s been collected and becoming my AI. And I’m just, when the time comes, we’ll just send that at the speed of light, catch up with my robot.
Lex Fridman
Oh, that’s brilliant. I mean, but for me, that’s sorta application-focused. But also, for me, the curiosity-maxing perspective, I just, all of those mysteries. There’s so much- … fascinating scientific questions there.
Oh, that’s brilliant. I mean, but for me, that’s sorta application-focused. But also, for me, the curiosity-maxing perspective, I just, all of those mysteries. There’s so much- … fascinating scientific questions there.
Jensen Huang
Understanding the biological machine is right around the corner. It’s, it’s not 10 years. It’s five years probably.
Understanding the biological machine is right around the corner. It’s, it’s not 10 years. It’s five years probably.
Lex Fridman
And then your biological machine, the, the human mind and cracking physics, theoretical physics open. It’s so exciting.
And then your biological machine, the, the human mind and cracking physics, theoretical physics open. It’s so exciting.
Jensen Huang
Explaining consciousness, that one would be awesome.
Explaining consciousness, that one would be awesome.
Lex Fridman
And it’s all within our reach. Jensen, thank you so much for everything you’ve done over the years. Thank you for everything you’re doing for the world. Thank you for being who you are. I can tell you’re a great human being, and I wish you incredible success this year. I can’t wait. As a fan, I can’t wait to see what you do next, and hopefully I’ll see you in Taiwan and thank you so much for talking today.
And it’s all within our reach. Jensen, thank you so much for everything you’ve done over the years. Thank you for everything you’re doing for the world. Thank you for being who you are. I can tell you’re a great human being, and I wish you incredible success this year. I can’t wait. As a fan, I can’t wait to see what you do next, and hopefully I’ll see you in Taiwan and thank you so much for talking today.
Jensen Huang
Thank you, Lex. I had a great time. And also, if I could just say one more thing. And thank you for all the interviews that you do, the depth, the respect that you go through with and the research that you do to reveal, you know, for all of us the amazing people that you’ve interviewed over the years. I’ve enjoyed them immensely. And as an innovator, to have created this long form, unbelievable, and yet, you know, it’s just captivating. So anyways, thank you for everything you do.
Thank you, Lex. I had a great time. And also, if I could just say one more thing. And thank you for all the interviews that you do, the depth, the respect that you go through with and the research that you do to reveal, you know, for all of us the amazing people that you’ve interviewed over the years. I’ve enjoyed them immensely. And as an innovator, to have created this long form, unbelievable, and yet, you know, it’s just captivating. So anyways, thank you for everything you do.
Lex Fridman
It means the world. Thank you, Jensen.
It means the world. Thank you, Jensen.
Jensen Huang
Thank you, Lex.
Thank you, Lex.
Lex Fridman
Thank you for listening to this conversation with Jensen Huang. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from Alan Kay. “The best way to predict the future is to invent it.” Thank you for listening, and hope to see you next time.
Thank you for listening to this conversation with Jensen Huang. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from Alan Kay. “The best way to predict the future is to invent it.” Thank you for listening, and hope to see you next time.
Transcript for Jeff Kaplan: World of Warcraft, Overwatch, Blizzard, and Future of Gaming | Lex Fridman Podcast #493
This is a transcript of Lex Fridman Podcast #493 with Jeff Kaplan.
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
- Go back to this episode’s main page
- Watch the full YouTube version of the podcast
Table of Contents
Here are the loose “chapters” in the conversation.
Click link to jump approximately to that part in the transcript:
Click link to jump approximately to that part in the transcript:
- 0:00 – Episode highlight
- 1:27 – Introduction
- 4:07 – Early games: Pac-Man, Zork, Doom, Quake
- 18:33 – Writing career
- 34:06 – EverQuest obsession
- 47:04 – Getting hired at Blizzard
- 1:02:32 – Lowest point in Jeff’s life
- 1:08:37 – One of Us
- 1:12:54 – Early Blizzard culture
- 1:32:36 – Building World of Warcraft
- 1:50:20 – How WoW changed video games
- 2:07:42 – Single-player vs Multi-player
- 2:28:35 – How Blizzard made great video games
- 2:54:25 – Online toxicity
- 3:01:59 – Why Titan failed
- 3:19:09 – Overwatch in six weeks
- 3:46:07 – Best Overwatch heroes
- 3:54:37 – The challenge of matchmaking
- 3:58:01 – Rust
- 4:08:22 – Why Jeff left Blizzard
- 4:30:35 – Diablo IV
- 4:32:03 – Getting back to making video games
- 4:40:59 – The Legend of California
- 4:54:44 – Greatest video game of all time
- 5:02:51 – AI and future of video games
Episode highlight
Jeff Kaplan
There’s three types of fun, fun for the player, fun for the designer, and fun for the computer.
There’s three types of fun, fun for the player, fun for the designer, and fun for the computer.
Lex Fridman
Is it PvP?
Is it PvP?
Jeff Kaplan
It’s all PvP. In fact, Rust is the most PvP thing in all of PvP.
It’s all PvP. In fact, Rust is the most PvP thing in all of PvP.
Lex Fridman
Well, I don’t know what that means, but-
Well, I don’t know what that means, but-
Jeff Kaplan
Rust players know what that means. My whole career and my family are thanks to EverQuest, so I think I won the game. And we’re idiots. We’re reading the forums, and the forums are just flaming us all the time. Like, “There’s lag on this server,” and, “Can’t log into that ser—” And that’s, that was our perspective of what was happening. And when I showed up at that show, it… One of the most emotional things in my life. It was nothing but an outpouring of love. I had believed I would never work any place but Blizzard. I loved it. It was a part of who I was and I felt I was a part of it, and I literally thought I would retire from the place. I never thought the day would come, and that was it.
Rust players know what that means. My whole career and my family are thanks to EverQuest, so I think I won the game. And we’re idiots. We’re reading the forums, and the forums are just flaming us all the time. Like, “There’s lag on this server,” and, “Can’t log into that ser—” And that’s, that was our perspective of what was happening. And when I showed up at that show, it… One of the most emotional things in my life. It was nothing but an outpouring of love. I had believed I would never work any place but Blizzard. I loved it. It was a part of who I was and I felt I was a part of it, and I literally thought I would retire from the place. I never thought the day would come, and that was it.
Lex Fridman
How painful was it to say goodbye?
How painful was it to say goodbye?
Jeff Kaplan
It broke me.
It broke me.
Introduction
Lex Fridman
Now, meanwhile, as far as the outside world is concerned, you’ve disappeared off the face of the earth, but you were actually working on a game. The following is a conversation with Jeff Kaplan, a legendary game designer of World of Warcraft and Overwatch, which are two of the biggest, most influential games ever made. He is genuinely one of the most amazing human beings I’ve ever met. In the many conversations I was fortunate enough to have with him, including while playing video games, he was always kind, thoughtful, hilarious, and still and forever a legit gamer, through and through. Of course, he’s always quick to celebrate the incredible teams of creative minds he has gotten a chance to work with over the years, and they are truly incredible.
Now, meanwhile, as far as the outside world is concerned, you’ve disappeared off the face of the earth, but you were actually working on a game. The following is a conversation with Jeff Kaplan, a legendary game designer of World of Warcraft and Overwatch, which are two of the biggest, most influential games ever made. He is genuinely one of the most amazing human beings I’ve ever met. In the many conversations I was fortunate enough to have with him, including while playing video games, he was always kind, thoughtful, hilarious, and still and forever a legit gamer, through and through. Of course, he’s always quick to celebrate the incredible teams of creative minds he has gotten a chance to work with over the years, and they are truly incredible.
Lex Fridman
Blizzard has created some of the greatest games ever made, games that to me personally have brought me thousands of hours of fun, meaning, and happiness, from Warcraft, to StarCraft, to Diablo, WoW, Overwatch and more. So for that, a big thank you to Jeff, to the entire Blizzard team, and to every creative mind in the video game industry, giving their heart and soul to build video game worlds that we fans get a chance to enjoy. This was a super fun, inspiring, whirlwind conversation, pun intended, with one of the most beloved gamers and game designers ever. Full of memes, lulz, wisdom, emotional rollercoaster moments, and of course, Blizzard video game lore.
Blizzard has created some of the greatest games ever made, games that to me personally have brought me thousands of hours of fun, meaning, and happiness, from Warcraft, to StarCraft, to Diablo, WoW, Overwatch and more. So for that, a big thank you to Jeff, to the entire Blizzard team, and to every creative mind in the video game industry, giving their heart and soul to build video game worlds that we fans get a chance to enjoy. This was a super fun, inspiring, whirlwind conversation, pun intended, with one of the most beloved gamers and game designers ever. Full of memes, lulz, wisdom, emotional rollercoaster moments, and of course, Blizzard video game lore.
Lex Fridman
Jeff left Blizzard in 2021, and has been secretly working on a new video game called The Legend of California that I got a chance to play with Jeff. It is incredibly beautiful. Set in the 1800s Gold Rush era of California, it’s an open world online multiplayer game, part adventure and action, part survival. Sometimes creating a feeling of loneliness and desperation, and sometimes just awe watching the sun rise over a beautiful landscape. It’s unlike any game that Jeff has ever worked on, and it’s a game that I genuinely can’t wait to play with all of you. You can wishlist it on Steam. Join the alpha later in March, I think, and early access is on the way.
Jeff left Blizzard in 2021, and has been secretly working on a new video game called The Legend of California that I got a chance to play with Jeff. It is incredibly beautiful. Set in the 1800s Gold Rush era of California, it’s an open world online multiplayer game, part adventure and action, part survival. Sometimes creating a feeling of loneliness and desperation, and sometimes just awe watching the sun rise over a beautiful landscape. It’s unlike any game that Jeff has ever worked on, and it’s a game that I genuinely can’t wait to play with all of you. You can wishlist it on Steam. Join the alpha later in March, I think, and early access is on the way.
Lex Fridman
This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Jeff Kaplan.
This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Jeff Kaplan.
Early games: Pac-Man, Zork, Doom, Quake
Lex Fridman
You were first a legendary video game player, in particular in EverQuest, before you ever became a legendary video game designer on World of Warcraft and on Overwatch, which I think is a wild journey to go through from gamer to designer. But first, let’s go way back. When did you first fall in love with video games?
You were first a legendary video game player, in particular in EverQuest, before you ever became a legendary video game designer on World of Warcraft and on Overwatch, which I think is a wild journey to go through from gamer to designer. But first, let’s go way back. When did you first fall in love with video games?
Jeff Kaplan
I was lucky. I was born in that golden era of coin-op. So, I literally remember the first time seeing Pac-Man. I was with my Uncle Ronnie, and he just kept feeding me quarters. I think he wanted to play, but was too scared to, so he, you know, his little nephew, he was just giving him quarters to play Pac-Man. I remember being at my brother’s graduation in Philadelphia, and they had an Asteroids machine in the lobby. That was one of the first coin-op machines I had played as well. And my brother and I would… we would try to get the high score, and we’d finally get it. But we had to go to bed early ’cause we were little kids. And then in the morning somebody else had like beat our high score. And then, you know, I grew up in Southern California in the ’80s. I was born in ’72.
I was lucky. I was born in that golden era of coin-op. So, I literally remember the first time seeing Pac-Man. I was with my Uncle Ronnie, and he just kept feeding me quarters. I think he wanted to play, but was too scared to, so he, you know, his little nephew, he was just giving him quarters to play Pac-Man. I remember being at my brother’s graduation in Philadelphia, and they had an Asteroids machine in the lobby. That was one of the first coin-op machines I had played as well. And my brother and I would… we would try to get the high score, and we’d finally get it. But we had to go to bed early ’cause we were little kids. And then in the morning somebody else had like beat our high score. And then, you know, I grew up in Southern California in the ’80s. I was born in ’72.
Jeff Kaplan
So, you know, I was a kid with that skateboard BMX culture where we’d ride two towns over. We knew all the pizza parlors and liquor stores and arcades, and we just lived in that coin-op phase. That was, that was where the love started. And then you started to see things like Pong. You’d go over to a friend’s house, they’d have Pong, and it was just mind blowing, like, we’re playing this thing on the TV and it was so much fun. Atari was a big thing at that time as well. But the big one for me was actually Intellivision, because my dad was an executive recruiter, and one of his clients was Mattel. And he said, “Hey, I… They gave me this thing,” and he would get discounts or free games. And my brothers and I just loved Intellivision. Like, we would just play it endlessly.
So, you know, I was a kid with that skateboard BMX culture where we’d ride two towns over. We knew all the pizza parlors and liquor stores and arcades, and we just lived in that coin-op phase. That was, that was where the love started. And then you started to see things like Pong. You’d go over to a friend’s house, they’d have Pong, and it was just mind blowing, like, we’re playing this thing on the TV and it was so much fun. Atari was a big thing at that time as well. But the big one for me was actually Intellivision, because my dad was an executive recruiter, and one of his clients was Mattel. And he said, “Hey, I… They gave me this thing,” and he would get discounts or free games. And my brothers and I just loved Intellivision. Like, we would just play it endlessly.
Jeff Kaplan
And the comparison was always like, “Is this game close to what’s in the arcades?” And it was just such a golden era. And I think the, the big moment where it really blew open and kind of hit the next level was when the NES came out. And that, like, NES with Super Mario- … Was kind of gaming at the next level at that point. And I have, like, warm, fuzzy memories even thinking about it to this day. I remember we played Super Mario for weeks, my brothers and I, and then I had a friend come over, and he showed me all the secret stuff-
And the comparison was always like, “Is this game close to what’s in the arcades?” And it was just such a golden era. And I think the, the big moment where it really blew open and kind of hit the next level was when the NES came out. And that, like, NES with Super Mario- … Was kind of gaming at the next level at that point. And I have, like, warm, fuzzy memories even thinking about it to this day. I remember we played Super Mario for weeks, my brothers and I, and then I had a friend come over, and he showed me all the secret stuff-
Jeff Kaplan
… in Super that I didn’t know existed at the time. And it’s… it was like suddenly, the world opened up more and games could be more. And then there was, like, a big PC gaming push that hit me. My parents ran their own business. Like I said, my dad was an executive recruiter, and they bought an IBM. And this is, like, when it was DOS before MS-DOS existed. And I was so disappointed, because, like, other kids had the Amiga or the Commodore- … which, you know, they were better for gaming than the IBM at the time. And my mom, she really encouraged my brother and I. She bought Zork. You know, it was just Infocom word games, and where your imagination would take you. Like, Zork holds a place in my heart I think few games will ever touch.
… in Super that I didn’t know existed at the time. And it’s… it was like suddenly, the world opened up more and games could be more. And then there was, like, a big PC gaming push that hit me. My parents ran their own business. Like I said, my dad was an executive recruiter, and they bought an IBM. And this is, like, when it was DOS before MS-DOS existed. And I was so disappointed, because, like, other kids had the Amiga or the Commodore- … which, you know, they were better for gaming than the IBM at the time. And my mom, she really encouraged my brother and I. She bought Zork. You know, it was just Infocom word games, and where your imagination would take you. Like, Zork holds a place in my heart I think few games will ever touch.
Lex Fridman
It’s a text-based game?
It’s a text-based game?
Jeff Kaplan
Text-based game. You know, you just type in, “Go west. Open mailbox.” You know? And… But it’s that power of imagination. It’s why the book is always better than the movie, you know?
Text-based game. You know, you just type in, “Go west. Open mailbox.” You know? And… But it’s that power of imagination. It’s why the book is always better than the movie, you know?
Lex Fridman
Yeah. So, you’re starting to see these creations of worlds that you can navigate. You can step into this world and you can lose yourself in that world.
Yeah. So, you’re starting to see these creations of worlds that you can navigate. You can step into this world and you can lose yourself in that world.
Jeff Kaplan
Yeah. You’re transported. You’re living there.
Yeah. You’re transported. You’re living there.
Lex Fridman
Was Zork popular?
Was Zork popular?
Jeff Kaplan
Zork was insanely popular. And then there was Zork II and Zork III.
Zork was insanely popular. And then there was Zork II and Zork III.
Lex Fridman
A trilogy. Zork trilogy. I see it. Okay.
A trilogy. Zork trilogy. I see it. Okay.
Jeff Kaplan
And it was weird, and, like, the… Sometime in the ’90s, there was this, there was this era of what they called CD-ROM games. That’s how they branded them. And they made a return to Zork, but it now had graphics. And somehow, that just shattered everything, because the Zork you knew in your head didn’t exist anymore. Yeah, Zork was fantastic. I think it might be open source now, which I think is fabulous. But I highly recommend Zork. There was also, in those days, on the PC that worked on our IBM, was Ultima-
And it was weird, and, like, the… Sometime in the ’90s, there was this, there was this era of what they called CD-ROM games. That’s how they branded them. And they made a return to Zork, but it now had graphics. And somehow, that just shattered everything, because the Zork you knew in your head didn’t exist anymore. Yeah, Zork was fantastic. I think it might be open source now, which I think is fabulous. But I highly recommend Zork. There was also, in those days, on the PC that worked on our IBM, was Ultima-
Jeff Kaplan
… which was the Richard Garriott series. And he was Lord British. We knew him as Lord British. He put himself in the game. And you want to talk about world-building. You know, there was Yew Forest and there was all the characters. And the first Ultima I played was Ultima II, ’cause Ultima I was before my time. And that series, it was this RPG group-based PC game, and the worlds were just so rich. Like, you could get on a rocket ship. You’re playing in this fantasy world, fighting demons, and yet somehow you could get on a rocket ship. And then there was just all of this sort of crazy stuff that would happen in games that are based in the world.
… which was the Richard Garriott series. And he was Lord British. We knew him as Lord British. He put himself in the game. And you want to talk about world-building. You know, there was Yew Forest and there was all the characters. And the first Ultima I played was Ultima II, ’cause Ultima I was before my time. And that series, it was this RPG group-based PC game, and the worlds were just so rich. Like, you could get on a rocket ship. You’re playing in this fantasy world, fighting demons, and yet somehow you could get on a rocket ship. And then there was just all of this sort of crazy stuff that would happen in games that are based in the world.
Jeff Kaplan
Like, there were bouncers in the towns, and merchants, but if you really wanted to, you could try to rob these people, or kill Lord British, you know? That was something that was super hard. And when you’re just a jackass kid, you spend your time endlessly trying to do these things over and over, and Ultima was really a profound kind of experience for me.
Like, there were bouncers in the towns, and merchants, but if you really wanted to, you could try to rob these people, or kill Lord British, you know? That was something that was super hard. And when you’re just a jackass kid, you spend your time endlessly trying to do these things over and over, and Ultima was really a profound kind of experience for me.
Lex Fridman
And, of course, that led to Ultima Online, which is a legendary game in itself, perhaps connected to EverQuest. Sort of starting to build these worlds that are massively multiplayer online video games. Can you take me to that journey? Like, as you started to get online, the MMO world. What were influential? What were fun for you?
And, of course, that led to Ultima Online, which is a legendary game in itself, perhaps connected to EverQuest. Sort of starting to build these worlds that are massively multiplayer online video games. Can you take me to that journey? Like, as you started to get online, the MMO world. What were influential? What were fun for you?
Jeff Kaplan
Well, the big one for me was EverQuest. But like you mentioned, Ultima Online sort of was the predecessor. It came before EverQuest. And it was, like, one of those unfortunate times in my life where I was actually at grad school.
Well, the big one for me was EverQuest. But like you mentioned, Ultima Online sort of was the predecessor. It came before EverQuest. And it was, like, one of those unfortunate times in my life where I was actually at grad school.
Lex Fridman
You were busy.
You were busy.
Jeff Kaplan
I was busy, and I missed Ultima Online. Like, I would have had that experience. And when you hear the Ultima Online stories, they’re some of the craziest, funniest… You know, I know somebody who, they learned how to poison in the game, and then they would poison apples, then leave them on the ground, and somebody else would be adventuring, then feed the apple to their horse and kill their horse. Then they’d steal all their stuff and… You know, Ultima Online was kind of… It was the earliest grief-based experiment. Really, like, when you’re treating the humans like ants in the ant farm. That was kind of Ultima Online.
I was busy, and I missed Ultima Online. Like, I would have had that experience. And when you hear the Ultima Online stories, they’re some of the craziest, funniest… You know, I know somebody who, they learned how to poison in the game, and then they would poison apples, then leave them on the ground, and somebody else would be adventuring, then feed the apple to their horse and kill their horse. Then they’d steal all their stuff and… You know, Ultima Online was kind of… It was the earliest grief-based experiment. Really, like, when you’re treating the humans like ants in the ant farm. That was kind of Ultima Online.
Lex Fridman
So-
So-
Jeff Kaplan
… my first, like, what online gaming, what defined online gaming for me was Quake and Doom and Duke Nukem. You know, it started with Doom and they had a… You could basically LAN. You could network with your friends or you could connect with a modem and hook up with somebody. And that was like a mind-blowing… Just seeing another entity in a video game and saying, “That’s a person on the other side of that.”
… my first, like, what online gaming, what defined online gaming for me was Quake and Doom and Duke Nukem. You know, it started with Doom and they had a… You could basically LAN. You could network with your friends or you could connect with a modem and hook up with somebody. And that was like a mind-blowing… Just seeing another entity in a video game and saying, “That’s a person on the other side of that.”
Jeff Kaplan
That was magical, like, that that moment happened and that person could be in another room or across town from you. And Quake kind of took it to the next level. Like, that’s where everybody knew what they were doing. The systems were more refined. And this Quake community formed with all of these, you know, great websites, mods. The community was divided into … There were two castes of players. The low ping bastards, the LPBs … and then the rest of us, you know. And I remember rolling into Quake matches, you know, on a dial-up modem with a 300 ping connection, and I thought it was the greatest thing ever. And just, just connecting with people. Like I said, the, the websites.
That was magical, like, that that moment happened and that person could be in another room or across town from you. And Quake kind of took it to the next level. Like, that’s where everybody knew what they were doing. The systems were more refined. And this Quake community formed with all of these, you know, great websites, mods. The community was divided into … There were two castes of players. The low ping bastards, the LPBs … and then the rest of us, you know. And I remember rolling into Quake matches, you know, on a dial-up modem with a 300 ping connection, and I thought it was the greatest thing ever. And just, just connecting with people. Like I said, the, the websites.
Jeff Kaplan
To this day, the only gaming website I read- I don’t read any of the news sites anymore, but I read Blue’s News. Which was like, like … Someone actually teased me recently. I linked him a story. I’m like, “Oh, did you hear this new thing’s coming out?” And I sent the link, and they’re like, “Dude, this is from Blue’s News. Like, what time machine did you just step out of?” And a guy named Stephen Heaslip… I’m probably pronouncing his name wrong. I apologize, but it was actually through that site that I learned about EverQuest.
To this day, the only gaming website I read- I don’t read any of the news sites anymore, but I read Blue’s News. Which was like, like … Someone actually teased me recently. I linked him a story. I’m like, “Oh, did you hear this new thing’s coming out?” And I sent the link, and they’re like, “Dude, this is from Blue’s News. Like, what time machine did you just step out of?” And a guy named Stephen Heaslip… I’m probably pronouncing his name wrong. I apologize, but it was actually through that site that I learned about EverQuest.
Jeff Kaplan
They had those programmer .plan updates, the .plan files. And guys like Carmack would … You know, they’d post about what code they were writing or how they had optimized something, or just their personal life. Like, you know, the Ferrari talk would always happen … once they had achieved success. And there was an id programmer named Brian Hook, and he said, “I’m leaving id to go work at Verant,” which became Sony Online, “to work on this game called EverQuest.” And I was like, “How does anybody leave id, the greatest institution in all of gaming ever, to work on any other game?” I’m like, “This guy must be crazy. Or whatever this EverQuest thing is, I need to see it. I need to know what’s going on.” And if he hadn’t have made that post, I never would have checked out EverQuest.
They had those programmer .plan updates, the .plan files. And guys like Carmack would … You know, they’d post about what code they were writing or how they had optimized something, or just their personal life. Like, you know, the Ferrari talk would always happen … once they had achieved success. And there was an id programmer named Brian Hook, and he said, “I’m leaving id to go work at Verant,” which became Sony Online, “to work on this game called EverQuest.” And I was like, “How does anybody leave id, the greatest institution in all of gaming ever, to work on any other game?” I’m like, “This guy must be crazy. Or whatever this EverQuest thing is, I need to see it. I need to know what’s going on.” And if he hadn’t have made that post, I never would have checked out EverQuest.
Lex Fridman
We’ll talk about EverQuest, but since you mentioned Carmack and Quake, what can we say about the genius of John Carmack? Why was he such an important and influential human in the history of gaming?
We’ll talk about EverQuest, but since you mentioned Carmack and Quake, what can we say about the genius of John Carmack? Why was he such an important and influential human in the history of gaming?
Jeff Kaplan
Those early geniuses at id … Like, I wouldn’t be sitting here talking to you right now if they hadn’t had the breakthroughs that they had at the time. Gaming engines were evolving, but the level of breakthrough that they achieved with Wolf 3D, that was the first … I remember playing Wolfenstein when it was a 2D game. You’d run around. You’d dress up as a German. You’d throw a grenade.
Those early geniuses at id … Like, I wouldn’t be sitting here talking to you right now if they hadn’t had the breakthroughs that they had at the time. Gaming engines were evolving, but the level of breakthrough that they achieved with Wolf 3D, that was the first … I remember playing Wolfenstein when it was a 2D game. You’d run around. You’d dress up as a German. You’d throw a grenade.
Jeff Kaplan
To see it in 3D… And it’s funny. You look back at the screenshots or videos of it now, and it seems almost childish. Like, “Oh, why were you so excited about that?” And you were transported. There… It was the intimacy of first person. You know, putting the hands in front of you, holding the gun, being transported to Nazi Germany, but you’re the hero fighting the Nazis. And then the evolution. Like, when Doom came out, I’m a huge Army of Darkness fan. Like, one of my favorite movies of all time. And I was like, “This is Army of Darkness, the video game.” You know? Like, “Give me the boom stick. Here we go.” And the graphical advances… But it wasn’t just how the game looked, it was how it played.
To see it in 3D… And it’s funny. You look back at the screenshots or videos of it now, and it seems almost childish. Like, “Oh, why were you so excited about that?” And you were transported. There… It was the intimacy of first person. You know, putting the hands in front of you, holding the gun, being transported to Nazi Germany, but you’re the hero fighting the Nazis. And then the evolution. Like, when Doom came out, I’m a huge Army of Darkness fan. Like, one of my favorite movies of all time. And I was like, “This is Army of Darkness, the video game.” You know? Like, “Give me the boom stick. Here we go.” And the graphical advances… But it wasn’t just how the game looked, it was how it played.
Jeff Kaplan
The smoothness kept getting better. The responsiveness, the sharpness of the gameplay. You have to credit id in those days and Carmack and Romero. I … As somebody who worked on an FPS, I … That wouldn’t have existed without them. Credit where credit’s due.
The smoothness kept getting better. The responsiveness, the sharpness of the gameplay. You have to credit id in those days and Carmack and Romero. I … As somebody who worked on an FPS, I … That wouldn’t have existed without them. Credit where credit’s due.
Lex Fridman
And by the way, we should say you’re … As a gamer, your range is incredible. You are a legit first-person shooter gamer, but you also obviously love the more MMO world, rich, exploratory kinda game. So it’s fascinating. But yeah, there is … On the technology stack that brought something like Quake or Wolfenstein 3D to life, there’s a threshold which you pass of realism where you can immerse yourself into that world. I had the same exact experience with Wolfenstein 2D taking a step to 3D, and it was like tears in my eyes. Like, “This is incredible.” Like, my memories of Wolfenstein 3D is it was like ultra realistic. It’s silly to say now.
And by the way, we should say you’re … As a gamer, your range is incredible. You are a legit first-person shooter gamer, but you also obviously love the more MMO world, rich, exploratory kinda game. So it’s fascinating. But yeah, there is … On the technology stack that brought something like Quake or Wolfenstein 3D to life, there’s a threshold which you pass of realism where you can immerse yourself into that world. I had the same exact experience with Wolfenstein 2D taking a step to 3D, and it was like tears in my eyes. Like, “This is incredible.” Like, my memories of Wolfenstein 3D is it was like ultra realistic. It’s silly to say now.
Writing career
Lex Fridman
It was the feeling like you were there. Yeah, what an incredible age. And some of that, the storytelling, a lot of that is the technology that brings that kind of 3D world to life. It’s incredible. But before we get too far on that tangent, you mentioned grad school. We should mention that you have a master’s degree in creative writing from NYU, and you wanted to be a writer. You told me your main influences were Kerouac, but also Hemingway, Salinger, Bukowski, Orwell. What drew you to storytelling in that medium of writing? What aspect of the human experience were you trying to put down on paper?
It was the feeling like you were there. Yeah, what an incredible age. And some of that, the storytelling, a lot of that is the technology that brings that kind of 3D world to life. It’s incredible. But before we get too far on that tangent, you mentioned grad school. We should mention that you have a master’s degree in creative writing from NYU, and you wanted to be a writer. You told me your main influences were Kerouac, but also Hemingway, Salinger, Bukowski, Orwell. What drew you to storytelling in that medium of writing? What aspect of the human experience were you trying to put down on paper?
Jeff Kaplan
Well, it started with being a fan first and being inspired and reading, and it’s the, not only being transported to a different world or into a different person, but also, you know, the way that stories can touch emotions in you and trigger feelings sometimes you didn’t even know you had. And that was very appealing for me. And the big challenge with it is, and I think this is for anybody who creates anything, is putting yourself out there. To some degree, there’s a lot of ego that goes into that moment where you say, “Well, I’ve been reading, you know, 1984 or Green Hills of Stranglethorn, and I think it’s amazing. And now I’m gonna try to write something that somebody is gonna read.”
Well, it started with being a fan first and being inspired and reading, and it’s the, not only being transported to a different world or into a different person, but also, you know, the way that stories can touch emotions in you and trigger feelings sometimes you didn’t even know you had. And that was very appealing for me. And the big challenge with it is, and I think this is for anybody who creates anything, is putting yourself out there. To some degree, there’s a lot of ego that goes into that moment where you say, “Well, I’ve been reading, you know, 1984 or Green Hills of Stranglethorn, and I think it’s amazing. And now I’m gonna try to write something that somebody is gonna read.”
Jeff Kaplan
That’s a giant leap of faith. You know, that’s a moment of putting yourself out there completely, and there’s gotta be some part of that that’s ego. There’s some part of it that’s masochistic. And I think for people who want to create and build stuff, they can’t help but to do it. You don’t really have an option. That’s just how you’re wired, and you’re gonna do it anyway. And, you know, I admire people like Dickinson who can just write all the poems and leave them in a drawer to be discovered by somebody else. You know, that’s one way to go about it.
That’s a giant leap of faith. You know, that’s a moment of putting yourself out there completely, and there’s gotta be some part of that that’s ego. There’s some part of it that’s masochistic. And I think for people who want to create and build stuff, they can’t help but to do it. You don’t really have an option. That’s just how you’re wired, and you’re gonna do it anyway. And, you know, I admire people like Dickinson who can just write all the poems and leave them in a drawer to be discovered by somebody else. You know, that’s one way to go about it.
Lex Fridman
Yeah, Franz Kafka, you know, a lot of the stories he wrote, never published, and he asked for all of them to be destroyed. And then it’s only because of his friend that ignored his request that we even have many of his stories. It’s like to be that kinda… I mean, clearly, there’s some masochism there, some tortured soul. But then there’s also the ego like you mentioned. I was entertained by this story of James Joyce when he was a young man, 18, 19 declared that he’s going to be the greatest writer of the 20th century. And he turned out in many, in the eyes of many to be one of the greatest writers of the 20th century. But there’s, like, millions of kids just like James Joyce, writers, they’re declaring exactly that, that turn out not to be.
Yeah, Franz Kafka, you know, a lot of the stories he wrote, never published, and he asked for all of them to be destroyed. And then it’s only because of his friend that ignored his request that we even have many of his stories. It’s like to be that kinda… I mean, clearly, there’s some masochism there, some tortured soul. But then there’s also the ego like you mentioned. I was entertained by this story of James Joyce when he was a young man, 18, 19 declared that he’s going to be the greatest writer of the 20th century. And he turned out in many, in the eyes of many to be one of the greatest writers of the 20th century. But there’s, like, millions of kids just like James Joyce, writers, they’re declaring exactly that, that turn out not to be.
Lex Fridman
But that is in some cases, in many cases, maybe most cases, you have to have that ego- … to say, “I’m gonna…” Yeah, right. “I read 1984,” “and I’m going to write the next 1984.”
But that is in some cases, in many cases, maybe most cases, you have to have that ego- … to say, “I’m gonna…” Yeah, right. “I read 1984,” “and I’m going to write the next 1984.”
Jeff Kaplan
Yeah. And I do think ego is a big part of it. It’s one of the many lessons I’ve learned. Hearing your Kafka story is funny, because fast-forwarding to how my writing career ended- … I literally threw away everything, I mean, in a dumpster. I used to keep copious notes, like journals, my writing journals, everything I ever read, every story idea. I probably had 20 volumes of just handwritten notes. And then I also kept personal journals of just, you know, to keep the writing habit up of just, you know, what happened in my day, how I was feeling, all of that. And then either digitally or typed, I had all of my manuscripts, and I threw it all in the dumpster.
Yeah. And I do think ego is a big part of it. It’s one of the many lessons I’ve learned. Hearing your Kafka story is funny, because fast-forwarding to how my writing career ended- … I literally threw away everything, I mean, in a dumpster. I used to keep copious notes, like journals, my writing journals, everything I ever read, every story idea. I probably had 20 volumes of just handwritten notes. And then I also kept personal journals of just, you know, to keep the writing habit up of just, you know, what happened in my day, how I was feeling, all of that. And then either digitally or typed, I had all of my manuscripts, and I threw it all in the dumpster.
Lex Fridman
What was that decision? Do you remember that decision? What was that like to just take that part of your life and just put it in a dumpster?
What was that decision? Do you remember that decision? What was that like to just take that part of your life and just put it in a dumpster?
Jeff Kaplan
Yeah. It was—I think it was necessary. It was necessary. This is like rationalizing it after the fact, you know, which is easy to do. You know? But at the time, I think I was so broken and so defeated with failure that I needed the moment. It was like throwing in the towel for a boxer, you know? It’s that moment of like, “I’m not gonna win this fight, and you need to move on from it.” And if there was any element of that sitting around, I’d be tempted to try again or bring it out of the drawer 10 years later.
Yeah. It was—I think it was necessary. It was necessary. This is like rationalizing it after the fact, you know, which is easy to do. You know? But at the time, I think I was so broken and so defeated with failure that I needed the moment. It was like throwing in the towel for a boxer, you know? It’s that moment of like, “I’m not gonna win this fight, and you need to move on from it.” And if there was any element of that sitting around, I’d be tempted to try again or bring it out of the drawer 10 years later.
Lex Fridman
We should mention that you did give it a real try. You’ve mentioned receiving over 170 rejection letters in one year when submitting your stories. So there’s a lot of rejection. So it was a long chain of rejection. And then what was that like, the rejection?
We should mention that you did give it a real try. You’ve mentioned receiving over 170 rejection letters in one year when submitting your stories. So there’s a lot of rejection. So it was a long chain of rejection. And then what was that like, the rejection?
Jeff Kaplan
It was hard. I had moved from New York. I did the most terrible dumb thing that I knew I was doing at the time. I had a really great group of writer friends from grad school in New York, and I think writing is a very lonely, solitary thing. But weirdly, writers kind of support each other and just, “Who do you give the story to?” You know, you don’t wanna give it to your mom or dad, you know. You kinda wanna give it to somebody who’s gonna really punch you in the nose and tell you what’s wrong with it. And I had left that writing circle to move back to California.
It was hard. I had moved from New York. I did the most terrible dumb thing that I knew I was doing at the time. I had a really great group of writer friends from grad school in New York, and I think writing is a very lonely, solitary thing. But weirdly, writers kind of support each other and just, “Who do you give the story to?” You know, you don’t wanna give it to your mom or dad, you know. You kinda wanna give it to somebody who’s gonna really punch you in the nose and tell you what’s wrong with it. And I had left that writing circle to move back to California.
Lex Fridman
Did you take a bunch of drugs, take your typewriter and drove across, uh-
Did you take a bunch of drugs, take your typewriter and drove across, uh-
Jeff Kaplan
No.
No.
Lex Fridman
… acro- Across the United States and then wrote a book about it? Or just to take Kerouac as an example. Anyway, sorry. You went just-
… acro- Across the United States and then wrote a book about it? Or just to take Kerouac as an example. Anyway, sorry. You went just-
Jeff Kaplan
I might have been more successful had- … I done that.
I might have been more successful had- … I done that.
Lex Fridman
So sorry. So you went back.
So sorry. So you went back.
Jeff Kaplan
So I moved back to California, and I did it for a girl. And I think within two months of moving back, we were broken up. So… And I knew it when I was standing in my studio apartment when it was empty in New York and I was about to close the door for the last time. I had that, like, you know, little me on the shoulder saying, “Dude, what are you doing?”
So I moved back to California, and I did it for a girl. And I think within two months of moving back, we were broken up. So… And I knew it when I was standing in my studio apartment when it was empty in New York and I was about to close the door for the last time. I had that, like, you know, little me on the shoulder saying, “Dude, what are you doing?”
Jeff Kaplan
“This… You’re making one of those epic life mistakes that is gonna come back to haunt you.” And I ended up alone in California, and I think it was a good three years that I structured my life where I was gonna write for eight hours a day, because it’s that writer’s habit. Like, you have to just force yourself: “This is a job. This isn’t a hobby. Whether I like it or not, rain or shine, sick or healthy, I’m gonna write for eight hours a day.” And I did. I was fortunate. Like I said, my dad had his company and he hired me as a research associate. So I was calling up, generating name lists for a recruiting company, and I would take… Whenever there was East Coast assignments, I would take those so I could start at like 5:00 in the morning.
“This… You’re making one of those epic life mistakes that is gonna come back to haunt you.” And I ended up alone in California, and I think it was a good three years that I structured my life where I was gonna write for eight hours a day, because it’s that writer’s habit. Like, you have to just force yourself: “This is a job. This isn’t a hobby. Whether I like it or not, rain or shine, sick or healthy, I’m gonna write for eight hours a day.” And I did. I was fortunate. Like I said, my dad had his company and he hired me as a research associate. So I was calling up, generating name lists for a recruiting company, and I would take… Whenever there was East Coast assignments, I would take those so I could start at like 5:00 in the morning.
Jeff Kaplan
And I created all this space for me to write, and I just… I had a dog named Jack- … who was… He was a Jack Russell Terrier. And so everybody’s like, “You’re a writer, you named your Jack Russell Terrier Jack.” I’m like, “Because I named him after Jack Kerouac.” “It’s poetic and epic,” and-
And I created all this space for me to write, and I just… I had a dog named Jack- … who was… He was a Jack Russell Terrier. And so everybody’s like, “You’re a writer, you named your Jack Russell Terrier Jack.” I’m like, “Because I named him after Jack Kerouac.” “It’s poetic and epic,” and-
Lex Fridman
Yeah, of course.
Yeah, of course.
Jeff Kaplan
… I just looked like a dumbass, but- … it was just me and this dog. And I was writing, you know, all that time intensely. And this was mid to late ’90s, so even though the internet existed, email was very primitive and you had to send a manuscript off, like printed paper- … to all… Like, I was trying to get short stories published in literary magazines, and you had to send an envelope with a return self-addressed stamp. So it was expensive, too. Like if you didn’t have money, you were just… There was a cost to it- … to every single one of them.
… I just looked like a dumbass, but- … it was just me and this dog. And I was writing, you know, all that time intensely. And this was mid to late ’90s, so even though the internet existed, email was very primitive and you had to send a manuscript off, like printed paper- … to all… Like, I was trying to get short stories published in literary magazines, and you had to send an envelope with a return self-addressed stamp. So it was expensive, too. Like if you didn’t have money, you were just… There was a cost to it- … to every single one of them.
Lex Fridman
You had to pay for the rejection letter that you would eventually receive.
You had to pay for the rejection letter that you would eventually receive.
Jeff Kaplan
Yeah. And the, like, big thing that you were hoping for was that the editor would write you a note with the rejection letter. Like, um-
Yeah. And the, like, big thing that you were hoping for was that the editor would write you a note with the rejection letter. Like, um-
Lex Fridman
Keep going.
Keep going.
Jeff Kaplan
Yeah. And you’d like cling onto this. Like, it was like, “Oh, Glimmer Train said, you know, showing promise.” You know, and you just hang onto that for like a week, you know, pretending like that was… But it was just soul-crushing. And I really stuck… And I became more and more isolated. Part of that was leaving that group of writing friends in New York. I’m prone to just introversion anyway. The type of person I am. Breaking up with the girlfriend at the time. I just sort of fell into that world of, like, all I was doing was writing. And it broke me. Like, I went into very deep and heavy depression. I drank too much. I really had a problem with alcohol. And all those things compounded into just deep, deep depression. And I don’t… There wasn’t like a magic rejection that broke me.
Yeah. And you’d like cling onto this. Like, it was like, “Oh, Glimmer Train said, you know, showing promise.” You know, and you just hang onto that for like a week, you know, pretending like that was… But it was just soul-crushing. And I really stuck… And I became more and more isolated. Part of that was leaving that group of writing friends in New York. I’m prone to just introversion anyway. The type of person I am. Breaking up with the girlfriend at the time. I just sort of fell into that world of, like, all I was doing was writing. And it broke me. Like, I went into very deep and heavy depression. I drank too much. I really had a problem with alcohol. And all those things compounded into just deep, deep depression. And I don’t… There wasn’t like a magic rejection that broke me.
Jeff Kaplan
That would have been epic if like- … someone out there is like, “The dude who…” “I’m the dude who broke Jeff that one day.” But I just had a moment where I said, “This is gonna destroy me.” And… Like, I don’t want to be discouraging to anybody, because I really do believe, like you hear it so much, like, “You have to work for your dreams, never give up.” Like, we’re trained this way. Like, “Never give up.” The universe… Actually, maybe not the universe. A group of editors at literary magazines across the United States was telling me it was time to give up as a writer, like I wasn’t cut out for it. And I stopped.
That would have been epic if like- … someone out there is like, “The dude who…” “I’m the dude who broke Jeff that one day.” But I just had a moment where I said, “This is gonna destroy me.” And… Like, I don’t want to be discouraging to anybody, because I really do believe, like you hear it so much, like, “You have to work for your dreams, never give up.” Like, we’re trained this way. Like, “Never give up.” The universe… Actually, maybe not the universe. A group of editors at literary magazines across the United States was telling me it was time to give up as a writer, like I wasn’t cut out for it. And I stopped.
Lex Fridman
Sometimes, you know, closing a door is required for another door to open.
Sometimes, you know, closing a door is required for another door to open.
Jeff Kaplan
Yes.
Yes.
Lex Fridman
That’s one of the hardest things to do, is to walk away.
That’s one of the hardest things to do, is to walk away.
Jeff Kaplan
Yeah. And I think, rightly so, our parents, our coaches, our mentors train us not to give up. And I think a lot of us take pride in that, “I’m never gonna give up. I’m gonna do this come hell or high water.” And sometimes there’s that reality, especially when you’re now in your mid-20s, where you have that moment of like, “Am I really gonna be this? Like, am I ever gonna sort of find the light here?” And, maybe, and it’s so hard, it’s so hard to have this moment, “Maybe this isn’t my calling in life,” especially when you don’t know what the next calling is gonna be.
Yeah. And I think, rightly so, our parents, our coaches, our mentors train us not to give up. And I think a lot of us take pride in that, “I’m never gonna give up. I’m gonna do this come hell or high water.” And sometimes there’s that reality, especially when you’re now in your mid-20s, where you have that moment of like, “Am I really gonna be this? Like, am I ever gonna sort of find the light here?” And, maybe, and it’s so hard, it’s so hard to have this moment, “Maybe this isn’t my calling in life,” especially when you don’t know what the next calling is gonna be.
Lex Fridman
That’s so painful. It’s ’cause you’ve invested so much of yourself, of who you are, of the dreams you’ve had, of this just whole conception of yourself, and you’re watching yourself slide down in terms of becoming isolated, suffering more and more. And then you just have to somehow figure out how to get out of that. And it is true. In that situation, the way to get out is the dumpster. Is to cut it off. Is there advice you can extract from that? There’s a lot of young folks who are in that same situation.
That’s so painful. It’s ’cause you’ve invested so much of yourself, of who you are, of the dreams you’ve had, of this just whole conception of yourself, and you’re watching yourself slide down in terms of becoming isolated, suffering more and more. And then you just have to somehow figure out how to get out of that. And it is true. In that situation, the way to get out is the dumpster. Is to cut it off. Is there advice you can extract from that? There’s a lot of young folks who are in that same situation.
Jeff Kaplan
Yeah. I, this is one of those hindsight things where, you know, having gone through it and ended up okay on the other side, which you don’t know at the time, you know? When you’re a young person in your late teens or early 20s, there’s so much pressure on you. And I really think adults don’t help. You know? Every time you run into the younger nephew or whoever and you start to say things like, “Oh, what’s your major? What are you gonna do with that?” “What do you wanna be?” It’s such bullshit to do to a human being. You know?
Yeah. I, this is one of those hindsight things where, you know, having gone through it and ended up okay on the other side, which you don’t know at the time, you know? When you’re a young person in your late teens or early 20s, there’s so much pressure on you. And I really think adults don’t help. You know? Every time you run into the younger nephew or whoever and you start to say things like, “Oh, what’s your major? What are you gonna do with that?” “What do you wanna be?” It’s such bullshit to do to a human being. You know?
Lex Fridman
You’re so lost in the world. I mean, most of us are lost our entire lives, but especially in your 20s, you know, like, you’re lost. So the questions like, yeah, “What are you doing? What’s your major? What’s the career?” And so on, that’s not the point, man. I’m trying to move through the world, I’m trying to run through the world to find the thing that sparks my heart, to find the passion, to find what I’m meant to be on this earth for. And there are really, I mean, that is a real hero’s journey of searching as a young person. That’s a real, like, you know, all the adults, with their wisdom, they’ve stopped searching often. They’ve done the lazy, the comfortable thing. They found their thing.
You’re so lost in the world. I mean, most of us are lost our entire lives, but especially in your 20s, you know, like, you’re lost. So the questions like, yeah, “What are you doing? What’s your major? What’s the career?” And so on, that’s not the point, man. I’m trying to move through the world, I’m trying to run through the world to find the thing that sparks my heart, to find the passion, to find what I’m meant to be on this earth for. And there are really, I mean, that is a real hero’s journey of searching as a young person. That’s a real, like, you know, all the adults, with their wisdom, they’ve stopped searching often. They’ve done the lazy, the comfortable thing. They found their thing.
Lex Fridman
And so now they look back, they don’t remember how much suffering and how much uncertainty that young people have to deal with.
And so now they look back, they don’t remember how much suffering and how much uncertainty that young people have to deal with.
Jeff Kaplan
It’s, there’s confusion, there’s pressure. Like, the pressure we exert on younger people for having it figured out is insane. So the advice that I always give, and it sounds so stupid, like this sounds really trite, but focus on what you wanna do, not what you wanna be. The pressure that society kind of puts on us is, you know, “Oh, do you wanna be an astronaut? Do you wanna be a firefighter? Do you wanna be a writer? Do you wanna be a game maker?” And I think we get lost in the trappings of, like a vision of what that role is- … and how to perform as a fake actor in that role. Versus when you’re off the clock and no one’s asking you any questions-
It’s, there’s confusion, there’s pressure. Like, the pressure we exert on younger people for having it figured out is insane. So the advice that I always give, and it sounds so stupid, like this sounds really trite, but focus on what you wanna do, not what you wanna be. The pressure that society kind of puts on us is, you know, “Oh, do you wanna be an astronaut? Do you wanna be a firefighter? Do you wanna be a writer? Do you wanna be a game maker?” And I think we get lost in the trappings of, like a vision of what that role is- … and how to perform as a fake actor in that role. Versus when you’re off the clock and no one’s asking you any questions-
Jeff Kaplan
… you, you know, you’re not at Thanksgiving dinner and your uncle’s pressuring you into, you know, what your future’s gonna be for the rest of your life. When you go home, how do you spend your time? Like, what makes you happy? What brings you fulfillment? And through those paths, you’re gonna find out what you’re gonna become, not what you wanna be. It’s, “What do you wanna do?”
… you, you know, you’re not at Thanksgiving dinner and your uncle’s pressuring you into, you know, what your future’s gonna be for the rest of your life. When you go home, how do you spend your time? Like, what makes you happy? What brings you fulfillment? And through those paths, you’re gonna find out what you’re gonna become, not what you wanna be. It’s, “What do you wanna do?”
Lex Fridman
What do you wanna do? The thing that brings you joy on a moment by moment basis. Yeah. That’s brilliantly put. And speaking of which, that’s where you took the pivot. You switched to video games. How did that happen? Gradually? Suddenly?
What do you wanna do? The thing that brings you joy on a moment by moment basis. Yeah. That’s brilliantly put. And speaking of which, that’s where you took the pivot. You switched to video games. How did that happen? Gradually? Suddenly?
EverQuest obsession
Jeff Kaplan
Gradually and suddenly. So when I had that fateful moment where I just sort of gave up with writing, I had these days where I’d structure eight-hour chunks of just, this was writing time, you know? I’d sit solitary typing. All that was gone. And, you know, I could still support myself, which was nice. And then I had this free time and I wasn’t spending it with anybody, I was just alone. Me and the dog, Jack. And I just poured it all into EverQuest. You know, I, it was 1999 when that game came out. And I had a friend, Victor, like kind of a lifelong friend. One of the few friends I had who played computer games, ’cause there was a stigma to that.
Gradually and suddenly. So when I had that fateful moment where I just sort of gave up with writing, I had these days where I’d structure eight-hour chunks of just, this was writing time, you know? I’d sit solitary typing. All that was gone. And, you know, I could still support myself, which was nice. And then I had this free time and I wasn’t spending it with anybody, I was just alone. Me and the dog, Jack. And I just poured it all into EverQuest. You know, I, it was 1999 when that game came out. And I had a friend, Victor, like kind of a lifelong friend. One of the few friends I had who played computer games, ’cause there was a stigma to that.
Jeff Kaplan
You know? It wasn’t, you didn’t walk around telling people you played games. They thought you wasted your time. And my friend, Vic, had bought EverQuest. I’m like, “That’s that game that that guy Brian Hook went to work on. Is it good?” And he’s like, “Yeah, you gotta play it.” And the moment I logged in, I was just transported. It was the world of Norrath. And it wasn’t just the world itself and how it looked, I thought the game was gorgeous, it was the mechanics, you know, that I was this halfling rogue that, you know, had to go out and adventure in the world, and when I killed stuff, I got experience, and I needed better loot to kill more stuff to get more experience. And the sort of draw of progression in the game was amazing.
You know? It wasn’t, you didn’t walk around telling people you played games. They thought you wasted your time. And my friend, Vic, had bought EverQuest. I’m like, “That’s that game that that guy Brian Hook went to work on. Is it good?” And he’s like, “Yeah, you gotta play it.” And the moment I logged in, I was just transported. It was the world of Norrath. And it wasn’t just the world itself and how it looked, I thought the game was gorgeous, it was the mechanics, you know, that I was this halfling rogue that, you know, had to go out and adventure in the world, and when I killed stuff, I got experience, and I needed better loot to kill more stuff to get more experience. And the sort of draw of progression in the game was amazing.
Jeff Kaplan
I, and I just lived my life of, “I can’t wait ’til the next time I log in.” There was a lot of escapism going. It wasn’t all healthy. When all was said and done, when I finally had quit EverQuest three days later, you could type in the command /played to see how much played time you had. I had, I think it was like 272 played days in three years. So you start to do the math on like, how much time- … in those three years I was living in that world. It was… it was kind of insane.
I, and I just lived my life of, “I can’t wait ’til the next time I log in.” There was a lot of escapism going. It wasn’t all healthy. When all was said and done, when I finally had quit EverQuest three days later, you could type in the command /played to see how much played time you had. I had, I think it was like 272 played days in three years. So you start to do the math on like, how much time- … in those three years I was living in that world. It was… it was kind of insane.
Lex Fridman
Well, that’s over 6,000 hours- … of gameplay. Wow. So here going to Perplexity, EverQuest is a long-running 3D fantasy, massively multiplayer online role-playing game, MMORPG, set in the world of Norrath, as you were saying. First released in March 1999, it is an online role-playing game where thousands of players create characters, group up, and explore a persistent shared world. It’s widely regarded as one of the foundational MMORPGs, helping define raid content, guild systems and 3D online worlds. That’s the other component of it. There’s… It’s all humans and they group up- … and they raid together in the game.
Well, that’s over 6,000 hours- … of gameplay. Wow. So here going to Perplexity, EverQuest is a long-running 3D fantasy, massively multiplayer online role-playing game, MMORPG, set in the world of Norrath, as you were saying. First released in March 1999, it is an online role-playing game where thousands of players create characters, group up, and explore a persistent shared world. It’s widely regarded as one of the foundational MMORPGs, helping define raid content, guild systems and 3D online worlds. That’s the other component of it. There’s… It’s all humans and they group up- … and they raid together in the game.
Jeff Kaplan
Yep. In the context of EverQuest, raiding is usually around 30 people or more getting together to conquer something that you couldn’t beat otherwise. And to do successful raiding, you usually needed to join what in EverQuest everyone referred to as an Uber Guild. So I had this great pride in my EverQuest journey that I… Most of the time leveling up I was unguilded or I was in like a role-playing guild with rogues only. And it was when I got to Level 50 in EverQuest, which was the top level, I got invited into this guild called Legacy of Steel, which on our server was the top. Every server had a top guild.
Yep. In the context of EverQuest, raiding is usually around 30 people or more getting together to conquer something that you couldn’t beat otherwise. And to do successful raiding, you usually needed to join what in EverQuest everyone referred to as an Uber Guild. So I had this great pride in my EverQuest journey that I… Most of the time leveling up I was unguilded or I was in like a role-playing guild with rogues only. And it was when I got to Level 50 in EverQuest, which was the top level, I got invited into this guild called Legacy of Steel, which on our server was the top. Every server had a top guild.
Jeff Kaplan
And I was on a server called The Nameless Server, and the top guild was Legacy of Steel. And that, the thrill of getting 30 people together to go see if you could beat, you know, Nagafen, who was the fire dragon, or Vox, who was the frost dragon, and needing perfect coordination to pull it off, it was insane how fun. Like, you would literally scream out. You’re alone in your room at home- … but you felt like you were there with these people and you would audibly cheer out when you won, and you’d feel depressed when you lost, and it was a game of high highs and low lows, and it did everything right. It was amazing.
And I was on a server called The Nameless Server, and the top guild was Legacy of Steel. And that, the thrill of getting 30 people together to go see if you could beat, you know, Nagafen, who was the fire dragon, or Vox, who was the frost dragon, and needing perfect coordination to pull it off, it was insane how fun. Like, you would literally scream out. You’re alone in your room at home- … but you felt like you were there with these people and you would audibly cheer out when you won, and you’d feel depressed when you lost, and it was a game of high highs and low lows, and it did everything right. It was amazing.
Lex Fridman
So that was a big leap for you to go from the proud lone warrior to a member of a guild, an Uber Guild. And then there’s that epic story of you rising to the top to become the leader of this Uber Guild.
So that was a big leap for you to go from the proud lone warrior to a member of a guild, an Uber Guild. And then there’s that epic story of you rising to the top to become the leader of this Uber Guild.
Jeff Kaplan
The leader… Yeah. So organizing people in an online game like EverQuest is like herding cats- … ’cause, you know, everyone has their own will. Some people are loot motivated, some people want the guild to do well, some people are just lonely and want people to hang out with. And there was also a lot of depression in the EverQuest community. It was something I suffered with, but a lot of people, you know, anytime you’re feeling sad or down, you’re looking for escape. And one of the great things video games brings us is escapism. And escapism isn’t always bad or negative- … but when you sort of abuse it to escape your real life problems, it’s bad and negative.
The leader… Yeah. So organizing people in an online game like EverQuest is like herding cats- … ’cause, you know, everyone has their own will. Some people are loot motivated, some people want the guild to do well, some people are just lonely and want people to hang out with. And there was also a lot of depression in the EverQuest community. It was something I suffered with, but a lot of people, you know, anytime you’re feeling sad or down, you’re looking for escape. And one of the great things video games brings us is escapism. And escapism isn’t always bad or negative- … but when you sort of abuse it to escape your real life problems, it’s bad and negative.
Lex Fridman
So there’s a mix of pain and darkness that pain can manifest as- … all part of this community.
So there’s a mix of pain and darkness that pain can manifest as- … all part of this community.
Jeff Kaplan
Yeah. And what’s weird is you enter the cycle where being with other people gives you comradery and relief and makes you feel like you’re not doing so bad in life, but you can quickly enter a cycle of… But then you’re withdrawing from life and it makes you feel that way more to where you can only get the fix from the game at that point. So it’s… Psychologically, there’s a lot going on there.
Yeah. And what’s weird is you enter the cycle where being with other people gives you comradery and relief and makes you feel like you’re not doing so bad in life, but you can quickly enter a cycle of… But then you’re withdrawing from life and it makes you feel that way more to where you can only get the fix from the game at that point. So it’s… Psychologically, there’s a lot going on there.
Lex Fridman
And so you had to work with all of that. You have to get a bunch of people together to do a raid, who are all human beings going through complicated psychological journeys of their own. Some are talking shit, some are just quietly lonely, just looking for some loot.
And so you had to work with all of that. You have to get a bunch of people together to do a raid, who are all human beings going through complicated psychological journeys of their own. Some are talking shit, some are just quietly lonely, just looking for some loot.
Jeff Kaplan
In the late ’90s, everyone was talking shit. You know what I mean? Like, the gaming culture was just a different thing back then. But it was a great group. It was super fun. It was people from all walks of life. And to coordinate these people, like you just had to repeat everything like 200 times. Like, “Okay, we’re gonna port from North Ro. Everybody get to North Ro.” And then you’d have to repeat that for like six hours-
In the late ’90s, everyone was talking shit. You know what I mean? Like, the gaming culture was just a different thing back then. But it was a great group. It was super fun. It was people from all walks of life. And to coordinate these people, like you just had to repeat everything like 200 times. Like, “Okay, we’re gonna port from North Ro. Everybody get to North Ro.” And then you’d have to repeat that for like six hours-
Jeff Kaplan
… to have any chance of like 20% of the people showing up in North Ro. And I sort of like… At first I joined the guild, I was just like the bright-eyed, bushy-tailed. Like, I was like one of the few rogues in the guild. I just wanted to be helpful. I really admired the people running the guild. Like, we had a great guild leader. And it was just a really fun experience. And, you know, the guild leader one day just disappeared. Like, he quit and he was going through, you know, his own thing, and that’s what would happen in EverQuest. Like, people would just kinda disappear all of a sudden. There wasn’t a, “Hey, in about a month, I’m gonna stop playing because I’m starting this new job.”
… to have any chance of like 20% of the people showing up in North Ro. And I sort of like… At first I joined the guild, I was just like the bright-eyed, bushy-tailed. Like, I was like one of the few rogues in the guild. I just wanted to be helpful. I really admired the people running the guild. Like, we had a great guild leader. And it was just a really fun experience. And, you know, the guild leader one day just disappeared. Like, he quit and he was going through, you know, his own thing, and that’s what would happen in EverQuest. Like, people would just kinda disappear all of a sudden. There wasn’t a, “Hey, in about a month, I’m gonna stop playing because I’m starting this new job.”
Jeff Kaplan
People had to quit in some dramatic way, where they just disappear, and basically, our guild leader stopped playing.
People had to quit in some dramatic way, where they just disappear, and basically, our guild leader stopped playing.
Lex Fridman
Did you miss them when they disappeared? Like, we, we should say that most of the people, maybe all of them, were anonymous. So you just- …have a username, and you don’t really say who you are in real life.
Did you miss them when they disappeared? Like, we, we should say that most of the people, maybe all of them, were anonymous. So you just- …have a username, and you don’t really say who you are in real life.
Jeff Kaplan
Absolutely. In those days, there was a great stigma to mentioning your, any real-life info. You just kind of kept it all really close to your chest, and you never knew who was male or female. You kind of assumed everybody was male.
Absolutely. In those days, there was a great stigma to mentioning your, any real-life info. You just kind of kept it all really close to your chest, and you never knew who was male or female. You kind of assumed everybody was male.
Lex Fridman
Safe assumption.
Safe assumption.
Jeff Kaplan
And then it was a surprise if they were actually female. Like my wife, for example, that’s how I met her.
And then it was a surprise if they were actually female. Like my wife, for example, that’s how I met her.
Lex Fridman
You met her in EverQuest?
You met her in EverQuest?
Jeff Kaplan
I met her in EverQuest.
I met her in EverQuest.
Lex Fridman
That is a true love story, right there.
That is a true love story, right there.
Jeff Kaplan
Yeah. Yeah. The funny part for me with EverQuest is, you know, you play a game as much as I played EverQuest, and people are like, “You threw years of your life away.” Like, “You can’t win a game like that.” And I’m like, “I don’t know, like, sitting here today, my whole career and my family are thanks to EverQuest, so I think I won the game.”
Yeah. Yeah. The funny part for me with EverQuest is, you know, you play a game as much as I played EverQuest, and people are like, “You threw years of your life away.” Like, “You can’t win a game like that.” And I’m like, “I don’t know, like, sitting here today, my whole career and my family are thanks to EverQuest, so I think I won the game.”
Lex Fridman
Yeah, yeah. You’re like the “Well, actually…” guy.
Yeah, yeah. You’re like the “Well, actually…” guy.
Jeff Kaplan
Well, yeah, exactly.
Well, yeah, exactly.
Lex Fridman
Your life will be on the Wikipedia page somewhere that says, “Well, here’s an example of somebody-” “… why video games are awesome.” Yeah, I mean, some of it… I should mention this as an aside. For me and many people I know, yes, it’s hundreds of hours, but some of the happiest hours and days of my life. Like, looking back, it all worked out. During it, you are pretty low, and you think, “What am I doing with my life?” All that kind of stuff.
Your life will be on the Wikipedia page somewhere that says, “Well, here’s an example of somebody-” “… why video games are awesome.” Yeah, I mean, some of it… I should mention this as an aside. For me and many people I know, yes, it’s hundreds of hours, but some of the happiest hours and days of my life. Like, looking back, it all worked out. During it, you are pretty low, and you think, “What am I doing with my life?” All that kind of stuff.
Lex Fridman
But, like, looking back, just the all-nighters you pull playing a particular video game, allowing yourself to really fully be immersed, seeing the sun come up—and by the way, many of those games, for me, were Blizzard games. It’s just an incredible thing that video games have been able to do. I think, you know, it used to be, and still is somewhat the case, that books do that same kind of thing. They- …they take you on a journey. But video games, for a long time, you’re right, they had a stigma. Like, I couldn’t tell people. I felt like I was doing, like, heroin or something. Like, I felt like I was doing this secret, dark thing. It usually is in the dark. There’s just a secretive nature to it, like I’m doing something really dark and shady.
But, like, looking back, just the all-nighters you pull playing a particular video game, allowing yourself to really fully be immersed, seeing the sun come up—and by the way, many of those games, for me, were Blizzard games. It’s just an incredible thing that video games have been able to do. I think, you know, it used to be, and still is somewhat the case, that books do that same kind of thing. They- …they take you on a journey. But video games, for a long time, you’re right, they had a stigma. Like, I couldn’t tell people. I felt like I was doing, like, heroin or something. Like, I felt like I was doing this secret, dark thing. It usually is in the dark. There’s just a secretive nature to it, like I’m doing something really dark and shady.
Jeff Kaplan
It wasn’t mainstream.
It wasn’t mainstream.
Lex Fridman
It wasn’t.
It wasn’t.
Jeff Kaplan
It wasn’t accept-… There was a stigma to it. And one of the weirdest parts of that is, you know, I mentioned, like, you could type in the /played in EverQuest. Well, if you did the /played on how much TV people watch, what would that look like? It would blow- …6,000 hours out of the water, easily. Well, it… 20 years ago it would have, you know? Not today.
It wasn’t accept-… There was a stigma to it. And one of the weirdest parts of that is, you know, I mentioned, like, you could type in the /played in EverQuest. Well, if you did the /played on how much TV people watch, what would that look like? It would blow- …6,000 hours out of the water, easily. Well, it… 20 years ago it would have, you know? Not today.
Lex Fridman
Now it’s the phone, yeah. Yeah. But then, it is hard to say goodbye to that world. Those are also really painful times. How hard was it to say goodbye for you?
Now it’s the phone, yeah. Yeah. But then, it is hard to say goodbye to that world. Those are also really painful times. How hard was it to say goodbye for you?
Jeff Kaplan
To EverQuest? It was really hard. And there were times where you try to quit.
To EverQuest? It was really hard. And there were times where you try to quit.
Lex Fridman
Oh, you took a break sometimes?
Oh, you took a break sometimes?
Jeff Kaplan
Yeah. You think you’re quitting for good. You’d have those moments of, like, “I’m doing this too much. I need to move on in life. I’m gonna put it down and walk away, and hopefully not come back.” And there were times where you did come back. When I finally did leave EverQuest, it was actually extremely easy, because I was psychologically done with the game at the time. It was not shortly, but not too long after a new expansion had come out. At the time, it was Shadows of Luclin.
Yeah. You think you’re quitting for good. You’d have those moments of, like, “I’m doing this too much. I need to move on in life. I’m gonna put it down and walk away, and hopefully not come back.” And there were times where you did come back. When I finally did leave EverQuest, it was actually extremely easy, because I was psychologically done with the game at the time. It was not shortly, but not too long after a new expansion had come out. At the time, it was Shadows of Luclin.
Jeff Kaplan
Which didn’t speak to me like the expansions before. Like, the one before that was called Scars of Velious, which was an amazing expansion. And I had gotten the job at Blizzard, and I guess I’m just an obsessive person. So all the time and energy that I had put into EverQuest, the second, you know, the second, my first minute started at Blizzard, that was my new obsession.
Which didn’t speak to me like the expansions before. Like, the one before that was called Scars of Velious, which was an amazing expansion. And I had gotten the job at Blizzard, and I guess I’m just an obsessive person. So all the time and energy that I had put into EverQuest, the second, you know, the second, my first minute started at Blizzard, that was my new obsession.
Getting hired at Blizzard
Lex Fridman
So speaking of which, you have to tell the epic origin story of how you got the job at Blizzard. As we said, you were this legendary gamer, and now legendary troll, on EverQuest. Username, Tigole. You gave a lot of edgy feedback to the devs, telling them in now famous… There’s several rants. There’s a famous one where you tell many of them to do a bunch of things, including to pull their heads out of their asses. You were loved and respected because you gave a lot of specific ways that the game could be improved. And that’s an important thing to say. You weren’t just talking shit. You actually really loved and cared for the game, and you gave them, in the language of the time, advice on how to improve their game.
So speaking of which, you have to tell the epic origin story of how you got the job at Blizzard. As we said, you were this legendary gamer, and now legendary troll, on EverQuest. Username, Tigole. You gave a lot of edgy feedback to the devs, telling them in now famous… There’s several rants. There’s a famous one where you tell many of them to do a bunch of things, including to pull their heads out of their asses. You were loved and respected because you gave a lot of specific ways that the game could be improved. And that’s an important thing to say. You weren’t just talking shit. You actually really loved and cared for the game, and you gave them, in the language of the time, advice on how to improve their game.
Lex Fridman
And it’s funny, because, like, you look back to those messages, it’s inspiring to me. It should be informative and inspiring to a lot of people, because you’re really, legit, full-time talking shit. And now, and you always have been, like, one of the kindest, most loved human beings in the entire gaming industry. Anyway, how did that lead to you getting a job at Blizzard?
And it’s funny, because, like, you look back to those messages, it’s inspiring to me. It should be informative and inspiring to a lot of people, because you’re really, legit, full-time talking shit. And now, and you always have been, like, one of the kindest, most loved human beings in the entire gaming industry. Anyway, how did that lead to you getting a job at Blizzard?
Jeff Kaplan
So when the first guild leader left, Legacy of Steel, the founder… He, he was a guy named… His online name was Dread. That was his name. He left, and our guild was kind of in this listless spin for a while. And eventually, somebody stepped up and took his position as guild leader, and that person’s name was Ariel-
So when the first guild leader left, Legacy of Steel, the founder… He, he was a guy named… His online name was Dread. That was his name. He left, and our guild was kind of in this listless spin for a while. And eventually, somebody stepped up and took his position as guild leader, and that person’s name was Ariel-
Jeff Kaplan
… who was this blonde wood elf warrior female, who always refused to wear a helmet because she thought their character was so pretty, wanted to show their face all the time. So Ariel was a great guild leader for us, and made me like an assistant guild leader, raid leader, officer type in the guild. And over time, Ariel got busier and busier, and, you know, would send me messages like, “Hey, I’m not gonna be online, you know, tomorrow,” or, “I’m not gonna be online tonight. Can you run the raid? Can you run the raid?” And running the raids was very natural for me. And it was my first experience with leadership in my life, of like how do you motivate people? Like, what does motivation look like? What does discipline look like? How do you inspire people?
… who was this blonde wood elf warrior female, who always refused to wear a helmet because she thought their character was so pretty, wanted to show their face all the time. So Ariel was a great guild leader for us, and made me like an assistant guild leader, raid leader, officer type in the guild. And over time, Ariel got busier and busier, and, you know, would send me messages like, “Hey, I’m not gonna be online, you know, tomorrow,” or, “I’m not gonna be online tonight. Can you run the raid? Can you run the raid?” And running the raids was very natural for me. And it was my first experience with leadership in my life, of like how do you motivate people? Like, what does motivation look like? What does discipline look like? How do you inspire people?
Jeff Kaplan
When do you force people versus encourage them, you know? So it was a learning experience for me on the fly, and I had the safety net of the real guild leader would log in eventually.
When do you force people versus encourage them, you know? So it was a learning experience for me on the fly, and I had the safety net of the real guild leader would log in eventually.
Lex Fridman
I should mention, I’m just now reading about, doing a bunch of research on Justinian of the Roman Empire, and he rose from being a peasant to being emperor, so I see a lot of parallels in your life journey, from peasant to emperor, but go ahead, I’m sorry.
I should mention, I’m just now reading about, doing a bunch of research on Justinian of the Roman Empire, and he rose from being a peasant to being emperor, so I see a lot of parallels in your life journey, from peasant to emperor, but go ahead, I’m sorry.
Jeff Kaplan
At- at least EverQuest guild leader, that’s- that’s as much-
At- at least EverQuest guild leader, that’s- that’s as much-
Lex Fridman
Uber guilded-
Uber guilded-
Jeff Kaplan
… as I could say.
… as I could say.
Lex Fridman
Uber guild leader.
Uber guild leader.
Jeff Kaplan
Uber guild leader. Best guild on the Nameless server. So as time went on, Ariel became busier and busier, and then one day, they contacted me and we were having this like whisper back and forth, and they said, “You’re gonna have to take over the guild. I’m just too busy.” And then it came out later … Well, let me back up a second. I started fooling around … Like around this time Half-Life 1 had come out, and with both Duke Nukem and Half-Life 1, one of the incredible things that those companies did back in the day was when they shipped the game, they shipped the editor on the CD.
Uber guild leader. Best guild on the Nameless server. So as time went on, Ariel became busier and busier, and then one day, they contacted me and we were having this like whisper back and forth, and they said, “You’re gonna have to take over the guild. I’m just too busy.” And then it came out later … Well, let me back up a second. I started fooling around … Like around this time Half-Life 1 had come out, and with both Duke Nukem and Half-Life 1, one of the incredible things that those companies did back in the day was when they shipped the game, they shipped the editor on the CD.
Jeff Kaplan
And if you were curious enough, you could like fire up that editor and fool around with it. So I made a- a Duke Nukem level, and you’d send it off to like those UK programming magazines, and you know, you’d get excited because your level was in, you know, some random magazine. And then I started making like Half-Life levels. And Ariel had stepped down as guild leader. I had become guild leader.
And if you were curious enough, you could like fire up that editor and fool around with it. So I made a- a Duke Nukem level, and you’d send it off to like those UK programming magazines, and you know, you’d get excited because your level was in, you know, some random magazine. And then I started making like Half-Life levels. And Ariel had stepped down as guild leader. I had become guild leader.
Jeff Kaplan
And then at one point, Ariel contacts me and says, “Hey, you know, you were talking about those Half-Life levels you made. I want to see those.” I’m like, “Oh, that’s cool.” Like, “I didn’t know you played Half-Life.” Like, “Yeah, maybe we can get a server up and I can play them.” And Ariel tells me, “No, mail them to this address in Irvine.” And- because I- again, to rewind in the time machine for a second, to send something like a Half-Life level over the internet would have- … taken like 12 hours. So you actually like burned it onto a CD and stuck it in the mail. So I put my Half-Life levels, I sent them to Ariel, and he says, “You know, my name’s Rob. I’m a designer at Blizzard Entertainment.”
And then at one point, Ariel contacts me and says, “Hey, you know, you were talking about those Half-Life levels you made. I want to see those.” I’m like, “Oh, that’s cool.” Like, “I didn’t know you played Half-Life.” Like, “Yeah, maybe we can get a server up and I can play them.” And Ariel tells me, “No, mail them to this address in Irvine.” And- because I- again, to rewind in the time machine for a second, to send something like a Half-Life level over the internet would have- … taken like 12 hours. So you actually like burned it onto a CD and stuck it in the mail. So I put my Half-Life levels, I sent them to Ariel, and he says, “You know, my name’s Rob. I’m a designer at Blizzard Entertainment.”
Jeff Kaplan
“I hear you’re in Pasadena ’cause you mentioned it.” You know, I would write about, you know, the Rose Parade and all these things on our website. You know, I kind of … It was blogging before blogging existed, so he knew I lived in Pasadena, and he’s like, “Irvine’s only an hour away. Why don’t you come down, see Blizzard, and you can also meet …” and he names like four people in the guild. And I’m like, “They all work at Blizzard too?” He’s like, “Yeah, we’re all Blizzard.” And it was so weird because during that era, I didn’t have a lot of money. It was not like … Kind of nowadays it feels like everybody plays every game, but you had to be selective. So like I never bought StarCraft or Diablo or Warcraft.
“I hear you’re in Pasadena ’cause you mentioned it.” You know, I would write about, you know, the Rose Parade and all these things on our website. You know, I kind of … It was blogging before blogging existed, so he knew I lived in Pasadena, and he’s like, “Irvine’s only an hour away. Why don’t you come down, see Blizzard, and you can also meet …” and he names like four people in the guild. And I’m like, “They all work at Blizzard too?” He’s like, “Yeah, we’re all Blizzard.” And it was so weird because during that era, I didn’t have a lot of money. It was not like … Kind of nowadays it feels like everybody plays every game, but you had to be selective. So like I never bought StarCraft or Diablo or Warcraft.
Jeff Kaplan
I was much more of the Half-Life, Quake, Quake III guy around that time, and I’d never played a Blizzard game, and I just got invited to go to Blizzard Entertainment.
I was much more of the Half-Life, Quake, Quake III guy around that time, and I’d never played a Blizzard game, and I just got invited to go to Blizzard Entertainment.
Lex Fridman
Was Blizzard already legendary, you know, with the Warcraft and StarCraft? Is there … Was it building this like great legend of this game company that seemingly doesn’t miss?
Was Blizzard already legendary, you know, with the Warcraft and StarCraft? Is there … Was it building this like great legend of this game company that seemingly doesn’t miss?
Jeff Kaplan
It was very much on its way to enshrining itself as being one of the legendary game … Like, it was beloved- … by gamers, but there were still ignorant people like me who hadn’t played, you know, War II or Diablo II or StarCraft, which was shocking to people.
It was very much on its way to enshrining itself as being one of the legendary game … Like, it was beloved- … by gamers, but there were still ignorant people like me who hadn’t played, you know, War II or Diablo II or StarCraft, which was shocking to people.
Lex Fridman
So you weren’t like freaking out, freaking out?
So you weren’t like freaking out, freaking out?
Jeff Kaplan
No, I was freaking out in a different sense. I’m like, “Am I gonna get mugged when I-” Like, “Who are … Is this a scam?” Because you didn’t meet people off the internet. So I drove down there. I ended up … There was Rob Pardo- … who at that time was the lead designer on Warcraft III, and he was Ariel. You know, so okay, it wasn’t a woman after all. It wasn’t this blonde wood elf. You know, I don’t know what you expect at that point.
No, I was freaking out in a different sense. I’m like, “Am I gonna get mugged when I-” Like, “Who are … Is this a scam?” Because you didn’t meet people off the internet. So I drove down there. I ended up … There was Rob Pardo- … who at that time was the lead designer on Warcraft III, and he was Ariel. You know, so okay, it wasn’t a woman after all. It wasn’t this blonde wood elf. You know, I don’t know what you expect at that point.
Jeff Kaplan
It was Rob Pardo. To this day, a great friend of mine named Scott Mercer was the enchanter in our EverQuest guild, a guy named Dalomin. There was a guy named Roman Kenney who was like this totally psychotic wizard who played in our guild. And I had lunch with these guys, you know, we just went out to Irvine to like a restaurant. And, you know, forgive me for the misuse of the phrase, but it was like my coming out moment. And we talked about games having that stigma and being embarrassed about who you are and what you like. Like I, up until that point, I would never tell friends, family, like, “I love games. I’m playing this game EverQuest. It’s so cool, we just killed a dragon.” And so you were hiding this part of your identity.
It was Rob Pardo. To this day, a great friend of mine named Scott Mercer was the enchanter in our EverQuest guild, a guy named Dalomin. There was a guy named Roman Kenney who was like this totally psychotic wizard who played in our guild. And I had lunch with these guys, you know, we just went out to Irvine to like a restaurant. And, you know, forgive me for the misuse of the phrase, but it was like my coming out moment. And we talked about games having that stigma and being embarrassed about who you are and what you like. Like I, up until that point, I would never tell friends, family, like, “I love games. I’m playing this game EverQuest. It’s so cool, we just killed a dragon.” And so you were hiding this part of your identity.
Jeff Kaplan
And I’m out to lunch with these guys in Irvine, and we’re talking about dragons and swords and, you know, raid tactics and talking shit on all the people in the guild. And I literally had this moment where I felt like myself for the first time. I just felt so comfortable, and that was an eye-opening moment. And after that, after that lunch happened, he invited me for a couple more lunches down, you know, just… I just saw it as like, “Oh, now, I’m…” You know, I made friends with these people online. Now we know each other in real life, and they happen to work for this game company. And at another one of the lunches, they invite this troll warrior to have lunch with us, whose name in the game was Barfa, the Troll Warrior.
And I’m out to lunch with these guys in Irvine, and we’re talking about dragons and swords and, you know, raid tactics and talking shit on all the people in the guild. And I literally had this moment where I felt like myself for the first time. I just felt so comfortable, and that was an eye-opening moment. And after that, after that lunch happened, he invited me for a couple more lunches down, you know, just… I just saw it as like, “Oh, now, I’m…” You know, I made friends with these people online. Now we know each other in real life, and they happen to work for this game company. And at another one of the lunches, they invite this troll warrior to have lunch with us, whose name in the game was Barfa, the Troll Warrior.
Jeff Kaplan
And Barfa wasn’t somebody who played with us all the time, but kind of like Ariel got into the guild kind of on the side. You know, it was one of those like inside invites of like, “Who’s Barfa?” “I don’t know, but Barfa is in the guild now.” And there was at the time, it was a new dungeon called The Hole, and we had never done it before. And we jumped down in this hole, and we’re doing this whole dungeon, and everything goes wrong, as it’s prone to do in EverQuest. And the whole guild escapes except for Barfa, whose troll character’s so big, he can’t jump out of the exit.
And Barfa wasn’t somebody who played with us all the time, but kind of like Ariel got into the guild kind of on the side. You know, it was one of those like inside invites of like, “Who’s Barfa?” “I don’t know, but Barfa is in the guild now.” And there was at the time, it was a new dungeon called The Hole, and we had never done it before. And we jumped down in this hole, and we’re doing this whole dungeon, and everything goes wrong, as it’s prone to do in EverQuest. And the whole guild escapes except for Barfa, whose troll character’s so big, he can’t jump out of the exit.
Jeff Kaplan
And I hand the potion to Barfa, and I say, “Here, use this. It’ll teleport you out.” And I’m a rogue, I can just stealth and get out of the dungeon on my own. So I saved Barfa, not really knowing who Barfa was, and I did it with a very expensive potion. Mm-hmm. Having lunch, Rob introduced me, “This is Allen Adham. He plays Barfa.” Mm-hmm. I’m like, “Oh, Barfa!” And we, you know, he has a… “You saved me in The Hole that time.” Well, it turns out Allen was the founder of Blizzard, and he was the head… He was sort of the head of everything at that time. It was Allen, Mike Morhaime, and Frank Pearce. And what I didn’t realize what these lunches were, like I just loved them because I felt like I was myself.
And I hand the potion to Barfa, and I say, “Here, use this. It’ll teleport you out.” And I’m a rogue, I can just stealth and get out of the dungeon on my own. So I saved Barfa, not really knowing who Barfa was, and I did it with a very expensive potion. Mm-hmm. Having lunch, Rob introduced me, “This is Allen Adham. He plays Barfa.” Mm-hmm. I’m like, “Oh, Barfa!” And we, you know, he has a… “You saved me in The Hole that time.” Well, it turns out Allen was the founder of Blizzard, and he was the head… He was sort of the head of everything at that time. It was Allen, Mike Morhaime, and Frank Pearce. And what I didn’t realize what these lunches were, like I just loved them because I felt like I was myself.
Jeff Kaplan
I felt true happiness being surrounded by these, you know, people who were talkin’ about video games and I felt comfortable around. And one day, Rob logs into EverQuest. He wasn’t playing much at the time, and he said, “I want you tomorrow to check the Blizzard job site.” Mm-hmm. I’m like, “Okay, like, I’ll check the Blizzard job site.” And they had announced World of Warcraft, and posted on the job site—mm-hmm—was the job for an associate quest designer. And the funniest part of it was, I forget if it was a requirement or a plus in the job description, but they’re like, “We really want somebody with a creative writing degree.” Hmm. And I’m like, “You guys set this up for me.” Like, they were just looking…
I felt true happiness being surrounded by these, you know, people who were talkin’ about video games and I felt comfortable around. And one day, Rob logs into EverQuest. He wasn’t playing much at the time, and he said, “I want you tomorrow to check the Blizzard job site.” Mm-hmm. I’m like, “Okay, like, I’ll check the Blizzard job site.” And they had announced World of Warcraft, and posted on the job site—mm-hmm—was the job for an associate quest designer. And the funniest part of it was, I forget if it was a requirement or a plus in the job description, but they’re like, “We really want somebody with a creative writing degree.” Hmm. And I’m like, “You guys set this up for me.” Like, they were just looking…
Jeff Kaplan
And it was that hindsight moment of like, actually, these guys were just interviewing me for six months. And they were actually friends, and they were really cool about it too. And I just had the fuck it moment like that, that job opened up. I applied with all my heart, you know? Like, they had a bunch of quest writing on it. And then I went through like a pretty hardcore six-month recruiting process because they never hired designers from out of the company. Traditionally, designers were promoted from within Blizzard. Either they would like transfer out of other disciplines, or they would come from quality assurance, tech support. So hiring somebody off the street was kind of a big deal for them, and they really put me through a grilling.
And it was that hindsight moment of like, actually, these guys were just interviewing me for six months. And they were actually friends, and they were really cool about it too. And I just had the fuck it moment like that, that job opened up. I applied with all my heart, you know? Like, they had a bunch of quest writing on it. And then I went through like a pretty hardcore six-month recruiting process because they never hired designers from out of the company. Traditionally, designers were promoted from within Blizzard. Either they would like transfer out of other disciplines, or they would come from quality assurance, tech support. So hiring somebody off the street was kind of a big deal for them, and they really put me through a grilling.
Jeff Kaplan
I met with… It was the first time I met Chris Metzen who is maybe the most inspirational, creative person on the planet. And you instantly… They paired me… They did this interview pairing. There were these two guys. It was Kevin Jordan who was one of the original designers on WoW. Really, he doesn’t get enough credit for his contributions. He was one of the earliest class designers, PvP designers. But he’s a really quiet guy. And they paired him with Chris, and Chris just owns the room, you know? Chris, you could just sit and listen to him. He’s so creative. He’s so passionate. And the way he articulates things, like you just instantly become a fan of Chris when you’re around Chris.
I met with… It was the first time I met Chris Metzen who is maybe the most inspirational, creative person on the planet. And you instantly… They paired me… They did this interview pairing. There were these two guys. It was Kevin Jordan who was one of the original designers on WoW. Really, he doesn’t get enough credit for his contributions. He was one of the earliest class designers, PvP designers. But he’s a really quiet guy. And they paired him with Chris, and Chris just owns the room, you know? Chris, you could just sit and listen to him. He’s so creative. He’s so passionate. And the way he articulates things, like you just instantly become a fan of Chris when you’re around Chris.
Jeff Kaplan
And Chris, Kevin, and I go to lunch at this Italian place that was across the street from Blizzard, and I remember… Chris made a stop to buy cigarettes, you know, on the way to the interview. And then every other word out of Chris’s mouth was like, “Fuck,” and, “Shit.” And I’d come from this whole, like, corporate culture from my dad’s recruiting business, where I’d never imagined somebody would curse in an interview, or stop to buy smokes. And again, it was like, “I’m around my people.” Like, I never smoked, but just, you know, being around people who didn’t care about-
And Chris, Kevin, and I go to lunch at this Italian place that was across the street from Blizzard, and I remember… Chris made a stop to buy cigarettes, you know, on the way to the interview. And then every other word out of Chris’s mouth was like, “Fuck,” and, “Shit.” And I’d come from this whole, like, corporate culture from my dad’s recruiting business, where I’d never imagined somebody would curse in an interview, or stop to buy smokes. And again, it was like, “I’m around my people.” Like, I never smoked, but just, you know, being around people who didn’t care about-
Jeff Kaplan
… what the corporate norms were was so inspiring. And then my last interview was with Allen and Rob, and a great programmer named Bob Fitch. Like, I think he’s one of the first five developers at Blizzard. And they took me to an ARCO station that had a Jack in the Box. You know, how, like- … sometimes they’ll combo? It was like ARCO Jack in the Box. And that was my final interview at Blizzard, was at the ARCO Jack in the Box. And I remember thinking to myself, “These guys just brought me to a Jack in the Box that’s in an ARCO station. I need to work here.” Like, this is… “These are my people.” “This is where I belong.” Like, it was the greatest thing ever. And so, yeah, that’s my crazy journey to Blizzard.
… what the corporate norms were was so inspiring. And then my last interview was with Allen and Rob, and a great programmer named Bob Fitch. Like, I think he’s one of the first five developers at Blizzard. And they took me to an ARCO station that had a Jack in the Box. You know, how, like- … sometimes they’ll combo? It was like ARCO Jack in the Box. And that was my final interview at Blizzard, was at the ARCO Jack in the Box. And I remember thinking to myself, “These guys just brought me to a Jack in the Box that’s in an ARCO station. I need to work here.” Like, this is… “These are my people.” “This is where I belong.” Like, it was the greatest thing ever. And so, yeah, that’s my crazy journey to Blizzard.
Lowest point in Jeff’s life
Lex Fridman
Started at the bottom and end up at the top in a Jack in the Box. Can you speak to… ‘Cause you mentioned some of the low points in the… in depression. Through that journey, how did you find your way out? So, can you just… A lot of people are sitting in those low points right now listening to this. What kind of wisdom can you draw about finding your way out, finding your people?
Started at the bottom and end up at the top in a Jack in the Box. Can you speak to… ‘Cause you mentioned some of the low points in the… in depression. Through that journey, how did you find your way out? So, can you just… A lot of people are sitting in those low points right now listening to this. What kind of wisdom can you draw about finding your way out, finding your people?
Jeff Kaplan
There were a lot of really low points. I started drinking a lot, and alcohol was something that I really wrestled with until my early 30s. And one of the things I’m most proud of today is sobriety and having been sober for such a long time now. And I remember I would like buy a bottle of Old Grand-Dad and- … like, drink the whole thing by myself, and then watch the Oscars. I remember I was … Of all things, I’m watching the Oscars, which is just such a fake, bullshit environment.
There were a lot of really low points. I started drinking a lot, and alcohol was something that I really wrestled with until my early 30s. And one of the things I’m most proud of today is sobriety and having been sober for such a long time now. And I remember I would like buy a bottle of Old Grand-Dad and- … like, drink the whole thing by myself, and then watch the Oscars. I remember I was … Of all things, I’m watching the Oscars, which is just such a fake, bullshit environment.
Jeff Kaplan
But I was like… You know, I was really drunk and all those people seemed so together and successful and polished, and I just… It made me… It was that contrast that made me feel like such a failure. And it all seems so stupid and unimportant to me now. Um, I became… You know, I got in that constant struggle of try not to drink, but drink to make it feel better. I was lucky my parents were very supportive of me, even in my 20s, even after I, you know, quote-unquote left the house. I went into therapy and that was very helpful. You know, extremely helpful. And one thing I learned is that you have to find the right therapist for you.
But I was like… You know, I was really drunk and all those people seemed so together and successful and polished, and I just… It made me… It was that contrast that made me feel like such a failure. And it all seems so stupid and unimportant to me now. Um, I became… You know, I got in that constant struggle of try not to drink, but drink to make it feel better. I was lucky my parents were very supportive of me, even in my 20s, even after I, you know, quote-unquote left the house. I went into therapy and that was very helpful. You know, extremely helpful. And one thing I learned is that you have to find the right therapist for you.
Jeff Kaplan
It’s not just checking a checkbox of, “I went to therapy.” It’s about finding somebody who sort of helps you get out of whatever rut you’re in, in a way that’s healthy for you. And I tried antidepressants, but I hated… I just hated taking pills and feeling like something was in me, and making me feel different. I- I never responded to it. And then the hardest thing, you know, which I’ve never mentioned to anyone, and is- is hard for me to talk about, but eventually I went through ECT, which is electroconvulsive therapy, shock therapy. And that broke me out. And I would never endorse that as a miracle. That was… I was at such a low point that people were very worried about me and my wellbeing-
It’s not just checking a checkbox of, “I went to therapy.” It’s about finding somebody who sort of helps you get out of whatever rut you’re in, in a way that’s healthy for you. And I tried antidepressants, but I hated… I just hated taking pills and feeling like something was in me, and making me feel different. I- I never responded to it. And then the hardest thing, you know, which I’ve never mentioned to anyone, and is- is hard for me to talk about, but eventually I went through ECT, which is electroconvulsive therapy, shock therapy. And that broke me out. And I would never endorse that as a miracle. That was… I was at such a low point that people were very worried about me and my wellbeing-
Jeff Kaplan
… and what was gonna happen, and that was sort of an extreme pull-the-rip-cord, like there’s-nothing-else-to-lose moment. And I think that was the difference maker. That, and starting at Blizzard.
… and what was gonna happen, and that was sort of an extreme pull-the-rip-cord, like there’s-nothing-else-to-lose moment. And I think that was the difference maker. That, and starting at Blizzard.
Lex Fridman
To find… I mean, there is a- there is a deep loneliness there when before you met those guys at lunch, you’re alone, like in a really deep fundamental way. Like, in the way you weren’t in New York with the writing- with the writer’s group, right? And so tha- that must’ve been an incredible experience just to see the guild.
To find… I mean, there is a- there is a deep loneliness there when before you met those guys at lunch, you’re alone, like in a really deep fundamental way. Like, in the way you weren’t in New York with the writing- with the writer’s group, right? And so tha- that must’ve been an incredible experience just to see the guild.
Jeff Kaplan
Yes. It was everything I… As such an introvert, you think that there are extroverts and introverts, and introverts don’t need anybody, but weirdly, I think introverts almost need people more. And we don’t always know how to engage-
Yes. It was everything I… As such an introvert, you think that there are extroverts and introverts, and introverts don’t need anybody, but weirdly, I think introverts almost need people more. And we don’t always know how to engage-
Jeff Kaplan
… in the right, healthy ways, and how to find people and how to connect with people. And it was- it was great. One… The thing that had attracted me to creative writing was the solitude of it, and the fact that you didn’t have to collaborate, and you could just write what you wanted to write and it was all you. You would succeed on your own or you would fail on your own, and that was very attractive to me. And the thought of creative collaboration was actually off-putting. I’d spent all four years of undergrad interning at Universal Pictures, ’cause I thought I wanted to be in film, and it was such an unhealthy creative collaboration in the film industry.
… in the right, healthy ways, and how to find people and how to connect with people. And it was- it was great. One… The thing that had attracted me to creative writing was the solitude of it, and the fact that you didn’t have to collaborate, and you could just write what you wanted to write and it was all you. You would succeed on your own or you would fail on your own, and that was very attractive to me. And the thought of creative collaboration was actually off-putting. I’d spent all four years of undergrad interning at Universal Pictures, ’cause I thought I wanted to be in film, and it was such an unhealthy creative collaboration in the film industry.
Jeff Kaplan
It’s a very, you know, I look up unhealthily to the film industry and admire it and, you know, grew up with all these legends who had come from there. But it’s like a caste system. And I was on the bottom of the caste system as an intern, and I was seeing how the other people who were low caste in the film industry were treated, and it was just horrible, you know. But games was different. Games was very flat. It didn’t matter if you were the CEO or the boss, like, the way Mike and Allen carried themselves with, you know, me, who was an associate game designer, you felt like an equal. And I think it… Not just the camaraderie, but the part that shouldn’t be overlooked is the work itself and the work ethic. That’s what really pulled me out.
It’s a very, you know, I look up unhealthily to the film industry and admire it and, you know, grew up with all these legends who had come from there. But it’s like a caste system. And I was on the bottom of the caste system as an intern, and I was seeing how the other people who were low caste in the film industry were treated, and it was just horrible, you know. But games was different. Games was very flat. It didn’t matter if you were the CEO or the boss, like, the way Mike and Allen carried themselves with, you know, me, who was an associate game designer, you felt like an equal. And I think it… Not just the camaraderie, but the part that shouldn’t be overlooked is the work itself and the work ethic. That’s what really pulled me out.
One of Us
Lex Fridman
Hard work on a thing you love. I have to, if you may allow me, read the prophetic “one of us” quote, “one of us” post you made on April 18th, 2002. Because in some deep sense, you, I think, remained one of us. The… I apologize to bring up Justinian the emperor, but remained a kind of peasant gamer, a true, true gamer, who happens to be also be designing the games. And so this post kind of speaks to that. It’s fascinating to read, because that was at the very beginning, right? You didn’t know anything. You didn’t know the games you would end up creating. Title of the post, “If you want something done right.” He wrote, “This week, I accepted a position as associate game designer with Blizzard Entertainment.
Hard work on a thing you love. I have to, if you may allow me, read the prophetic “one of us” quote, “one of us” post you made on April 18th, 2002. Because in some deep sense, you, I think, remained one of us. The… I apologize to bring up Justinian the emperor, but remained a kind of peasant gamer, a true, true gamer, who happens to be also be designing the games. And so this post kind of speaks to that. It’s fascinating to read, because that was at the very beginning, right? You didn’t know anything. You didn’t know the games you would end up creating. Title of the post, “If you want something done right.” He wrote, “This week, I accepted a position as associate game designer with Blizzard Entertainment.
Lex Fridman
Specifically, I will be designing quests for World of Warcraft, Blizzard’s MMORPG based on the popular Warcraft series. In addition to my duties as quest designer, I will also be expected to contribute to helping design the end game content for World of Warcraft. The reason I’m sharing this information, besides the fact that I have a masochistic love of reading rants and flames about myself, is because I know that the fans of this site are hardcore MMORPG players. The readers of the site have also come to know my personal opinions on what constitutes a fun gaming experience versus what feels like a complete waste of time or poorly designed encounter.” Wow, you’re very eloquent in this post and without too much shit talking.
Specifically, I will be designing quests for World of Warcraft, Blizzard’s MMORPG based on the popular Warcraft series. In addition to my duties as quest designer, I will also be expected to contribute to helping design the end game content for World of Warcraft. The reason I’m sharing this information, besides the fact that I have a masochistic love of reading rants and flames about myself, is because I know that the fans of this site are hardcore MMORPG players. The readers of the site have also come to know my personal opinions on what constitutes a fun gaming experience versus what feels like a complete waste of time or poorly designed encounter.” Wow, you’re very eloquent in this post and without too much shit talking.
Lex Fridman
“You’ve all read my opinions on such things as tedious key camps, obvious time sinks devoid of any story or linear narrative, quests which reward the lucky over the skilled, and quest rewards which are out of sync with the amount of time and effort required to complete them. I hope that my association with World of Warcraft will serve to comfort MMORPG fans that one of us is on the other side of the fence, looking out for the interest of the player.” And you go on to describe some of the high hopes you have for World of Warcraft, which is really fun to read because you don’t realize-
“You’ve all read my opinions on such things as tedious key camps, obvious time sinks devoid of any story or linear narrative, quests which reward the lucky over the skilled, and quest rewards which are out of sync with the amount of time and effort required to complete them. I hope that my association with World of Warcraft will serve to comfort MMORPG fans that one of us is on the other side of the fence, looking out for the interest of the player.” And you go on to describe some of the high hopes you have for World of Warcraft, which is really fun to read because you don’t realize-
Jeff Kaplan
Now-
Now-
Lex Fridman
… it’s gonna be, like, one of the greatest games of all time played by millions of human beings, just where those millions of human beings are playing for hundreds of hours, thousands of hours. It’s crazy. It’s funny that this… one of us is writing at the dawn of a new age. The final paragraph is, “So with all that is going on with me, you’ll have to excuse any lapse in updates to the site here. I will try my hardest to give you slack or something to read while you should be working. But in the meantime, there’s a whole world of NPCs. They need to learn the words kaksagur and mo’fucker, and the like. Although something tells me I’m already in trouble with the boss.” “One of us,” Jeff, “one of us.” That was a beautiful, beautiful post. Did you in fact get in trouble with the boss?
… it’s gonna be, like, one of the greatest games of all time played by millions of human beings, just where those millions of human beings are playing for hundreds of hours, thousands of hours. It’s crazy. It’s funny that this… one of us is writing at the dawn of a new age. The final paragraph is, “So with all that is going on with me, you’ll have to excuse any lapse in updates to the site here. I will try my hardest to give you slack or something to read while you should be working. But in the meantime, there’s a whole world of NPCs. They need to learn the words kaksagur and mo’fucker, and the like. Although something tells me I’m already in trouble with the boss.” “One of us,” Jeff, “one of us.” That was a beautiful, beautiful post. Did you in fact get in trouble with the boss?
Jeff Kaplan
No. No. My boss was Allen. And Allen was very understanding and he… they kind of knew what they were getting into- … when they hired me. And that post actually embarrasses me when I hear it now. There’s so much ego in it- … and I think that’s… it’s got that 20 year old- … you know, “I don’t know what I don’t know.”
No. No. My boss was Allen. And Allen was very understanding and he… they kind of knew what they were getting into- … when they hired me. And that post actually embarrasses me when I hear it now. There’s so much ego in it- … and I think that’s… it’s got that 20 year old- … you know, “I don’t know what I don’t know.”
Lex Fridman
“I know exactly how to fix this video game and all video games and-” But there’s brilliance behind that. There’s a passion behind that. Like, when you’re a gamer and you really put in the hours in a game like EverQuest, you understand what makes for a compelling experience. You don’t, at that time, understand how much hard work is required to create that experience and how much uncertainty there is, how difficult it is, how many trade-offs there are. How your designs, when they actually are brought to the world and are experienced by thousands of people, millions of people, they are different from the vision you had for it. So all those elements you don’t know, but you have to have that ego in the beginning, right?
“I know exactly how to fix this video game and all video games and-” But there’s brilliance behind that. There’s a passion behind that. Like, when you’re a gamer and you really put in the hours in a game like EverQuest, you understand what makes for a compelling experience. You don’t, at that time, understand how much hard work is required to create that experience and how much uncertainty there is, how difficult it is, how many trade-offs there are. How your designs, when they actually are brought to the world and are experienced by thousands of people, millions of people, they are different from the vision you had for it. So all those elements you don’t know, but you have to have that ego in the beginning, right?
Early Blizzard culture
Lex Fridman
Do you even have the guts to try? Do you have the guts to put in all that work? So what were the… what was it like? What were the vibes of early Blizzard like? They’ve… at this point, Warcraft I and II, Warcraft III is in production. StarCraft. These are legendary games. I don’t… I spent probably over 1,000 hours in these games combined. I played Warcraft I, II, III. I played StarCraft I and II. I played WoW, of course. Diablo I, II, III, IV. I played Diablo II with “Stay a while and listen,” with Deckard Cain.
Do you even have the guts to try? Do you have the guts to put in all that work? So what were the… what was it like? What were the vibes of early Blizzard like? They’ve… at this point, Warcraft I and II, Warcraft III is in production. StarCraft. These are legendary games. I don’t… I spent probably over 1,000 hours in these games combined. I played Warcraft I, II, III. I played StarCraft I and II. I played WoW, of course. Diablo I, II, III, IV. I played Diablo II with “Stay a while and listen,” with Deckard Cain.
Jeff Kaplan
“Stay a while and listen.”
“Stay a while and listen.”
Lex Fridman
I mean, some of these characters, some of these experiences just, they’ll stay with me forever. Anyway, so big thank you to those early Blizzard folks. What was it like? What was the team like? What were the developers like? What was the vibes like in those early days?
I mean, some of these characters, some of these experiences just, they’ll stay with me forever. Anyway, so big thank you to those early Blizzard folks. What was it like? What was the team like? What were the developers like? What was the vibes like in those early days?
Jeff Kaplan
It was the dream. When I showed up at Blizzard on my first day, the office was on the University of California Irvine campus at the time. They have this research and development park where, if you’re like a tech company, you can get office space there, and Blizzard took up… When I joined, it was three-fourths of the building was Blizzard, and there were… There was like a building right next to it that had like Cisco and, you know, it was like all kind of techy places.
It was the dream. When I showed up at Blizzard on my first day, the office was on the University of California Irvine campus at the time. They have this research and development park where, if you’re like a tech company, you can get office space there, and Blizzard took up… When I joined, it was three-fourths of the building was Blizzard, and there were… There was like a building right next to it that had like Cisco and, you know, it was like all kind of techy places.
Jeff Kaplan
And it was so funny because you drive up and, like, everything was very serious and corporate, and then outside of the Blizzard offices, it’s everybody is wearing black T-shirt and shorts and throwing frisbees and playing Hacky Sack and on scooters and skateboards, and you’re like, “Okay, that’s where, that’s where Blizzard is.” So it was that environment. I remember walking in the door and thinking like, “It feels like I’m walking into a dorm room-“
And it was so funny because you drive up and, like, everything was very serious and corporate, and then outside of the Blizzard offices, it’s everybody is wearing black T-shirt and shorts and throwing frisbees and playing Hacky Sack and on scooters and skateboards, and you’re like, “Okay, that’s where, that’s where Blizzard is.” So it was that environment. I remember walking in the door and thinking like, “It feels like I’m walking into a dorm room-“
Jeff Kaplan
“… ’cause it was just posters on the wall.” And there were actually, like people would have futons because they’d be sleeping because we would work so much back then. But the vibe was… It was very small. Like Blizzard, the day I joined in May of 2002, was fewer than 200 people, and that included… There was a whole group up in San Mateo called Blizzard North. So Blizzard South, the Irvine group, was responsible for StarCraft and Warcraft, and there were two development teams at Blizzard. It was called Team One and Team Two at Blizzard South. Team One was revered. These are the RTS guys. They made, you know, StarCraft, Warcraft II, and they were, at that time, they were working on Warcraft III.
“… ’cause it was just posters on the wall.” And there were actually, like people would have futons because they’d be sleeping because we would work so much back then. But the vibe was… It was very small. Like Blizzard, the day I joined in May of 2002, was fewer than 200 people, and that included… There was a whole group up in San Mateo called Blizzard North. So Blizzard South, the Irvine group, was responsible for StarCraft and Warcraft, and there were two development teams at Blizzard. It was called Team One and Team Two at Blizzard South. Team One was revered. These are the RTS guys. They made, you know, StarCraft, Warcraft II, and they were, at that time, they were working on Warcraft III.
Jeff Kaplan
Team Two was kind of the red-headed stepchild. Like apparently, before I joined, they had tried to spin off a second team multiple times and failed, and then they finally decided they were gonna make World of Warcraft. There was a game called Nomad. I don’t know what that game was exactly, but that was what Team Two was working on at first. That got scrapped, and Allen steered the team towards World of Warcraft. And there’s an amazing designer named Eric Dodds. He’d go on later in his career to be the game director of Hearthstone. Him and Ben Brode basically were the core designers behind that. But Eric and Kevin Jordan were these two key designers working on World of Warcraft for Team Two, and then you had this tech group that was headed up by John Cash.
Team Two was kind of the red-headed stepchild. Like apparently, before I joined, they had tried to spin off a second team multiple times and failed, and then they finally decided they were gonna make World of Warcraft. There was a game called Nomad. I don’t know what that game was exactly, but that was what Team Two was working on at first. That got scrapped, and Allen steered the team towards World of Warcraft. And there’s an amazing designer named Eric Dodds. He’d go on later in his career to be the game director of Hearthstone. Him and Ben Brode basically were the core designers behind that. But Eric and Kevin Jordan were these two key designers working on World of Warcraft for Team Two, and then you had this tech group that was headed up by John Cash.
Jeff Kaplan
And John Cash, the, the first day that I showed up to work on Team Two, they said, “You have to go get your login from John Cash.” I’m like, “John… The John Cash from id?”
And John Cash, the, the first day that I showed up to work on Team Two, they said, “You have to go get your login from John Cash.” I’m like, “John… The John Cash from id?”
Jeff Kaplan
And you know, John Cash has a skin. You could be John Cash in Quake III. So, and then he saw me, and he was a huge EverQuest player, and he was like, “You’re the guy who runs Legacy of Steel.” I’m like, “You’re John Cash.” We had that moment where we kind of fanboyed out on each other. And it was just… The vibe was so cool there. Like, there were very few producers. So a game team, there are five core disciplines that make a video game. You’ve got engineers or programmers who are writing the code. You’ve got the art team that’s making all the visuals for the game, and that spans everything from like 3D modeling, characters, environments, to also animation, tech art, you know, making it all work.
And you know, John Cash has a skin. You could be John Cash in Quake III. So, and then he saw me, and he was a huge EverQuest player, and he was like, “You’re the guy who runs Legacy of Steel.” I’m like, “You’re John Cash.” We had that moment where we kind of fanboyed out on each other. And it was just… The vibe was so cool there. Like, there were very few producers. So a game team, there are five core disciplines that make a video game. You’ve got engineers or programmers who are writing the code. You’ve got the art team that’s making all the visuals for the game, and that spans everything from like 3D modeling, characters, environments, to also animation, tech art, you know, making it all work.
Jeff Kaplan
You’ve got game design, which some companies don’t have design. The artists and the engineers do it. Valve famously has very few designers because everybody there is a designer. But in companies where design is a discipline, which it very much is so at Blizzard, game designers are sort of the creating the game experience people, you know, setting up all the systems and content in a way that gets the player to navigate through the game.
You’ve got game design, which some companies don’t have design. The artists and the engineers do it. Valve famously has very few designers because everybody there is a designer. But in companies where design is a discipline, which it very much is so at Blizzard, game designers are sort of the creating the game experience people, you know, setting up all the systems and content in a way that gets the player to navigate through the game.
Lex Fridman
So that’s part of a story, part of this quest design, part of it is like how you move through the game world.
So that’s part of a story, part of this quest design, part of it is like how you move through the game world.
Jeff Kaplan
Yes. So game designers, there’s a spectrum, like same with art, same with engineering, of roles within game design. Some are more heavy on the systems side. So like any game that you’ve played where loot drops- … you know, Diablo IV, World of Warcraft, you know, Escape from Tarkov, whatever. If there’s loot dropping, a designer has planned out very carefully what drops where and at what percentages. That would be like a systems designer. A content designer is somebody who’s gonna make quests or write storylines, or there might even be a narrative designer, which is even more focused on a story. But designers, you know, run the gamut, and then you’ve got these jack-of-all-trade designers that can do it all.
Yes. So game designers, there’s a spectrum, like same with art, same with engineering, of roles within game design. Some are more heavy on the systems side. So like any game that you’ve played where loot drops- … you know, Diablo IV, World of Warcraft, you know, Escape from Tarkov, whatever. If there’s loot dropping, a designer has planned out very carefully what drops where and at what percentages. That would be like a systems designer. A content designer is somebody who’s gonna make quests or write storylines, or there might even be a narrative designer, which is even more focused on a story. But designers, you know, run the gamut, and then you’ve got these jack-of-all-trade designers that can do it all.
Jeff Kaplan
So that’s the design group. There’s production, which is project management, and production is different at every game company you go to. So if you talk to someone from EA or Blizzard, production might be very different. They might be the boss. They might actually be a designer or they might be more of a project manager. And then one of my favorite disciplines on a game team that’s often overlooked is sound and-
So that’s the design group. There’s production, which is project management, and production is different at every game company you go to. So if you talk to someone from EA or Blizzard, production might be very different. They might be the boss. They might actually be a designer or they might be more of a project manager. And then one of my favorite disciplines on a game team that’s often overlooked is sound and-
Jeff Kaplan
… you know, audio, which is comprised of the sound designers and composers. And there are two things, I think there are two things that no one realizes how much they bring to a game until they’re missing, and that’s audio and lighting. Because most of the time, we’re playing without these things, and it just feels a little off and wrong. And when you have a great lighting artist or you have a great composer or sound designer, like, it… the experience. You’re just tapping into these senses that you wouldn’t otherwise. But that’s who comprises the game team.
… you know, audio, which is comprised of the sound designers and composers. And there are two things, I think there are two things that no one realizes how much they bring to a game until they’re missing, and that’s audio and lighting. Because most of the time, we’re playing without these things, and it just feels a little off and wrong. And when you have a great lighting artist or you have a great composer or sound designer, like, it… the experience. You’re just tapping into these senses that you wouldn’t otherwise. But that’s who comprises the game team.
Lex Fridman
Is the lighting, you know, all the different kinds of graphics, would that be under the art team?
Is the lighting, you know, all the different kinds of graphics, would that be under the art team?
Jeff Kaplan
Yeah. Lighting, you’re gonna have lighting under the art team, but they’re gonna be best friends with the graphics programmer. And, you know, like I mentioned with design, there’s this wide spectrum on the engineering team. You have some guys who are like architectural geniuses who are coming up with, you know, the server client model or the networking or whatever. Others are more, like, gameplay focus. On Overwatch, we had an audio programmer just doing nothing but audio hooks for the audio team. And on every game team, you’re gonna have graphics programmers who will work with people like the lighting artists or the environmental artists, character artists on shaders, and basically any way to make the game. They’ll always ask, “What’s your vision?”
Yeah. Lighting, you’re gonna have lighting under the art team, but they’re gonna be best friends with the graphics programmer. And, you know, like I mentioned with design, there’s this wide spectrum on the engineering team. You have some guys who are like architectural geniuses who are coming up with, you know, the server client model or the networking or whatever. Others are more, like, gameplay focus. On Overwatch, we had an audio programmer just doing nothing but audio hooks for the audio team. And on every game team, you’re gonna have graphics programmers who will work with people like the lighting artists or the environmental artists, character artists on shaders, and basically any way to make the game. They’ll always ask, “What’s your vision?”
Jeff Kaplan
What are you trying to get it to look like?” They’ll want an illustration of what should the world look like, and they’ll be the ones who say, “I know how to write code that will let you do that.” So you partner a great graphics programmer with a great lighting artist, and that’s… That’s actually the creative tension behind games and what makes game teams so unique, is if we were to line them up on some crazy spectrum on one end, you’re gonna have the artists who… They’re creative, dare I say emotional- … you know, they are artistes on that end. And on the other end, you have the most logical, brilliant programmers who their minds just work very differently from the most creative art-
What are you trying to get it to look like?” They’ll want an illustration of what should the world look like, and they’ll be the ones who say, “I know how to write code that will let you do that.” So you partner a great graphics programmer with a great lighting artist, and that’s… That’s actually the creative tension behind games and what makes game teams so unique, is if we were to line them up on some crazy spectrum on one end, you’re gonna have the artists who… They’re creative, dare I say emotional- … you know, they are artistes on that end. And on the other end, you have the most logical, brilliant programmers who their minds just work very differently from the most creative art-
Jeff Kaplan
Like artists could be sitting, you have a meeting with them and they’ll just sit illustrating. If there’s any piece of paper, they’re drawing on it. And programmers, you know, they’re just so brilliant and organized in their thinking and everything is so logical. And then in the middle are people like the sound designers, the game designers, and the producers. They’re kind of a little bit in all those fields, but it’s the brilliance of taking people who are so vastly different in their interests and talents, but aiming them at that shared goal or that shared vision of the game that, like, really makes something special.
Like artists could be sitting, you have a meeting with them and they’ll just sit illustrating. If there’s any piece of paper, they’re drawing on it. And programmers, you know, they’re just so brilliant and organized in their thinking and everything is so logical. And then in the middle are people like the sound designers, the game designers, and the producers. They’re kind of a little bit in all those fields, but it’s the brilliance of taking people who are so vastly different in their interests and talents, but aiming them at that shared goal or that shared vision of the game that, like, really makes something special.
Lex Fridman
And there, I mean, you showed me the size of the team for World of Warcraft, but you also are well known for working on quite small teams to create these incredibly huge games. What is the power of a small team in this kind of context where a lot… there’s that creative tension? Is it because a small team avoids maybe the compartmentalization, like the modular where the artists now have their own wing building where they never talk to the engineers, that kind of thing?
And there, I mean, you showed me the size of the team for World of Warcraft, but you also are well known for working on quite small teams to create these incredibly huge games. What is the power of a small team in this kind of context where a lot… there’s that creative tension? Is it because a small team avoids maybe the compartmentalization, like the modular where the artists now have their own wing building where they never talk to the engineers, that kind of thing?
Jeff Kaplan
Absolutely. I mean, you hit the nail on the head. The bigger the team, the more you become a cog in the machine. And on a small team, the way I like to describe it is you get to have a loud voice. If we’re a small team, let’s say we’re gonna make a game and it’s at sort of the incubation period of a game and there’s only 10 of us, all 10 of us are in the room for every decision. You know, I’m not a server networking guy, but I’m in the room for that discussion. I’m not an illustrator, but I’m gonna sit in the room when we decide what the art style looks like. As soon as the team starts to grow, we become compartmentalized.
Absolutely. I mean, you hit the nail on the head. The bigger the team, the more you become a cog in the machine. And on a small team, the way I like to describe it is you get to have a loud voice. If we’re a small team, let’s say we’re gonna make a game and it’s at sort of the incubation period of a game and there’s only 10 of us, all 10 of us are in the room for every decision. You know, I’m not a server networking guy, but I’m in the room for that discussion. I’m not an illustrator, but I’m gonna sit in the room when we decide what the art style looks like. As soon as the team starts to grow, we become compartmentalized.
Jeff Kaplan
It’s exactly like you said. And there’s a weird thing that happens that’s just kind of a human nature thing. The less you interact with somebody, the more you sort of become alienated from them and vilify their point of view. You tend to look at what they do and say with skepticism- … rather than trust and belief in them. And I find on smaller teams where we all know each other’s names, I know what everybody’s working on every day, they know what I’m working on, everybody can talk to each other, there’s none of that stereotyping of a discipline. On big unhealthy teams, you start to say things like, “Well, the artists just don’t get it.”
It’s exactly like you said. And there’s a weird thing that happens that’s just kind of a human nature thing. The less you interact with somebody, the more you sort of become alienated from them and vilify their point of view. You tend to look at what they do and say with skepticism- … rather than trust and belief in them. And I find on smaller teams where we all know each other’s names, I know what everybody’s working on every day, they know what I’m working on, everybody can talk to each other, there’s none of that stereotyping of a discipline. On big unhealthy teams, you start to say things like, “Well, the artists just don’t get it.”
Jeff Kaplan
“They don’t understand what we’re trying to make.” And when you back up and you think about the statement that you just said, it’s like… such an asshole statement. Like, really, all the artists don’t get it? Like, that’s… A, that’s not true. B, that’s sort of demeaning to them. Like, they signed up for the… This is their life’s work, too. This game is gonna be as much theirs as it is mine. So who am I to say a statement like that?
“They don’t understand what we’re trying to make.” And when you back up and you think about the statement that you just said, it’s like… such an asshole statement. Like, really, all the artists don’t get it? Like, that’s… A, that’s not true. B, that’s sort of demeaning to them. Like, they signed up for the… This is their life’s work, too. This game is gonna be as much theirs as it is mine. So who am I to say a statement like that?
Lex Fridman
Yeah. It’s harmful to a discipline to think that you understand the world. Most other folks don’t, and you have nothing to learn from them, really, and they’re deluded in some kind of way. That’s so powerful.
Yeah. It’s harmful to a discipline to think that you understand the world. Most other folks don’t, and you have nothing to learn from them, really, and they’re deluded in some kind of way. That’s so powerful.
Jeff Kaplan
Fast-forwarding a little bit, when we formed Team 4 and… Which went on to make Titan and ultimately fail, and then that got rebooted as the Overwatch team, the idea that I tried to get through to the team was to make an assumption. And really, like, Blizzard is one of the top game developers in the world, and we were very fortunate when I was there, and I imagine it’s this way today, that we could recruit whatever talent we wanted. The best of the best wanted to come work at Blizzard. And if you sort of go through the paces of that and say, “Okay, when we recruit somebody…” Let’s say we’re recruiting an artist to make props.
Fast-forwarding a little bit, when we formed Team 4 and… Which went on to make Titan and ultimately fail, and then that got rebooted as the Overwatch team, the idea that I tried to get through to the team was to make an assumption. And really, like, Blizzard is one of the top game developers in the world, and we were very fortunate when I was there, and I imagine it’s this way today, that we could recruit whatever talent we wanted. The best of the best wanted to come work at Blizzard. And if you sort of go through the paces of that and say, “Okay, when we recruit somebody…” Let’s say we’re recruiting an artist to make props.
Jeff Kaplan
Boxes, chairs, whatever. That is the best prop artist in the industry. That’s who’s gonna show up on our doorstep, so when they show up here, we should treat them like the best prop artist in the industry instead of starting from a place of doubt and cynicism. So, when that person speaks up and says, “I think…” Like, with Overwatch, for example, “I think we should do this.” You know, “We should do X instead of Y.” Instead of saying, “Well, I’m a believer in Y, why are you against my idea X?” You should take a moment, have a deep breath, and say, “Man, the best prop artist in the industry is suggesting something.” Why don’t I listen to it?”
Boxes, chairs, whatever. That is the best prop artist in the industry. That’s who’s gonna show up on our doorstep, so when they show up here, we should treat them like the best prop artist in the industry instead of starting from a place of doubt and cynicism. So, when that person speaks up and says, “I think…” Like, with Overwatch, for example, “I think we should do this.” You know, “We should do X instead of Y.” Instead of saying, “Well, I’m a believer in Y, why are you against my idea X?” You should take a moment, have a deep breath, and say, “Man, the best prop artist in the industry is suggesting something.” Why don’t I listen to it?”
Lex Fridman
I actually do it for myself, like this kinda thought framework or thought experiment. Whenever I’m talking to a new person, especially if I feel, myself, a little bit tinge of that feeling. Usually, it happens with, like, a really young person, like an undergraduate student or someone like this. I pretend that they are the smartest person in the world in my head, and then not… Like, it puts me in the mode of, like, assuming I have a lot to learn from them, and it helps. You actually, like, really listen. I literally think they’re the smartest, wisest human on Earth. It helps me.
I actually do it for myself, like this kinda thought framework or thought experiment. Whenever I’m talking to a new person, especially if I feel, myself, a little bit tinge of that feeling. Usually, it happens with, like, a really young person, like an undergraduate student or someone like this. I pretend that they are the smartest person in the world in my head, and then not… Like, it puts me in the mode of, like, assuming I have a lot to learn from them, and it helps. You actually, like, really listen. I literally think they’re the smartest, wisest human on Earth. It helps me.
Jeff Kaplan
I had that, like, I think… You know, I’m no expert. I’m a game designer, so, like, as much psychology as I know is how to manipulate people into having fun, hopefully. Like, I don’t know, I don’t have an important job.
I had that, like, I think… You know, I’m no expert. I’m a game designer, so, like, as much psychology as I know is how to manipulate people into having fun, hopefully. Like, I don’t know, I don’t have an important job.
Lex Fridman
Yeah, right.
Yeah, right.
Jeff Kaplan
But psychologically speaking-
But psychologically speaking-
Lex Fridman
That’s fun.
That’s fun.
Jeff Kaplan
… I think a lot about is ego, and I think about insecurity. And insecurity, we all have. Like, all of us as human beings have insecurity. It just manifests itself in different ways.
… I think a lot about is ego, and I think about insecurity. And insecurity, we all have. Like, all of us as human beings have insecurity. It just manifests itself in different ways.
Jeff Kaplan
And as we kind of go through our life journey, the insecurity also changes. So, like, some people, for example, use their insecurity to rip other people apart. Some people destroy themselves through their own insecurity. Some people destroy everybody with their insecurity. But I had that moment as a young lead, when I first was made a lead on, like, World of Warcraft, where I felt it was very important to be right and to, you know, be shepherding the correct idea. And I actually got pulled aside. Like, Pardo and I had a meeting with a couple people who weren’t game designers, and it’s always tricky as a game designer because constantly everybody is throwing ideas out on a game team. Like, there’s no shortage of ideas ever.
And as we kind of go through our life journey, the insecurity also changes. So, like, some people, for example, use their insecurity to rip other people apart. Some people destroy themselves through their own insecurity. Some people destroy everybody with their insecurity. But I had that moment as a young lead, when I first was made a lead on, like, World of Warcraft, where I felt it was very important to be right and to, you know, be shepherding the correct idea. And I actually got pulled aside. Like, Pardo and I had a meeting with a couple people who weren’t game designers, and it’s always tricky as a game designer because constantly everybody is throwing ideas out on a game team. Like, there’s no shortage of ideas ever.
Jeff Kaplan
And we were in some meeting about something, and these people kind of threw out these ideas. And I wasn’t mean to them, but I very kind of systematically, like an insecure, you know, ego-driven new lead would do, I kind of, “Let me tell you why that’s wrong, and let me tell you what we’re gonna do instead.” And after the meeting, you know, Pardo pulled me aside, and he said, “You’re a very smart designer, but you shouldn’t do what you just did to those people. You should always listen to what people have to say and try to make their ideas work.” And I just…
And we were in some meeting about something, and these people kind of threw out these ideas. And I wasn’t mean to them, but I very kind of systematically, like an insecure, you know, ego-driven new lead would do, I kind of, “Let me tell you why that’s wrong, and let me tell you what we’re gonna do instead.” And after the meeting, you know, Pardo pulled me aside, and he said, “You’re a very smart designer, but you shouldn’t do what you just did to those people. You should always listen to what people have to say and try to make their ideas work.” And I just…
Jeff Kaplan
Over and over, I was like, “Okay, anytime an idea comes my way, let’s try to make it work.” And it went from this kind of thing that I didn’t believe into to actually, like, a core part of who I am today as a leader, as a game designer, as a game director. And some of the best ideas have come from developing other peoples’ ideas-
Over and over, I was like, “Okay, anytime an idea comes my way, let’s try to make it work.” And it went from this kind of thing that I didn’t believe into to actually, like, a core part of who I am today as a leader, as a game designer, as a game director. And some of the best ideas have come from developing other peoples’ ideas-
Jeff Kaplan
… where your first reaction is like, “No, that’s wrong,” and then just kind of sticking with it and going, “But how could we make it work?” And the most gratifying part when it succeeds is they get all the credit, and you’ve sort of elevated this person whose idea wouldn’t have been championed, whose idea by the insecure, egotistical lead of, you know, early 2000s would have just said no. Now their idea is the thing everybody in World of Warcraft or Overwatch is just loving, and they get all the credit.
… where your first reaction is like, “No, that’s wrong,” and then just kind of sticking with it and going, “But how could we make it work?” And the most gratifying part when it succeeds is they get all the credit, and you’ve sort of elevated this person whose idea wouldn’t have been championed, whose idea by the insecure, egotistical lead of, you know, early 2000s would have just said no. Now their idea is the thing everybody in World of Warcraft or Overwatch is just loving, and they get all the credit.
Lex Fridman
I should give context to the listener who don’t know about the great Jeffrey Kaplan that you’re one of the most humble and always give credit to the team for everything and anything. And so everything we talk about today, I know you’re probably resisting constantly giving credit to the team on everything. So you’re the famous, “Hi, I’m Jeff from the Overwatch team,” right? So just as a small aside, thank you for your humility through your career, and thank you for always celebrating the team. But let’s talk about WoW. Let’s talk about World of Warcraft. Tell me what the early days of developing WoW was like. Maybe we should talk about what World of Warcraft, WoW is, going to Perplexity here.
I should give context to the listener who don’t know about the great Jeffrey Kaplan that you’re one of the most humble and always give credit to the team for everything and anything. And so everything we talk about today, I know you’re probably resisting constantly giving credit to the team on everything. So you’re the famous, “Hi, I’m Jeff from the Overwatch team,” right? So just as a small aside, thank you for your humility through your career, and thank you for always celebrating the team. But let’s talk about WoW. Let’s talk about World of Warcraft. Tell me what the early days of developing WoW was like. Maybe we should talk about what World of Warcraft, WoW is, going to Perplexity here.
Building World of Warcraft
Lex Fridman
World of Warcraft is a massively multiplayer online RPG where you create a character, level it up doing quests and dungeons, and progress your gear and power in an open fantasy world called Azeroth. At a basic level, you move, use abilities from your action bar, follow quests, and gradually learn a combat rotation that fits your class. And there’s all kinds of characters and roles and classes. You pick a race, appearance, starting zone, small racial bonuses. In a class, how you fight, what your role is in groups. Can you continue, fill in some of the gaps, what is World of Warcraft?
World of Warcraft is a massively multiplayer online RPG where you create a character, level it up doing quests and dungeons, and progress your gear and power in an open fantasy world called Azeroth. At a basic level, you move, use abilities from your action bar, follow quests, and gradually learn a combat rotation that fits your class. And there’s all kinds of characters and roles and classes. You pick a race, appearance, starting zone, small racial bonuses. In a class, how you fight, what your role is in groups. Can you continue, fill in some of the gaps, what is World of Warcraft?
Jeff Kaplan
World of Warcraft, first of all, more than anything, is a world. Like, it’s a world that you can live in with real other people, and everybody’s kinda living out their fantasy. Chris Metzen, who was the creative director on World of Warcraft, and really, like, Allen Adham, who’s one of the founders of Blizzard, calls Chris “the heart and soul of Blizzard.” And it’s almost like when you’re making a Blizzard game, you’re making Chris’ imagination at some point. And Chris famously said, “The lead character of World of Warcraft is the world.” And I always believed that. So you’re trying to create this place that’s exciting and dangerous, but comfortable, but uncomfortable and gorgeous, and, you know it should feel massive, and it really is.
World of Warcraft, first of all, more than anything, is a world. Like, it’s a world that you can live in with real other people, and everybody’s kinda living out their fantasy. Chris Metzen, who was the creative director on World of Warcraft, and really, like, Allen Adham, who’s one of the founders of Blizzard, calls Chris “the heart and soul of Blizzard.” And it’s almost like when you’re making a Blizzard game, you’re making Chris’ imagination at some point. And Chris famously said, “The lead character of World of Warcraft is the world.” And I always believed that. So you’re trying to create this place that’s exciting and dangerous, but comfortable, but uncomfortable and gorgeous, and, you know it should feel massive, and it really is.
Jeff Kaplan
It’s, you know, can take a half an hour to get from one end of the world to the other. But it’s this world you’re living in. The world is divided into two warring factions. There’s the Horde and the Alliance, and that was a very important, very controversial decision that was made by Allen Adham, was the champion of the Horde and Alliance.
It’s, you know, can take a half an hour to get from one end of the world to the other. But it’s this world you’re living in. The world is divided into two warring factions. There’s the Horde and the Alliance, and that was a very important, very controversial decision that was made by Allen Adham, was the champion of the Horde and Alliance.
Lex Fridman
And that in the early days, there was a really strong division.
And that in the early days, there was a really strong division.
Jeff Kaplan
Strong division.
Strong division.
Lex Fridman
Like… You pick a side and then you hang around with only people of your kind.
Like… You pick a side and then you hang around with only people of your kind.
Jeff Kaplan
Yeah, and you get it tattooed in real life on you. Like, the amount of people who walk up to me and show me their Horde tattoo.
Yeah, and you get it tattooed in real life on you. Like, the amount of people who walk up to me and show me their Horde tattoo.
Lex Fridman
That’s awesome.
That’s awesome.
Jeff Kaplan
Like, it’s epic. It’s like it’s become who they are. Like, if you were to say, like, “Hey, Lex, come play World of Warcraft with me. We’re Alliance on Tichondrius,” you’d be like- … “Dude-“
Like, it’s epic. It’s like it’s become who they are. Like, if you were to say, like, “Hey, Lex, come play World of Warcraft with me. We’re Alliance on Tichondrius,” you’d be like- … “Dude-“
Lex Fridman
Lose my number.
Lose my number.
Jeff Kaplan
“… Alliance?” Like, “Okay, I don’t think we can be friends anymore.” But the Horde-Alliance decision was really controversial because in EverQuest, it was mixed race. They had all the races kind of like WoW did, but they could all group with each other. And Pardo and I came from EverQuest, where we felt like this was a horrible decision Allen was making. And we argued—Allen, Rob, Bob Fitch, and I would have lunch every single day, and we would just talk about WoW and the core design of WoW. Rob wasn’t even on WoW at that time. He was finishing Warcraft III. And we would fight over the Horde-Alliance split, if it was a good idea or not. And Allen had… He came from more of the Dark Age of Camelot community, which was another massively multiplayer online game that was more PvP based.
“… Alliance?” Like, “Okay, I don’t think we can be friends anymore.” But the Horde-Alliance decision was really controversial because in EverQuest, it was mixed race. They had all the races kind of like WoW did, but they could all group with each other. And Pardo and I came from EverQuest, where we felt like this was a horrible decision Allen was making. And we argued—Allen, Rob, Bob Fitch, and I would have lunch every single day, and we would just talk about WoW and the core design of WoW. Rob wasn’t even on WoW at that time. He was finishing Warcraft III. And we would fight over the Horde-Alliance split, if it was a good idea or not. And Allen had… He came from more of the Dark Age of Camelot community, which was another massively multiplayer online game that was more PvP based.
Jeff Kaplan
And he said the magic of that game was they had three factions, and he liked the fact that you were instantly on a team. You weren’t a loner in the world. And whether you liked it or not, you had people on your side. And Rob and I just argued and argued against it, and then sometime before beta, Allen retired. He went on to run a hedge fund, of all things. Like, got super into poker, got super into finance, left, and retires, like, I think it was nine months to a year before WoW shipped, which is kinda nuts. And Rob takes over as lead designer in Allen’s stead, and to Rob’s credit, the first thing he did was go… Speaking to what we were speaking about earlier, he said, “Allen’s a smart guy. The fact that he was fighting so hard for-“
And he said the magic of that game was they had three factions, and he liked the fact that you were instantly on a team. You weren’t a loner in the world. And whether you liked it or not, you had people on your side. And Rob and I just argued and argued against it, and then sometime before beta, Allen retired. He went on to run a hedge fund, of all things. Like, got super into poker, got super into finance, left, and retires, like, I think it was nine months to a year before WoW shipped, which is kinda nuts. And Rob takes over as lead designer in Allen’s stead, and to Rob’s credit, the first thing he did was go… Speaking to what we were speaking about earlier, he said, “Allen’s a smart guy. The fact that he was fighting so hard for-“
Jeff Kaplan
“… Horde Alliance, we gotta do it.” And Rob and I sort of changed our point of view and got on board with Horde Alliance and went all in. And so, you know, the early days of WoW was… It was a great team. It was a mix of these veterans that we all looked up to. You know, we had Mark Kern running the team. Shane Dabiri was, you know, a legendary Blizzard developer. Bill Petras was the art director, and then we had Metzen, who was sort of like… Metzen was the cool big brother we all, you know, aspired to be.
“… Horde Alliance, we gotta do it.” And Rob and I sort of changed our point of view and got on board with Horde Alliance and went all in. And so, you know, the early days of WoW was… It was a great team. It was a mix of these veterans that we all looked up to. You know, we had Mark Kern running the team. Shane Dabiri was, you know, a legendary Blizzard developer. Bill Petras was the art director, and then we had Metzen, who was sort of like… Metzen was the cool big brother we all, you know, aspired to be.
Jeff Kaplan
I’m older than Metzen, but I looked up to him like a big brother. And then there were a lot of us who had never done it before, or they had also pulled a lot of people from other teams and other game types. Like, for example, the guys building the dungeons, they hired out of the Quake community. And because they didn’t have any hardcore MMO designer on the staff at that time, it was, you know, Kevin and Eric and Alan were sort of the only designers, they started building Quake dungeons- … as, like, Quake levels as the dungeons. At one point, WoW was even made in QeRadiant, which was the Quake engine. And then they later, you know, retooled to where they were using a proprietary engine. So we were like this hodgepodge, like the Bad News Bears-
I’m older than Metzen, but I looked up to him like a big brother. And then there were a lot of us who had never done it before, or they had also pulled a lot of people from other teams and other game types. Like, for example, the guys building the dungeons, they hired out of the Quake community. And because they didn’t have any hardcore MMO designer on the staff at that time, it was, you know, Kevin and Eric and Alan were sort of the only designers, they started building Quake dungeons- … as, like, Quake levels as the dungeons. At one point, WoW was even made in QeRadiant, which was the Quake engine. And then they later, you know, retooled to where they were using a proprietary engine. So we were like this hodgepodge, like the Bad News Bears-
Jeff Kaplan
… is how I would describe the WoW team, of this mix of veterans and then people like me. Like, I’m just some fucking idiot, you know- … who played a lot of EverQuest. And I end up at Blizzard.
… is how I would describe the WoW team, of this mix of veterans and then people like me. Like, I’m just some fucking idiot, you know- … who played a lot of EverQuest. And I end up at Blizzard.
Lex Fridman
Designing quests.
Designing quests.
Jeff Kaplan
Yeah. Like, okay, we’re gonna design World of Warcraft now. And I’ve said this later with hindsight, I think a huge part of WoW’s success, especially with the early WoW team, Team Two in its earliest formation, was that we didn’t know what we were doing. You kind of… Like, it’s that… Titan was the example for me. Titan was the attempt at making an MMO after World of Warcraft at Blizzard. And we failed horribly, and we had the best of the best on that team. And it’s because everybody was too much of an expert on how to make a groundbreaking phenomenon MMO. World of Warcraft was a bunch of people, like a very successful, sure-of-itself company who had made StarCraft, Diablo, Warcraft, with a bunch of yayhoos basically- … Who was like, “Yeah, we can compete with Sony Online.”
Yeah. Like, okay, we’re gonna design World of Warcraft now. And I’ve said this later with hindsight, I think a huge part of WoW’s success, especially with the early WoW team, Team Two in its earliest formation, was that we didn’t know what we were doing. You kind of… Like, it’s that… Titan was the example for me. Titan was the attempt at making an MMO after World of Warcraft at Blizzard. And we failed horribly, and we had the best of the best on that team. And it’s because everybody was too much of an expert on how to make a groundbreaking phenomenon MMO. World of Warcraft was a bunch of people, like a very successful, sure-of-itself company who had made StarCraft, Diablo, Warcraft, with a bunch of yayhoos basically- … Who was like, “Yeah, we can compete with Sony Online.”
Jeff Kaplan
At the time, they were making EverQuest II. Like, if we go back in the time machine, EverQuest II had been announced. And EverQuest fans, we were just drooling for EverQuest II. It wasn’t, “Oh, cool, World of Warcraft.” It was EQ2 was gonna take, you know, the chalice and run with it. And then, of all things, they announced Star Wars Galaxies, and they had a brilliant designer on that, a guy named Raph Koster, who had come from Ultima Online, and he’s just a really smart game designer. If you ever watch one of his lectures, like, he lectured a lot at GDC, and, you know, we’re like, “Oh my God, they’re- they’re making EverQuest II and Star Wars Galaxies, and they have the Star Wars intellectual property.” “We’re fucked.”
At the time, they were making EverQuest II. Like, if we go back in the time machine, EverQuest II had been announced. And EverQuest fans, we were just drooling for EverQuest II. It wasn’t, “Oh, cool, World of Warcraft.” It was EQ2 was gonna take, you know, the chalice and run with it. And then, of all things, they announced Star Wars Galaxies, and they had a brilliant designer on that, a guy named Raph Koster, who had come from Ultima Online, and he’s just a really smart game designer. If you ever watch one of his lectures, like, he lectured a lot at GDC, and, you know, we’re like, “Oh my God, they’re- they’re making EverQuest II and Star Wars Galaxies, and they have the Star Wars intellectual property.” “We’re fucked.”
Jeff Kaplan
Like, “How are we gonna compete?” And everybody had seen the success of EQ, EverQuest, and everybody was gonna make an MMO, and it was just a question of who was gonna win.
Like, “How are we gonna compete?” And everybody had seen the success of EQ, EverQuest, and everybody was gonna make an MMO, and it was just a question of who was gonna win.
Lex Fridman
So you’re feeling this immense pressure. You have this small team of just this hodgepodge of this unlikely team that kind of looks fast forwarding to Overwatch, the heroes in Overwatch, but working extremely hard. Now, you- you told me about crazy, crazy work hours, and not because you were forced to, but because you wanted to, because your heart was in it, because you’re like, “This is everything.” Like, you loved it.
So you’re feeling this immense pressure. You have this small team of just this hodgepodge of this unlikely team that kind of looks fast forwarding to Overwatch, the heroes in Overwatch, but working extremely hard. Now, you- you told me about crazy, crazy work hours, and not because you were forced to, but because you wanted to, because your heart was in it, because you’re like, “This is everything.” Like, you loved it.
Jeff Kaplan
Yeah. The games industry has a terrible reputation for insane amounts of overtime. It’s just called crunch. Like, do you crunch or not? These days, crunch is not allowed, not permitted, heavily frowned upon. If we were to work overtime, somebody’d write an article about it next week and say how horrible we are for working overtime. Back then, we worked insane, and I mean insane hours. The longest shift I ever worked straight was 30 hours. That’s when we were gold mastering Warcraft III. This was in my… I think War III shipped on July 3rd, 2002, so this would have been, like, late June, early July. Probably late June. And I had nothing to do with War III.
Yeah. The games industry has a terrible reputation for insane amounts of overtime. It’s just called crunch. Like, do you crunch or not? These days, crunch is not allowed, not permitted, heavily frowned upon. If we were to work overtime, somebody’d write an article about it next week and say how horrible we are for working overtime. Back then, we worked insane, and I mean insane hours. The longest shift I ever worked straight was 30 hours. That’s when we were gold mastering Warcraft III. This was in my… I think War III shipped on July 3rd, 2002, so this would have been, like, late June, early July. Probably late June. And I had nothing to do with War III.
Jeff Kaplan
I should just say that. Like, in the credits, I’m additional help or additional testing or something like that. When I showed up in May of 2002, it was all-hands-on-deck World of Warcraft for E3. We got through E3, and then all hands on deck, the whole company, get War III out the door.
I should just say that. Like, in the credits, I’m additional help or additional testing or something like that. When I showed up in May of 2002, it was all-hands-on-deck World of Warcraft for E3. We got through E3, and then all hands on deck, the whole company, get War III out the door.
Lex Fridman
For shipping Warcraft III.
For shipping Warcraft III.
Jeff Kaplan
For shipping Warcraft III, and because I had not been involved with the game at all, and I was a brand new wet-behind-the-ears game designer, they’re like, “You’re just gonna help test whatever we tell you to test.” So we’re trying to gold master, and there’s a crash that happens rarely. If you run one of the cinematics, like you have to be watching the cinematic after one of the levels, and then there was a crash that happened. And so a programmer put in some logging to catch it, and then they needed somebody to just over and over again, “I need the crash to happen so I can fix the bug.” And I sat there for 30 hours and just watched the cinematic for 30 hours-
For shipping Warcraft III, and because I had not been involved with the game at all, and I was a brand new wet-behind-the-ears game designer, they’re like, “You’re just gonna help test whatever we tell you to test.” So we’re trying to gold master, and there’s a crash that happens rarely. If you run one of the cinematics, like you have to be watching the cinematic after one of the levels, and then there was a crash that happened. And so a programmer put in some logging to catch it, and then they needed somebody to just over and over again, “I need the crash to happen so I can fix the bug.” And I sat there for 30 hours and just watched the cinematic for 30 hours-
Jeff Kaplan
… straight. And it was the funniest thing, like it was almost surreal watching everybody leave at the… which was a trickle out. Like, everybody kind of trickles out, like, at- … different hours, you know? The family guys go much earlier than the single guys. And then watching everybody show up again in… the next morning, and they’re all, like, dressed different, and they look all refreshed. And I’m just like in the same position. You know, like eyes are beet red.
… straight. And it was the funniest thing, like it was almost surreal watching everybody leave at the… which was a trickle out. Like, everybody kind of trickles out, like, at- … different hours, you know? The family guys go much earlier than the single guys. And then watching everybody show up again in… the next morning, and they’re all, like, dressed different, and they look all refreshed. And I’m just like in the same position. You know, like eyes are beet red.
Lex Fridman
To the soundtrack of the cinematic and yeah.
To the soundtrack of the cinematic and yeah.
Jeff Kaplan
Yeah. But we crunched World of Warcraft, we crunched… The date slipped, so you do this thing. I remember Mark Kern standing the team up and saying, “We’re gonna crunch early so we don’t have to crunch later in the project.” And I really believe he wasn’t manipulating us. Like, I really genuinely believe that he believed in that. But with games, anything can happen, and they’re just… We slip uncontrollably all the time. And we slipped, and it sort of created just this death march, endless death march that… Like to this day, members of the WoW team will remember, like, Newport Rib. If I say that, they’ll have, like, twitches because, like, they would cater the dinner. They’d bring it in at, like, 6:00 or 7:00 at night.
Yeah. But we crunched World of Warcraft, we crunched… The date slipped, so you do this thing. I remember Mark Kern standing the team up and saying, “We’re gonna crunch early so we don’t have to crunch later in the project.” And I really believe he wasn’t manipulating us. Like, I really genuinely believe that he believed in that. But with games, anything can happen, and they’re just… We slip uncontrollably all the time. And we slipped, and it sort of created just this death march, endless death march that… Like to this day, members of the WoW team will remember, like, Newport Rib. If I say that, they’ll have, like, twitches because, like, they would cater the dinner. They’d bring it in at, like, 6:00 or 7:00 at night.
Jeff Kaplan
And they’d… Everybody was eating Newport Rib or Panda Express. It was like the worst diet ever. I actually like Newport Rib, no shade- … on them. But you can only eat so much of it. And the carpets are stained and, like, dudes are falling asleep on the couches. And it was an unhealthy work environment. It gets pinned on… ‘Cause at a lot of places it is executive driven. And it is mandated from the top, but the hours that I worked, I never blamed on anyone but myself. I just wanted to. I remember, you know, coming in on Memorial Day, like, with sand from the beach on my feet because I really wanted to get some work done that day, and working through Christmas, and those were things I wanted to do. I never felt like somebody, you know, held my feet to the coals.
And they’d… Everybody was eating Newport Rib or Panda Express. It was like the worst diet ever. I actually like Newport Rib, no shade- … on them. But you can only eat so much of it. And the carpets are stained and, like, dudes are falling asleep on the couches. And it was an unhealthy work environment. It gets pinned on… ‘Cause at a lot of places it is executive driven. And it is mandated from the top, but the hours that I worked, I never blamed on anyone but myself. I just wanted to. I remember, you know, coming in on Memorial Day, like, with sand from the beach on my feet because I really wanted to get some work done that day, and working through Christmas, and those were things I wanted to do. I never felt like somebody, you know, held my feet to the coals.
Lex Fridman
Yeah, it’s such a complicated thing because yeah, okay, you could say that’s unhealthy, but I know a large number of people, especially in their 20s, but actually throughout their career, that have been at companies that do crunch for a thing they believe in, for a thing they love, and it’s some of the most fulfilling years of their life, months and years of their life. And they also… it’s not just fulfilling, they grow from it, they learn from it, and it… You know, and when they… Especially when they talk back about it, about that time, they can see how incredible it was. Of course, when you’re going through it, sometimes it’s extremely difficult, you don’t know.
Yeah, it’s such a complicated thing because yeah, okay, you could say that’s unhealthy, but I know a large number of people, especially in their 20s, but actually throughout their career, that have been at companies that do crunch for a thing they believe in, for a thing they love, and it’s some of the most fulfilling years of their life, months and years of their life. And they also… it’s not just fulfilling, they grow from it, they learn from it, and it… You know, and when they… Especially when they talk back about it, about that time, they can see how incredible it was. Of course, when you’re going through it, sometimes it’s extremely difficult, you don’t know.
Lex Fridman
And then the crunch, like you mentioned, it’s supposed to be a month or two, and then it, it turns out to be a half a year, and then maybe it turns out to be something like a Titan type game where you never actually ship it, and it’s heartbreaking and the pain, it’s all… But then you look back and you realize how incredible that journey was.
And then the crunch, like you mentioned, it’s supposed to be a month or two, and then it, it turns out to be a half a year, and then maybe it turns out to be something like a Titan type game where you never actually ship it, and it’s heartbreaking and the pain, it’s all… But then you look back and you realize how incredible that journey was.
Jeff Kaplan
I think, like, my reflections on it many years later, and having gone through, like, pretty crazy levels of crunch to more controlled, I think where crunch is problematic and people are good to be vocal about being opposed to it, is when it’s forced and unnecessary. There’s a lot of like, “Hey, if anybody on the team stays, we all stay”-
I think, like, my reflections on it many years later, and having gone through, like, pretty crazy levels of crunch to more controlled, I think where crunch is problematic and people are good to be vocal about being opposed to it, is when it’s forced and unnecessary. There’s a lot of like, “Hey, if anybody on the team stays, we all stay”-
Lex Fridman
Yeah
Yeah
Jeff Kaplan
… kind of, which I think is not necessary. I don’t think executives who take off and work 40-hour weeks should be telling anybody to stay late. I think that’s wrong and immoral. But to me as an individual, as long as I’m not telling other people to do it, my life’s work is my passion and I want to do it as much as possible. I find myself, I don’t think I’ve ever worked less than 10 hours in a day. Like that… 10 hours is like a normal-ish day to me.
… kind of, which I think is not necessary. I don’t think executives who take off and work 40-hour weeks should be telling anybody to stay late. I think that’s wrong and immoral. But to me as an individual, as long as I’m not telling other people to do it, my life’s work is my passion and I want to do it as much as possible. I find myself, I don’t think I’ve ever worked less than 10 hours in a day. Like that… 10 hours is like a normal-ish day to me.
Jeff Kaplan
And I enjoy lots of weekends working because I enjoy it. It brings me pleasure and fulfillment. And all of that said, from a place of caution, especially in this era when people are very touchy about it. I don’t try to impose that on anybody else. I don’t want anybody to feel like they’re obligated to, but please understand it’s what makes me who I am, that work ethic. I enjoy it. I actually… Some of my fondest memories are from those WoW crunches.
And I enjoy lots of weekends working because I enjoy it. It brings me pleasure and fulfillment. And all of that said, from a place of caution, especially in this era when people are very touchy about it. I don’t try to impose that on anybody else. I don’t want anybody to feel like they’re obligated to, but please understand it’s what makes me who I am, that work ethic. I enjoy it. I actually… Some of my fondest memories are from those WoW crunches.
Lex Fridman
And then looking back and reading some of these stories, it’s pretty cool because me, as a fan, on the receiving end of some of those video games, you bring joy to millions of people. It’s awesome. Let me ask you about quests, but first, quick bathroom break if it’s okay.
And then looking back and reading some of these stories, it’s pretty cool because me, as a fan, on the receiving end of some of those video games, you bring joy to millions of people. It’s awesome. Let me ask you about quests, but first, quick bathroom break if it’s okay.
Lex Fridman
Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Fin for customer service AI agents, Blitzy for code generation in large code bases, BetterHelp for mental health, Shopify for selling stuff online, CodeRabbit for AI-powered code review, and Perplexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Jeff Kaplan. Okay, we’re back. So I think it’s fair to say that before WoW, MMO leveling, like in EverQuest, consisted of, maybe that’s simplifying it a bit, but standing in one spot and killing monsters for hours.
Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Fin for customer service AI agents, Blitzy for code generation in large code bases, BetterHelp for mental health, Shopify for selling stuff online, CodeRabbit for AI-powered code review, and Perplexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Jeff Kaplan. Okay, we’re back. So I think it’s fair to say that before WoW, MMO leveling, like in EverQuest, consisted of, maybe that’s simplifying it a bit, but standing in one spot and killing monsters for hours.
How WoW changed video games
Lex Fridman
You helped develop with WoW, I would say a revolutionary idea of quest-driven leveling, where there’s a story driven, quest driven guide through the world, and it so happens that as part of doing that, you’re also leveling the character. So the leveling is both fun and is the engine that drives the story that then also immerses you into the world and pulls you in more and more and more and more. So take me through this process of developing that idea of quest driven design.
You helped develop with WoW, I would say a revolutionary idea of quest-driven leveling, where there’s a story driven, quest driven guide through the world, and it so happens that as part of doing that, you’re also leveling the character. So the leveling is both fun and is the engine that drives the story that then also immerses you into the world and pulls you in more and more and more and more. So take me through this process of developing that idea of quest driven design.
Jeff Kaplan
Sure. Yeah, there were actually a lot of people involved in it, and they all kind of contributed in their own unique ways. Alan Adham was the lead designer on WoW. When we first sort of decided we were gonna have a quest-based game, we used to joke that, like, EverQuest barely had any quests in it.
Sure. Yeah, there were actually a lot of people involved in it, and they all kind of contributed in their own unique ways. Alan Adham was the lead designer on WoW. When we first sort of decided we were gonna have a quest-based game, we used to joke that, like, EverQuest barely had any quests in it.
Jeff Kaplan
It did have quests, they just… They weren’t really in front of the player in an obvious way. You kind of had to seek them out on a website. And Alan knew that he wanted quests to be a big part of World of Warcraft. And so he hired me. That was my entry level position at Blizzard. And on the same day, he hired a guy named Pat Nagle, which was hilarious to me, because Pat was the… He had this funny title of HR and Facilities at Blizzard, because it was such a small company. So, like, if you sent an application in, Pat would deal with the application, or if the toilet overflowed, Pat would have to deal with it.
It did have quests, they just… They weren’t really in front of the player in an obvious way. You kind of had to seek them out on a website. And Alan knew that he wanted quests to be a big part of World of Warcraft. And so he hired me. That was my entry level position at Blizzard. And on the same day, he hired a guy named Pat Nagle, which was hilarious to me, because Pat was the… He had this funny title of HR and Facilities at Blizzard, because it was such a small company. So, like, if you sent an application in, Pat would deal with the application, or if the toilet overflowed, Pat would have to deal with it.
Jeff Kaplan
And so the whole time I was applying at Blizzard, I was going through Pat, and then on my first day, they put Pat and I in an office together, and he’s like, “Yeah, they hired me also as the quest designer.” And so Pat… And he was the most wonderful guy. We had so much fun. So Pat and I kind of designed the quest system. It was Alan’s idea to have it in the first place. And then there was that great designer I mentioned, Eric Dodds, who helped a lot with the interface of it all. And the idea was… At first, we actually on a whiteboard in Alan’s office, we estimated how many quests we thought EverQuest had to date.
And so the whole time I was applying at Blizzard, I was going through Pat, and then on my first day, they put Pat and I in an office together, and he’s like, “Yeah, they hired me also as the quest designer.” And so Pat… And he was the most wonderful guy. We had so much fun. So Pat and I kind of designed the quest system. It was Alan’s idea to have it in the first place. And then there was that great designer I mentioned, Eric Dodds, who helped a lot with the interface of it all. And the idea was… At first, we actually on a whiteboard in Alan’s office, we estimated how many quests we thought EverQuest had to date.
Jeff Kaplan
And EverQuest had had, you know, I think four or three expansions at that point in time, and we’re like, “Wow, we have to make all of these quests like EverQuest has.” It’s gonna be a lot of quests, and it’s kind of up to me and Pat to do it all. And we believed all we had to do was match that EverQuest number. And Pat and I started working on, like, the design of the system and how it would interact, and Eric Dodds was really involved in how the interface… You know, like how you were going to interact with the NPCs and all of that. And we split up the world into like two zones. He was gonna take Elwynn Forest, which was the starting area for the humans, and I was gonna take Westfall, which was the sophomore zone after Elwynn for the humans.
And EverQuest had had, you know, I think four or three expansions at that point in time, and we’re like, “Wow, we have to make all of these quests like EverQuest has.” It’s gonna be a lot of quests, and it’s kind of up to me and Pat to do it all. And we believed all we had to do was match that EverQuest number. And Pat and I started working on, like, the design of the system and how it would interact, and Eric Dodds was really involved in how the interface… You know, like how you were going to interact with the NPCs and all of that. And we split up the world into like two zones. He was gonna take Elwynn Forest, which was the starting area for the humans, and I was gonna take Westfall, which was the sophomore zone after Elwynn for the humans.
Jeff Kaplan
Pat and I would meet with Chris Metzen, and those were the funnest meetings ever because Chris just has stories in his head and visions. Chris is, like, artist, storyteller, world builder extraordinaire, and he sort of described what he wanted going on in those zones. You know, you want the gameplay to follow the flow of what was going on with the stories of those areas. So we finished Elwynn and Westfall, and we did, like, a team play test. And our assumption was because the way EverQuest worked, players just wanted to level up. It was a level based game.
Pat and I would meet with Chris Metzen, and those were the funnest meetings ever because Chris just has stories in his head and visions. Chris is, like, artist, storyteller, world builder extraordinaire, and he sort of described what he wanted going on in those zones. You know, you want the gameplay to follow the flow of what was going on with the stories of those areas. So we finished Elwynn and Westfall, and we did, like, a team play test. And our assumption was because the way EverQuest worked, players just wanted to level up. It was a level based game.
Jeff Kaplan
You go out. You kill a creature. You get experience points. You level up a little bit. And so the way people played EverQuest is they’d find these areas where there were lots of creatures, and you’d usually find the best experience efficiency cycle you could find, so, like, fast respawn kind of easy things to kill, and that’s how you would progress through EverQuest. And I remember Alan kind of telling us, like, “Hey, the quests… When Pat and Jeff write quests, they’ll aim us to where the creatures are.” You’ll do a quest, and then you’ll spend a few hours killing creatures in that area afterwards, and that’s how he imagined it would work. So we kind of set up the world that way. You know, Pat probably did a dozen, maybe 20 quests in Elwynn.
You go out. You kill a creature. You get experience points. You level up a little bit. And so the way people played EverQuest is they’d find these areas where there were lots of creatures, and you’d usually find the best experience efficiency cycle you could find, so, like, fast respawn kind of easy things to kill, and that’s how you would progress through EverQuest. And I remember Alan kind of telling us, like, “Hey, the quests… When Pat and Jeff write quests, they’ll aim us to where the creatures are.” You’ll do a quest, and then you’ll spend a few hours killing creatures in that area afterwards, and that’s how he imagined it would work. So we kind of set up the world that way. You know, Pat probably did a dozen, maybe 20 quests in Elwynn.
Jeff Kaplan
I’d do a dozen, 20 quests in Westfall, and we’d do this team play test. And we had a bunch of people on the team who never played MMOs, like guys with shooter background, you know, StarCraft fans, et cetera. And they’d play World of Warcraft. I think we played for, like, an hour or two, and we only did Elwynn Forest. And the overwhelming feedback from our team… And these are people who really didn’t play EverQuest, they’re like—My God, Pat, that was horrible. I ran out of quests, like, right away. And we’re like, wait a second. You expect to just have quests just keep going? And they’re like, yeah, we expect to have quests just keep going the whole way.
I’d do a dozen, 20 quests in Westfall, and we’d do this team play test. And we had a bunch of people on the team who never played MMOs, like guys with shooter background, you know, StarCraft fans, et cetera. And they’d play World of Warcraft. I think we played for, like, an hour or two, and we only did Elwynn Forest. And the overwhelming feedback from our team… And these are people who really didn’t play EverQuest, they’re like—My God, Pat, that was horrible. I ran out of quests, like, right away. And we’re like, wait a second. You expect to just have quests just keep going? And they’re like, yeah, we expect to have quests just keep going the whole way.
Jeff Kaplan
And we kinda had an oh shit moment right after that Elwynn Forest play test, where we realized, like, we had vastly underestimated the number of quests we were gonna need. And we changed, we developed this philosophy that’s kind of a shared philosophy across Blizzard games in general at this point. And I’ve heard it outside of Blizzard, other people in the industry, which is you design along the path of least resistance. So, basically what that means, like, in EverQuest, the path of least resistance if you wanted your character to hit max level is to find the easiest creatures and kill them over and over again in place, which to some people think is very boring. To me, I would do that for eight hours ’cause I think that’s fun.
And we kinda had an oh shit moment right after that Elwynn Forest play test, where we realized, like, we had vastly underestimated the number of quests we were gonna need. And we changed, we developed this philosophy that’s kind of a shared philosophy across Blizzard games in general at this point. And I’ve heard it outside of Blizzard, other people in the industry, which is you design along the path of least resistance. So, basically what that means, like, in EverQuest, the path of least resistance if you wanted your character to hit max level is to find the easiest creatures and kill them over and over again in place, which to some people think is very boring. To me, I would do that for eight hours ’cause I think that’s fun.
Jeff Kaplan
But we decided in World of Warcraft, we said, why don’t we make the path of least resistance, so in this case, the way to get the best experience the fastest not to be killing creatures in one place, but will overload the experience into the quests themselves, and then that will move you through the world, which will get you to see everything. It will enable us to tell these awesome storylines. It sort of did a lot for the game, and I think it was like a fundamental change in the genre. Like, if you look at the things that… EverQuest was very popular and very successful, and it was hitting like hundreds of thousands of players. And WoW blew the doors open and was tens of millions of players.
But we decided in World of Warcraft, we said, why don’t we make the path of least resistance, so in this case, the way to get the best experience the fastest not to be killing creatures in one place, but will overload the experience into the quests themselves, and then that will move you through the world, which will get you to see everything. It will enable us to tell these awesome storylines. It sort of did a lot for the game, and I think it was like a fundamental change in the genre. Like, if you look at the things that… EverQuest was very popular and very successful, and it was hitting like hundreds of thousands of players. And WoW blew the doors open and was tens of millions of players.
Jeff Kaplan
And I think the fundamental difference there was that WoW allowed you to play as a single player. And what makes an MMO, massively multiplayer online game, massive is having the other people there. And they’re so important or else the world feels kind of wrong and dead. But the concept that we have to force you to interact with them to do anything is very off-putting to a lot of people. And the fact that people could come into WoW and just kind of the game design, the game design way of des- describing it is directed gameplay.
And I think the fundamental difference there was that WoW allowed you to play as a single player. And what makes an MMO, massively multiplayer online game, massive is having the other people there. And they’re so important or else the world feels kind of wrong and dead. But the concept that we have to force you to interact with them to do anything is very off-putting to a lot of people. And the fact that people could come into WoW and just kind of the game design, the game design way of des- describing it is directed gameplay.
Jeff Kaplan
And some games have extremely tight directed gameplay. Like, for example, if you were to play a single player game like Last of Us, you know, you’ll have those moments where they’ll be like, you’ll come up to a log and then press triangle to duck or else, or whatever the duck button is-
And some games have extremely tight directed gameplay. Like, for example, if you were to play a single player game like Last of Us, you know, you’ll have those moments where they’ll be like, you’ll come up to a log and then press triangle to duck or else, or whatever the duck button is-
Jeff Kaplan
… left stick to duck to go under. And that’s like the ultimate in directed gameplay. Like, they’re telling you exactly what to do. On the other end of the spectrum is a game like Minecraft, like vanilla Minecraft, where you’ll find it’s very divisive amongst gamers who love Minecraft or hate it. The ones who hate it are like, “I don’t know what I’m supposed to do.” Like, “You drop me in this world. I’m supposed to dig or something.” And that’s the type of player that needs directed gameplay or they’re gonna cycle out. Not all players need it. And what WoW did, that it doesn’t seem like an innovation, it doesn’t seem like revolutionary, but it sort of created this directed gameplay that felt optional, but really wasn’t.
… left stick to duck to go under. And that’s like the ultimate in directed gameplay. Like, they’re telling you exactly what to do. On the other end of the spectrum is a game like Minecraft, like vanilla Minecraft, where you’ll find it’s very divisive amongst gamers who love Minecraft or hate it. The ones who hate it are like, “I don’t know what I’m supposed to do.” Like, “You drop me in this world. I’m supposed to dig or something.” And that’s the type of player that needs directed gameplay or they’re gonna cycle out. Not all players need it. And what WoW did, that it doesn’t seem like an innovation, it doesn’t seem like revolutionary, but it sort of created this directed gameplay that felt optional, but really wasn’t.
Lex Fridman
I mean, I think it’s absolutely revolutionary. It basically changed gaming. It changed the way we see games. And it was so successful in part because it became a mechanism by which you could spend hundreds of hours, thousands of hours in the game. I mean, it’s kind of a, like, obviously… It’s one of those… All these great ideas are always like this, right? In retrospect, you’re like, “Well, obviously if you make the path of least resistance quest-driven gameplay, then it’s gonna be the reason that most people play.” But it is true that… I’m with you on… I both like the quests and Cow Level.
I mean, I think it’s absolutely revolutionary. It basically changed gaming. It changed the way we see games. And it was so successful in part because it became a mechanism by which you could spend hundreds of hours, thousands of hours in the game. I mean, it’s kind of a, like, obviously… It’s one of those… All these great ideas are always like this, right? In retrospect, you’re like, “Well, obviously if you make the path of least resistance quest-driven gameplay, then it’s gonna be the reason that most people play.” But it is true that… I’m with you on… I both like the quests and Cow Level.
Lex Fridman
I guess you have to design for everybody. That’s the tricky thing. Like, how do you fine-tune this? If you think of it as a loop of like accept quest, kill 10 rats, turn in quest, ding, level up—that loop. Like, how do you fine-tune that so it’s maximum fun or fun for the maximum number of people? Is it… How- how difficult is that?
I guess you have to design for everybody. That’s the tricky thing. Like, how do you fine-tune this? If you think of it as a loop of like accept quest, kill 10 rats, turn in quest, ding, level up—that loop. Like, how do you fine-tune that so it’s maximum fun or fun for the maximum number of people? Is it… How- how difficult is that?
Jeff Kaplan
It’s extremely difficult. And not everybody’s good at doing that. We all, to some degree, lack the self-awareness of how we tick. So we’re all different types of gamers, but if you ask me to describe the type of gamer I am, I might actually be giving more of a picture of the type of gamer I wish I was or the type of gamer I want you to think I am versus the type of gamer I actually am.
It’s extremely difficult. And not everybody’s good at doing that. We all, to some degree, lack the self-awareness of how we tick. So we’re all different types of gamers, but if you ask me to describe the type of gamer I am, I might actually be giving more of a picture of the type of gamer I wish I was or the type of gamer I want you to think I am versus the type of gamer I actually am.
Jeff Kaplan
By playing lots of games, you cannot be an exceptional game designer without playing the shit out of as much as you can and understanding on a deep level. And the weirdest part about it is you’re not just looking for the greatest hits. You learn just as much from a shitty game that you do from an amazing game. And also—like, a lousy game can have a great system that was tuned wrong, or lacked the correct interface, or they didn’t put the right visceral polish on it. There’s an executional aspect to all of it. When I’m playing, I’m not only, like, thinking about what makes this fun, I’m thinking about what makes this not fun. But I’m also watching everyone around me. My wife plays games, my kids play games.
By playing lots of games, you cannot be an exceptional game designer without playing the shit out of as much as you can and understanding on a deep level. And the weirdest part about it is you’re not just looking for the greatest hits. You learn just as much from a shitty game that you do from an amazing game. And also—like, a lousy game can have a great system that was tuned wrong, or lacked the correct interface, or they didn’t put the right visceral polish on it. There’s an executional aspect to all of it. When I’m playing, I’m not only, like, thinking about what makes this fun, I’m thinking about what makes this not fun. But I’m also watching everyone around me. My wife plays games, my kids play games.
Jeff Kaplan
And understanding, like, well, what do they do and how are they different to me? Why are they finding enjoyment in this? Why are they not? What’s frustrating? What did they miss?
And understanding, like, well, what do they do and how are they different to me? Why are they finding enjoyment in this? Why are they not? What’s frustrating? What did they miss?
Lex Fridman
And being raw honest with exactly what you’re saying. I mean, if I were to analyze the kind of gamer I am, why do I enjoy Cow Level? And why, above that, why do I enjoy loot? Why is loot so fun? Like, what is it about opening a chest and getting a bunch of stuff? I mean, that might be like at the core of what I enjoy about gaming. That, and walking around a beautiful world with nice music.
And being raw honest with exactly what you’re saying. I mean, if I were to analyze the kind of gamer I am, why do I enjoy Cow Level? And why, above that, why do I enjoy loot? Why is loot so fun? Like, what is it about opening a chest and getting a bunch of stuff? I mean, that might be like at the core of what I enjoy about gaming. That, and walking around a beautiful world with nice music.
Jeff Kaplan
As a game designer, I am, at best, a quack psychologist. You know? We can motivate you to do some weird things. The two driving motivators are extrinsic and intrinsic. And all of us, at different times in our lives, in our gaming careers, whatever, we can shift from being intrinsically motivated to being extrinsically motivated. Obviously, loot is a big extrinsic motivation, but even saying that is too simplistic. Like, for example, on the loot boxes of Overwatch, there’s a masterfully designed system that was designed by a game designer, not by a businessperson or whatever. Like, not a commercial person. But beyond that, we also had a really good team who said the visceral opening of the box, the sound it makes-
As a game designer, I am, at best, a quack psychologist. You know? We can motivate you to do some weird things. The two driving motivators are extrinsic and intrinsic. And all of us, at different times in our lives, in our gaming careers, whatever, we can shift from being intrinsically motivated to being extrinsically motivated. Obviously, loot is a big extrinsic motivation, but even saying that is too simplistic. Like, for example, on the loot boxes of Overwatch, there’s a masterfully designed system that was designed by a game designer, not by a businessperson or whatever. Like, not a commercial person. But beyond that, we also had a really good team who said the visceral opening of the box, the sound it makes-
Jeff Kaplan
… the graphics, like the way things spill out and animate, all of that is as satisfying as well. And you’re trying to… Like, there’s the lizard brain part of it. Of like, how does it… Like, I see chest. I know I’m gonna… It’s gonna feel good. It’s gonna feel good. And then there’s the spreadsheety part of it. Of, what does it have? Is it an upgrade? And I think great game designers know how to tap into both of those things.
… the graphics, like the way things spill out and animate, all of that is as satisfying as well. And you’re trying to… Like, there’s the lizard brain part of it. Of like, how does it… Like, I see chest. I know I’m gonna… It’s gonna feel good. It’s gonna feel good. And then there’s the spreadsheety part of it. Of, what does it have? Is it an upgrade? And I think great game designers know how to tap into both of those things.
Jeff Kaplan
You know, tap into the intrinsic and extrinsic. There’s… Like, when I was studying writing, you would study the elements of fiction. And, you know, these are just like basic things like plot and character development and setting and theme and whatever. And there’s no, like, textbook that exists for game design, at least none that has been introduced to me yet. But I think about, like, elements of fun.
You know, tap into the intrinsic and extrinsic. There’s… Like, when I was studying writing, you would study the elements of fiction. And, you know, these are just like basic things like plot and character development and setting and theme and whatever. And there’s no, like, textbook that exists for game design, at least none that has been introduced to me yet. But I think about, like, elements of fun.
Jeff Kaplan
What are the things that create fun for players? And they’re not the same. Like, it really… Every human being is different. Like, progression is fun. Sense of progression that I’m investing. I’m putting an investment into this game, and then the game is recognizing my investment. That things like leveling, things like the amount of gold you have, those are all investment based. There’s mastery. There’s just pure raw skill. Creativity is one. And hand-in-hand with creativity is customization. And some of those can be aesthetic. Like, look at my customized character, and I have the black curly hair, and I put an earring in my character and I’m customizing in that way. The other is customizing my build.
What are the things that create fun for players? And they’re not the same. Like, it really… Every human being is different. Like, progression is fun. Sense of progression that I’m investing. I’m putting an investment into this game, and then the game is recognizing my investment. That things like leveling, things like the amount of gold you have, those are all investment based. There’s mastery. There’s just pure raw skill. Creativity is one. And hand-in-hand with creativity is customization. And some of those can be aesthetic. Like, look at my customized character, and I have the black curly hair, and I put an earring in my character and I’m customizing in that way. The other is customizing my build.
Jeff Kaplan
I’m gonna come up with a whirlwind barbarian and I’m the first to do it. These are all elements of fun that designers can tap into, and in fact are frequently tapping into. But they’re never defined anywhere, and I find that players drift. Like, I’m the type of player who’s not really loot motivated. I’m more motivated by seeing the content the world has to offer. And often that takes me on a detour of being loot motivated, because there might be a dragon or a demon somewhere that I can’t beat without this level of armor and sword. So now I’m loot motivated for some period of time, to get back to being content motivated.
I’m gonna come up with a whirlwind barbarian and I’m the first to do it. These are all elements of fun that designers can tap into, and in fact are frequently tapping into. But they’re never defined anywhere, and I find that players drift. Like, I’m the type of player who’s not really loot motivated. I’m more motivated by seeing the content the world has to offer. And often that takes me on a detour of being loot motivated, because there might be a dragon or a demon somewhere that I can’t beat without this level of armor and sword. So now I’m loot motivated for some period of time, to get back to being content motivated.
Jeff Kaplan
Or if I’m having trouble defeating a boss, I might have to go back and look at the skills and abilities that my character’s using, and I have to go into creativity mode. “Oh, he has that one AE where he…” Area of effect. “…where he puts a curse on me.” And, you know, “If I had this counterability to the curse, I could beat the boss to get the loot, to get to the next boss.” These are all cycles that are tapping into all those different elements of fun.
Or if I’m having trouble defeating a boss, I might have to go back and look at the skills and abilities that my character’s using, and I have to go into creativity mode. “Oh, he has that one AE where he…” Area of effect. “…where he puts a curse on me.” And, you know, “If I had this counterability to the curse, I could beat the boss to get the loot, to get to the next boss.” These are all cycles that are tapping into all those different elements of fun.
Single-player vs Multi-player
Lex Fridman
And ultimately enjoying and discovering what the world gives you. Has to offer to you. And you’re… You have a lot of hats as a gamer, so you love the RPG/MMORPG world, but you’re also a big shooter guy. Can you explain to me what fun in a shooter context is? And we’ll talk about Overwatch as a specific kind of fun. Maybe… But you’re also a huge fan of the ultra-realistic shooters. Call of Duty. What is the definition of fun there?
And ultimately enjoying and discovering what the world gives you. Has to offer to you. And you’re… You have a lot of hats as a gamer, so you love the RPG/MMORPG world, but you’re also a big shooter guy. Can you explain to me what fun in a shooter context is? And we’ll talk about Overwatch as a specific kind of fun. Maybe… But you’re also a huge fan of the ultra-realistic shooters. Call of Duty. What is the definition of fun there?
Jeff Kaplan
There’s a lot of skill and mastery. Off the cuff, flippant comment would be clicking heads, you know? I’m just trying to click heads.
There’s a lot of skill and mastery. Off the cuff, flippant comment would be clicking heads, you know? I’m just trying to click heads.
Lex Fridman
Okay.
Okay.
Jeff Kaplan
There’s an intimacy also to the first person camera. And now, not all shooters are first person. There is a large trend these days to third person. I really think PUBG and Fortnite sort of opened that third person shooter door. And you’re seeing games like ARC Raiders are third person. But to me, nothing is as pure as first person. Like you’re- … literally living in the world as that being. You can look at your hands, and it’s that pure visceral test of skill of, “Can you click on the thing fast enough?” And when it’s PvP based, you know that’s coming at you.
There’s an intimacy also to the first person camera. And now, not all shooters are first person. There is a large trend these days to third person. I really think PUBG and Fortnite sort of opened that third person shooter door. And you’re seeing games like ARC Raiders are third person. But to me, nothing is as pure as first person. Like you’re- … literally living in the world as that being. You can look at your hands, and it’s that pure visceral test of skill of, “Can you click on the thing fast enough?” And when it’s PvP based, you know that’s coming at you.
Lex Fridman
Could you lay out for people who don’t necessarily know what PvP and PvE is? And single player-
Could you lay out for people who don’t necessarily know what PvP and PvE is? And single player-
Jeff Kaplan
Absolutely.
Absolutely.
Lex Fridman
… multiplayer, massively online multiplayer?
… multiplayer, massively online multiplayer?
Jeff Kaplan
So PvP is player versus player. So that means a combative, you know… If Lex and I are up against each other, we’re attacking each other. We call that PvP. You can get killed by another player. Player versus environment is anytime you’re shooting computer-controlled opponents. So if it’s a game about dragons, the dragon is the E, the environment in PvE.
So PvP is player versus player. So that means a combative, you know… If Lex and I are up against each other, we’re attacking each other. We call that PvP. You can get killed by another player. Player versus environment is anytime you’re shooting computer-controlled opponents. So if it’s a game about dragons, the dragon is the E, the environment in PvE.
Lex Fridman
And we should say that PvP and PvE, the P might be multiple players. It could be five versus five, six versus six for PvP. And for PvE, it could be, like, raids where it’s multiple people, large groups of people going against the AI.
And we should say that PvP and PvE, the P might be multiple players. It could be five versus five, six versus six for PvP. And for PvE, it could be, like, raids where it’s multiple people, large groups of people going against the AI.
Jeff Kaplan
Yep. So single player, that’s a game that you play totally by yourself. Like, you don’t play with anybody else. You can’t play with anybody else. It’s not networked to play with other people. For example, I’m playing a game called Story of Seasons right now on the Switch, which I just play by myself. I have my farm. You know, there’s a town. I’m meeting people in the town, and no one can come and join me and interact with that. So it’s a very controlled experience. Single player games are very difficult, or they can be very difficult and expensive in terms of production to create. Like, if you think of a game, like Uncharted or Last of Us that’s made by Naughty Dog, like, those are kind of the preeminent best single player games you could talk about.
Yep. So single player, that’s a game that you play totally by yourself. Like, you don’t play with anybody else. You can’t play with anybody else. It’s not networked to play with other people. For example, I’m playing a game called Story of Seasons right now on the Switch, which I just play by myself. I have my farm. You know, there’s a town. I’m meeting people in the town, and no one can come and join me and interact with that. So it’s a very controlled experience. Single player games are very difficult, or they can be very difficult and expensive in terms of production to create. Like, if you think of a game, like Uncharted or Last of Us that’s made by Naughty Dog, like, those are kind of the preeminent best single player games you could talk about.
Jeff Kaplan
They’re very handcrafted. Every experience is made just for you. One up from that is what I call co-op. And these terms become interchangeable, so I’m using some semantics here. But co-op is any cooperative experience that we can play together, but we’re sharing an exact same experience very intentionally. And it’s me sharing that experience only with other people that I know. So a great example of a cooperative game, maybe one of the best of all time, was Left 4 Dead, which is a game where you and three other people go in and you fight, like, hordes of zombies, and you try to progress through to the end safe room. It’s a very cooperative experience. A game like Diablo IV, you can play cooperatively with other people.
They’re very handcrafted. Every experience is made just for you. One up from that is what I call co-op. And these terms become interchangeable, so I’m using some semantics here. But co-op is any cooperative experience that we can play together, but we’re sharing an exact same experience very intentionally. And it’s me sharing that experience only with other people that I know. So a great example of a cooperative game, maybe one of the best of all time, was Left 4 Dead, which is a game where you and three other people go in and you fight, like, hordes of zombies, and you try to progress through to the end safe room. It’s a very cooperative experience. A game like Diablo IV, you can play cooperatively with other people.
Jeff Kaplan
Now, one up from that is multiplayer, and that’s when you’re engaging with strangers who are in the same world that you might not have the same cooperative goals as. You might have very opposed goals to them. You might PvP them, or they might just be random strangers that you pass in a town or city and never see again. And then massively multiplayer, which is what the MMO online sort of stands for, massively multiplayer online game, that’s when you’re breaking into thousands of players. And the worlds become really, really big at that point.
Now, one up from that is multiplayer, and that’s when you’re engaging with strangers who are in the same world that you might not have the same cooperative goals as. You might have very opposed goals to them. You might PvP them, or they might just be random strangers that you pass in a town or city and never see again. And then massively multiplayer, which is what the MMO online sort of stands for, massively multiplayer online game, that’s when you’re breaking into thousands of players. And the worlds become really, really big at that point.
Lex Fridman
By the way, we should say that the co-op could be remote connection, but there’s also, what would you call it, couch co-op where you have two people. Some games are really designed well for the experience of two humans sitting together and playing the game together. Which is a really tricky thing to design for, but if it’s done well, it’s a… It’s a really fulfilling experience. Like, with a friend, with a loved one, you can, like, play a game together. And Diablo IV, I should say, is an example of a game that does that really well. They do couch co-op. Like, two people can play Diablo sitting together and there’s a real intimate experience in that.
By the way, we should say that the co-op could be remote connection, but there’s also, what would you call it, couch co-op where you have two people. Some games are really designed well for the experience of two humans sitting together and playing the game together. Which is a really tricky thing to design for, but if it’s done well, it’s a… It’s a really fulfilling experience. Like, with a friend, with a loved one, you can, like, play a game together. And Diablo IV, I should say, is an example of a game that does that really well. They do couch co-op. Like, two people can play Diablo sitting together and there’s a real intimate experience in that.
Jeff Kaplan
Yeah, couch co-op—it’s funny, ’cause it actually, like, predates the couch even. Some of those old arcade games- … like, would have two joysticks on them and then you could play- … with somebody else. Or there’s, you know, the famous game Gauntlet—
Yeah, couch co-op—it’s funny, ’cause it actually, like, predates the couch even. Some of those old arcade games- … like, would have two joysticks on them and then you could play- … with somebody else. Or there’s, you know, the famous game Gauntlet—
Jeff Kaplan
… had four joysticks and four people playing together. And then anybody who grew up in that early console era, like, you know, NES, Sega Genesis was a legendary one. We would sit and we’d play NHL 93- … on the couch. And anybody who lost, you’d lose the controller. And you could play that with up to four people playing, or we… I remember one of the big games that came out was Mortal Kombat. And we would play Mortal Kombat on the Sega Genesis, and it was the house rules were, you know, whoever lost, so whether you were in your college dorm or just some buddy’s apartment and there’s five people there, you’re constantly cycling everybody in and out. But there’s just a magic to multiplayer, of engaging and sharing in the experience-
… had four joysticks and four people playing together. And then anybody who grew up in that early console era, like, you know, NES, Sega Genesis was a legendary one. We would sit and we’d play NHL 93- … on the couch. And anybody who lost, you’d lose the controller. And you could play that with up to four people playing, or we… I remember one of the big games that came out was Mortal Kombat. And we would play Mortal Kombat on the Sega Genesis, and it was the house rules were, you know, whoever lost, so whether you were in your college dorm or just some buddy’s apartment and there’s five people there, you’re constantly cycling everybody in and out. But there’s just a magic to multiplayer, of engaging and sharing in the experience-
Jeff Kaplan
… with other people. That’s why I’ve always… I’ve never made a single-player game. Uh, I have great admiration for them. I don’t know if I could do it. The challenge… The reason I love multiplayer so much, the way I describe being a game director or game designer on a multiplayer game, it’s like imagine if you were gonna be a movie director, and you were gonna have all these actors and set designers and props and, you know, writers and scripts and all of this stuff, and your goal was to get a certain movie made. But we’re gonna ask you, the director, to just… You’re gonna leave the room. You can set it all up ahead of time, and then you’re not allowed to be there or talk to anybody involved in it.
… with other people. That’s why I’ve always… I’ve never made a single-player game. Uh, I have great admiration for them. I don’t know if I could do it. The challenge… The reason I love multiplayer so much, the way I describe being a game director or game designer on a multiplayer game, it’s like imagine if you were gonna be a movie director, and you were gonna have all these actors and set designers and props and, you know, writers and scripts and all of this stuff, and your goal was to get a certain movie made. But we’re gonna ask you, the director, to just… You’re gonna leave the room. You can set it all up ahead of time, and then you’re not allowed to be there or talk to anybody involved in it.
Jeff Kaplan
And now you need the actors to have an experience, and it’s just kind of the wildest, funnest experiment. Like-
And now you need the actors to have an experience, and it’s just kind of the wildest, funnest experiment. Like-
Lex Fridman
From a designer/creator perspective, ’cause you don’t know what the players will create, so that’s fun to see. You, you, you lay out the chessboard, you lay out the world, and then you get to watch what they create together. That’s true.
From a designer/creator perspective, ’cause you don’t know what the players will create, so that’s fun to see. You, you, you lay out the chessboard, you lay out the world, and then you get to watch what they create together. That’s true.
Jeff Kaplan
I struggle because sometimes people call me the anti-story guy in games, and that really hurts me because, like, I actually love story in games, and I counter that I’m the anti-shitty story guy. And what I mean by that is like, A, the most magical stories that I’ve ever heard come out of video games are player stories about, you know, the time I gave Barfa a potion and then I met him in real life. Like, that’s better than any video game writing that I’ve heard in a long while. The player story is so much more interesting. You know? “Lex, why do you like the cow level so much?” “Tell me about some goofy time-” “… like a loot goblin drew you into the most danger.”
I struggle because sometimes people call me the anti-story guy in games, and that really hurts me because, like, I actually love story in games, and I counter that I’m the anti-shitty story guy. And what I mean by that is like, A, the most magical stories that I’ve ever heard come out of video games are player stories about, you know, the time I gave Barfa a potion and then I met him in real life. Like, that’s better than any video game writing that I’ve heard in a long while. The player story is so much more interesting. You know? “Lex, why do you like the cow level so much?” “Tell me about some goofy time-” “… like a loot goblin drew you into the most danger.”
Jeff Kaplan
“And… But there was another player there, and then…” You know, like, those are the stories that I think are more interesting from games. There are some exceptional writers in video games and some exceptional games at story. You know, I’ve mentioned Naughty Dog, like they’re kind of on another level. But Valve has amazing writing. The writing behind Half-Life 2, Marc Laidlaw; the writing behind Portal- … and Portal 2. I think it was Erik Wolpaw, who is hilarious, just amazing, and Rockstar.
“And… But there was another player there, and then…” You know, like, those are the stories that I think are more interesting from games. There are some exceptional writers in video games and some exceptional games at story. You know, I’ve mentioned Naughty Dog, like they’re kind of on another level. But Valve has amazing writing. The writing behind Half-Life 2, Marc Laidlaw; the writing behind Portal- … and Portal 2. I think it was Erik Wolpaw, who is hilarious, just amazing, and Rockstar.
Jeff Kaplan
Red Dead Redemption 2 is one of my favorite games of all time, and that’s a game where you can see the expertise and mastery of the game design and the narrative design, and the fact that you can have those player stories of just the goofy shit. Like, I remember… ‘Cause the controls are a little awkward in Red Dead for a PC player who’s playing on console. Like, I always get confused about, like, taking out my gun and putting it away, and what’s, you know, the L1 and L2.
Red Dead Redemption 2 is one of my favorite games of all time, and that’s a game where you can see the expertise and mastery of the game design and the narrative design, and the fact that you can have those player stories of just the goofy shit. Like, I remember… ‘Cause the controls are a little awkward in Red Dead for a PC player who’s playing on console. Like, I always get confused about, like, taking out my gun and putting it away, and what’s, you know, the L1 and L2.
Jeff Kaplan
Like, as a PC gamer, I’m just like, “Let me bind this stuff to where I want it.” And so like, you know, a guy in town rides by and he’s like, “Howdy, partner.” And I go to, like, give him the Arthur Morgan, you know, “Hey, what’s up?” back, and I just whip out my sawed-off shotgun and, like, blow his fucking head off. And then the whole town is like… Suddenly I’m, like, under… I’m wanted and I’m being chased, and then there’s a train that, like, takes out the posse, and-
Like, as a PC gamer, I’m just like, “Let me bind this stuff to where I want it.” And so like, you know, a guy in town rides by and he’s like, “Howdy, partner.” And I go to, like, give him the Arthur Morgan, you know, “Hey, what’s up?” back, and I just whip out my sawed-off shotgun and, like, blow his fucking head off. And then the whole town is like… Suddenly I’m, like, under… I’m wanted and I’m being chased, and then there’s a train that, like, takes out the posse, and-
Lex Fridman
Yes.
Yes.
Jeff Kaplan
It’s like those stories, and the fact that Red Dead can have, you know, this, like, touching, heartbreaking story of Arthur Morgan and his journey, but you can also have, you know, the player story of blowing off the poor guy that’s just trying to-
It’s like those stories, and the fact that Red Dead can have, you know, this, like, touching, heartbreaking story of Arthur Morgan and his journey, but you can also have, you know, the player story of blowing off the poor guy that’s just trying to-
Lex Fridman
And that’s the combination. And then Rockstar does a really good job with, you know, even in Grand Theft Auto with the radio. It can be kind of a side aspect to the game; that great writing there can create—help create the world— … with humor, with color, with depth, with heartbreak, all that kind of stuff.
And that’s the combination. And then Rockstar does a really good job with, you know, even in Grand Theft Auto with the radio. It can be kind of a side aspect to the game; that great writing there can create—help create the world— … with humor, with color, with depth, with heartbreak, all that kind of stuff.
Jeff Kaplan
There was a moment in Red Dead where it… There’s the Daniel Lanois song— …”That’s the Way It Is”. I just… I love Daniel Lanois, so the fact that somehow Rockstar landed him and like, was able to get that song out of him. And there’s this moment where you’re, like, riding back and they start that song, and- Everything up to then had been gorgeous, like, more of a score. There’s Woody Jackson, who’s, like, a really amazing game composer. He had done the score for that, and so nothing had been, like, lyrical with words. And then they play the Daniel Lanois song, and there’s, like, the quotes are coming back—
There was a moment in Red Dead where it… There’s the Daniel Lanois song— …”That’s the Way It Is”. I just… I love Daniel Lanois, so the fact that somehow Rockstar landed him and like, was able to get that song out of him. And there’s this moment where you’re, like, riding back and they start that song, and- Everything up to then had been gorgeous, like, more of a score. There’s Woody Jackson, who’s, like, a really amazing game composer. He had done the score for that, and so nothing had been, like, lyrical with words. And then they play the Daniel Lanois song, and there’s, like, the quotes are coming back—
Jeff Kaplan
… from, like, Dutch and Arthur Morgan, and I’m just like, “Goddam, this is, like… This is art.” You know, this is like—I know it’s supposed to be entertainment, I know it’s a business, but the top of the pyramid is art, and- … it just hit me emotionally.
… from, like, Dutch and Arthur Morgan, and I’m just like, “Goddam, this is, like… This is art.” You know, this is like—I know it’s supposed to be entertainment, I know it’s a business, but the top of the pyramid is art, and- … it just hit me emotionally.
Lex Fridman
Yeah, there’s certain games where, you know… I mean, that moment, you just imagine the number of people who shed a tear during that moment, and that’s just a reflection of how much you’re invested into this world, into these characters, and it’s a beautiful thing. I have to ask you about this, this image that you sent me. It’s super cool, so I’d love it if we could nerd out about it a little bit, the zone flow for the original World of Warcraft. There’s a bunch of zones. It’d be awesome if you kinda talk through how, like, this world is built. Take me to that time when you were designing this, before anyone else got a chance to play it.
Yeah, there’s certain games where, you know… I mean, that moment, you just imagine the number of people who shed a tear during that moment, and that’s just a reflection of how much you’re invested into this world, into these characters, and it’s a beautiful thing. I have to ask you about this, this image that you sent me. It’s super cool, so I’d love it if we could nerd out about it a little bit, the zone flow for the original World of Warcraft. There’s a bunch of zones. It’d be awesome if you kinda talk through how, like, this world is built. Take me to that time when you were designing this, before anyone else got a chance to play it.
Jeff Kaplan
All WoW stuff. It would start from that inspiration of Chris and the world. And, you know, it was so fun hanging out with Chris because we had whiteboards all over the place, and, you know, “Hey, Chris, we should make Eastern Kingdoms. What do you think it should be?” And he would just tell you the story of each of these as he’s just drawing. And Chris is a really talented artist, so the map would be gorgeous. I have lots of, like, photographs of Chris maps that he would just kind of whiteboard up. He’s like, you know, “Here’s the Dwarven Lands, there’s Wetlands with Khaz Modan up there, and that’s where this, you know, tribe of dwarves were from.” And then they, you know, humans are going to be down with Elwynn Forest.
All WoW stuff. It would start from that inspiration of Chris and the world. And, you know, it was so fun hanging out with Chris because we had whiteboards all over the place, and, you know, “Hey, Chris, we should make Eastern Kingdoms. What do you think it should be?” And he would just tell you the story of each of these as he’s just drawing. And Chris is a really talented artist, so the map would be gorgeous. I have lots of, like, photographs of Chris maps that he would just kind of whiteboard up. He’s like, you know, “Here’s the Dwarven Lands, there’s Wetlands with Khaz Modan up there, and that’s where this, you know, tribe of dwarves were from.” And then they, you know, humans are going to be down with Elwynn Forest.
Jeff Kaplan
And then Westfall, there’s, you know, this group called the Defias Brotherhood and they have a place called Deadmines.” So I would talk to Chris because you want to capture the spirit, like, as a game designer, you want to capture the experience that’s in people’s heads. So, like, take Burning Steppes, for example. Supposed to be one of the scariest places with lava and dragons and, you know, all this kind of stuff. That doesn’t feel like where you want to start. It feels like where you want to end, so you kind of work the world flow in a way that puts that at the end. But there was also kind of some magic to the original starting areas, where we gave the dwarves and the humans a free flight path between… The dwarf hometown was called Ironforge, the human hometown was called Stormwind.
And then Westfall, there’s, you know, this group called the Defias Brotherhood and they have a place called Deadmines.” So I would talk to Chris because you want to capture the spirit, like, as a game designer, you want to capture the experience that’s in people’s heads. So, like, take Burning Steppes, for example. Supposed to be one of the scariest places with lava and dragons and, you know, all this kind of stuff. That doesn’t feel like where you want to start. It feels like where you want to end, so you kind of work the world flow in a way that puts that at the end. But there was also kind of some magic to the original starting areas, where we gave the dwarves and the humans a free flight path between… The dwarf hometown was called Ironforge, the human hometown was called Stormwind.
Jeff Kaplan
And we allowed you to fly for free. So, like, these little newbies who were, you know, level five or something, if you played a dwarf and I played a human, I’m like, “Oh, Lex, don’t worry, I’ll come. You know, I’ll come to Ironforge and we’ll hook up and I’ll just fly out to you,” which is the magic of World of Warcraft. You have to fly over Burning Steppes and Searing Gorge, and you look down and you’re like, “Holy shit, that looks scary and dangerous.” And it plants that seed of things to come.
And we allowed you to fly for free. So, like, these little newbies who were, you know, level five or something, if you played a dwarf and I played a human, I’m like, “Oh, Lex, don’t worry, I’ll come. You know, I’ll come to Ironforge and we’ll hook up and I’ll just fly out to you,” which is the magic of World of Warcraft. You have to fly over Burning Steppes and Searing Gorge, and you look down and you’re like, “Holy shit, that looks scary and dangerous.” And it plants that seed of things to come.
Lex Fridman
So you’ve designed some incredible quests. Is there any that stand out that you’re proud of or ashamed of? I mean, you famously have designed the Green Hills of Stranglethorn quest. One of the most infamous quests in the history of WoW, of gaming, where you had to collect a bunch of pages, or… Green Hills of Stranglethorn, maybe, can you comment on that one or any quest that just springs to mind?
So you’ve designed some incredible quests. Is there any that stand out that you’re proud of or ashamed of? I mean, you famously have designed the Green Hills of Stranglethorn quest. One of the most infamous quests in the history of WoW, of gaming, where you had to collect a bunch of pages, or… Green Hills of Stranglethorn, maybe, can you comment on that one or any quest that just springs to mind?
Jeff Kaplan
Green Hills of Stranglethorn holds a lot of emotional value for me because amongst WoW players back in the day, it was unanimously hated as one of the shittiest, most annoying quests. But it holds a really special place in my heart. First of all, it’s one of the few times that I just, like, wrote a short story that’s actually in the game. It’s me paying homage to Hemingway, and the guy who gives you the quest, his name is Hemet Nesingwary, which is just me rearranging the letters of Hemingway. There’s another quest giver there that’s Kerouac’s name also mixed up. And then it was the typical hubris of a junior game designer who thinks he’s clever but is actually a dipshit. That’s- That’s the Green Hills of Stranglethorn, like, summed up.
Green Hills of Stranglethorn holds a lot of emotional value for me because amongst WoW players back in the day, it was unanimously hated as one of the shittiest, most annoying quests. But it holds a really special place in my heart. First of all, it’s one of the few times that I just, like, wrote a short story that’s actually in the game. It’s me paying homage to Hemingway, and the guy who gives you the quest, his name is Hemet Nesingwary, which is just me rearranging the letters of Hemingway. There’s another quest giver there that’s Kerouac’s name also mixed up. And then it was the typical hubris of a junior game designer who thinks he’s clever but is actually a dipshit. That’s- That’s the Green Hills of Stranglethorn, like, summed up.
Jeff Kaplan
So, like, I wrote the story over, like, it was, I think, winter break, like, everybody was gone and I just was so happy to be in the office, you know, I’m at Blizzard by myself writing late at night. And the whole idea, and this is, this is very much what I call ant farm designer, which is bad. Which is, you know, you’re the game designer who’s playing God, and players are the ants in your ant farm, and you want to see what they’re gonna do, which is not the correct way to be a good multiplayer designer. But I hadn’t learned that yet, and there’s a really great famous Sid Meier quote where he says there’s three types of fun. Fun for the player, fun for the designer, and fun for the computer.
So, like, I wrote the story over, like, it was, I think, winter break, like, everybody was gone and I just was so happy to be in the office, you know, I’m at Blizzard by myself writing late at night. And the whole idea, and this is, this is very much what I call ant farm designer, which is bad. Which is, you know, you’re the game designer who’s playing God, and players are the ants in your ant farm, and you want to see what they’re gonna do, which is not the correct way to be a good multiplayer designer. But I hadn’t learned that yet, and there’s a really great famous Sid Meier quote where he says there’s three types of fun. Fun for the player, fun for the designer, and fun for the computer.
Jeff Kaplan
And we catch ourselves, we’re like, you know, we gotta be really care… It has to be fun for the player, not fun for us. So this Green Hills of Stranglethorn quest was like an ant farm design of, I’m gonna write this, honestly, probably pretty shitty story, I haven’t read it since 2003 so God only knows if it’s any good. But I wrote the story and then I divided it up into all of these different pages. And the quest giver, Hemet Nesingwary, wants you to put together, like, the story’s like, he wrote this book, but then the pages got scattered across Stranglethorn Vale. And some… When you’re doing quest design, you’re really thinking about the player flow and you’re directing them from quest giver hubs out until these destinations, and you want them to do all the destinations.
And we catch ourselves, we’re like, you know, we gotta be really care… It has to be fun for the player, not fun for us. So this Green Hills of Stranglethorn quest was like an ant farm design of, I’m gonna write this, honestly, probably pretty shitty story, I haven’t read it since 2003 so God only knows if it’s any good. But I wrote the story and then I divided it up into all of these different pages. And the quest giver, Hemet Nesingwary, wants you to put together, like, the story’s like, he wrote this book, but then the pages got scattered across Stranglethorn Vale. And some… When you’re doing quest design, you’re really thinking about the player flow and you’re directing them from quest giver hubs out until these destinations, and you want them to do all the destinations.
Jeff Kaplan
But sometimes we would do these bridging quests where you could do anything in the zone and it sort of had this overlap. And so the pages of Green Hills of Stranglethorn could be looted off of any creature anywhere in Stranglethorn Vale, and it was kind of like that McDonald’s Monopoly game where you have to have all the pieces or else you’re not gonna win. But where I really went south, I don’t think the idea in a vacuum is horrible, but where this really fell apart was the interface of World of Warcraft wasn’t set up. Like the pages didn’t stack, there wasn’t a dedicated container to put all the pages in, so players had very limited bag space. And as they’re fighting in Stranglethorn Vale, I’m just shitting up their inventory with all of these pages and they only needed so many.
But sometimes we would do these bridging quests where you could do anything in the zone and it sort of had this overlap. And so the pages of Green Hills of Stranglethorn could be looted off of any creature anywhere in Stranglethorn Vale, and it was kind of like that McDonald’s Monopoly game where you have to have all the pieces or else you’re not gonna win. But where I really went south, I don’t think the idea in a vacuum is horrible, but where this really fell apart was the interface of World of Warcraft wasn’t set up. Like the pages didn’t stack, there wasn’t a dedicated container to put all the pages in, so players had very limited bag space. And as they’re fighting in Stranglethorn Vale, I’m just shitting up their inventory with all of these pages and they only needed so many.
Jeff Kaplan
Like you might get unlucky and you have like three page fives that are just junk in your inventory, and I might have like eight page sixes. And then everybody… And this was the goal, like the, the designer trying to puppeteer everybody. Everybody in Stranglethorn chat is like, “Hey, I’m looking for a page six. Anyone got a page three?” And that was like my fantasy as a designer of like, and then they’re gonna be social and meet each other, and players are gonna be appreciative for each other, but really all everybody did was just no… Eventually, no one did the quest. They just were super annoyed, or they went to the Auction House. So the quest is famous in that it was so aggravating and annoying and it just became a way…
Like you might get unlucky and you have like three page fives that are just junk in your inventory, and I might have like eight page sixes. And then everybody… And this was the goal, like the, the designer trying to puppeteer everybody. Everybody in Stranglethorn chat is like, “Hey, I’m looking for a page six. Anyone got a page three?” And that was like my fantasy as a designer of like, and then they’re gonna be social and meet each other, and players are gonna be appreciative for each other, but really all everybody did was just no… Eventually, no one did the quest. They just were super annoyed, or they went to the Auction House. So the quest is famous in that it was so aggravating and annoying and it just became a way…
Jeff Kaplan
It not only became a way for me to learn from my mistakes, but because I was very open with the fact that I didn’t think it was good and that the quest had failed, it opened the door for us at Blizzard to be critical of our own work. Like it’s always easier if you’re the first one to go out and say, “Hey, guys, I think I made one of the shittiest quests in the game and here’s why.” And then it sort of challenged people to make better versions of it.
It not only became a way for me to learn from my mistakes, but because I was very open with the fact that I didn’t think it was good and that the quest had failed, it opened the door for us at Blizzard to be critical of our own work. Like it’s always easier if you’re the first one to go out and say, “Hey, guys, I think I made one of the shittiest quests in the game and here’s why.” And then it sort of challenged people to make better versions of it.
How Blizzard made great video games
Lex Fridman
I mean, again, you continue to speak with so much humility. But WoW turned out to be one of the biggest games of all time both in terms of popularity, how many players play it, revenue, and critical acclaim. And then you rose to become a game director of WoW helping release Wrath of the Lich King, which by many is considered to be the greatest expansion. I mean, there’s a million questions I can ask here, but maybe this is also a good place to ask about the famous Blizzard polish. So Blizzard as a company has historically, and you were certainly a big part of that, delivered these games.
I mean, again, you continue to speak with so much humility. But WoW turned out to be one of the biggest games of all time both in terms of popularity, how many players play it, revenue, and critical acclaim. And then you rose to become a game director of WoW helping release Wrath of the Lich King, which by many is considered to be the greatest expansion. I mean, there’s a million questions I can ask here, but maybe this is also a good place to ask about the famous Blizzard polish. So Blizzard as a company has historically, and you were certainly a big part of that, delivered these games.
Lex Fridman
They were just, got so many pieces right and well-functioning and well-coordinated, and just feel finished in a way that a lot of other games don’t get right. So what does it take to take this gigantic game, this game played by millions of people, loved by millions of people, and deliver it in a way where it’s like it all just works?
They were just, got so many pieces right and well-functioning and well-coordinated, and just feel finished in a way that a lot of other games don’t get right. So what does it take to take this gigantic game, this game played by millions of people, loved by millions of people, and deliver it in a way where it’s like it all just works?
Jeff Kaplan
To have a level of polish is like a studio wide culture that has to be instilled in everybody, like no one can be satisfied with a bug. Every game is gonna have bugs, and Blizzard games have bugs. It’s a question of, how quickly do you fix them and with what urgency? And as players ourselves, if we’re playing as much as anybody else, we’re gonna be motivated to fix the bugs. There are some really tactical aspects to it, too. The quality assurance department at Blizzard is the best in the industry. Like the people who come and do QA at Blizzard, they are passionate gamers. Many of them want to be developers themselves, and they’re not just doing it for a job. They do it because they fucking love the game.
To have a level of polish is like a studio wide culture that has to be instilled in everybody, like no one can be satisfied with a bug. Every game is gonna have bugs, and Blizzard games have bugs. It’s a question of, how quickly do you fix them and with what urgency? And as players ourselves, if we’re playing as much as anybody else, we’re gonna be motivated to fix the bugs. There are some really tactical aspects to it, too. The quality assurance department at Blizzard is the best in the industry. Like the people who come and do QA at Blizzard, they are passionate gamers. Many of them want to be developers themselves, and they’re not just doing it for a job. They do it because they fucking love the game.
Jeff Kaplan
And the relationship we tried to develop between us on the development teams and QA was extremely tight. And whenever possible, we also tried to sit as many QA members up with the development team as possible, depending on the logistics of… You know, in the early days, we didn’t always have the space for all of QA to sit with us. We were very fortunate on the Overwatch team to have a large amount of QA sitting with us, and then developing that relationship. You know, in the early days there, there were these fears of like, “Well, QA can’t talk to the developers,” and trying to shatter that-
And the relationship we tried to develop between us on the development teams and QA was extremely tight. And whenever possible, we also tried to sit as many QA members up with the development team as possible, depending on the logistics of… You know, in the early days, we didn’t always have the space for all of QA to sit with us. We were very fortunate on the Overwatch team to have a large amount of QA sitting with us, and then developing that relationship. You know, in the early days there, there were these fears of like, “Well, QA can’t talk to the developers,” and trying to shatter that-
Jeff Kaplan
… of, because some of our QA members knew the game so inside out, you would just say to ’em like, “Hey, dude. Just message me anytime. Here’s my home number. Like, call me if there’s a bug. If you think we’re gonna get raked over the coals on this, you gotta speak up. I don’t care what the chain of command is. Like, we gotta fix this thing.” So QA was amazing.
… of, because some of our QA members knew the game so inside out, you would just say to ’em like, “Hey, dude. Just message me anytime. Here’s my home number. Like, call me if there’s a bug. If you think we’re gonna get raked over the coals on this, you gotta speak up. I don’t care what the chain of command is. Like, we gotta fix this thing.” So QA was amazing.
Lex Fridman
I mean, so can you speak to QA, quality assurance? At the peak of the craft, what does it entail? Like you’re basically experiencing the game and trying to figure out particular slices of that experience that could be improved?
I mean, so can you speak to QA, quality assurance? At the peak of the craft, what does it entail? Like you’re basically experiencing the game and trying to figure out particular slices of that experience that could be improved?
Jeff Kaplan
Yeah. People simplify the role by just, “Oh, these guys just get to play games all day and then, like, let us know if there’s a bug.” They are so systematic in the way they test stuff. They come up with these plans that are actually amazing of, like, who’s gonna test what. There’s a lot of regression testing that goes on. Within QA there will also be compatibility testing. The Blizzard compatibility department was amazing. Like, they had every card, every machine, every configuration, and they would roll through to make sure there wasn’t some quirk that was gonna come up on some video card or some motherboard that you weren’t expecting. But it was all very systematic. It wasn’t just Wild West, let’s play the game.
Yeah. People simplify the role by just, “Oh, these guys just get to play games all day and then, like, let us know if there’s a bug.” They are so systematic in the way they test stuff. They come up with these plans that are actually amazing of, like, who’s gonna test what. There’s a lot of regression testing that goes on. Within QA there will also be compatibility testing. The Blizzard compatibility department was amazing. Like, they had every card, every machine, every configuration, and they would roll through to make sure there wasn’t some quirk that was gonna come up on some video card or some motherboard that you weren’t expecting. But it was all very systematic. It wasn’t just Wild West, let’s play the game.
Jeff Kaplan
And then as a developer interacting with QA, you would find that there were certain specialists whether like, like for example, on Overwatch, there were a couple of players that… Like, we all were shooter players when we were making Overwatch, but I’m not like esports level shooter player. I’m like, you know, Gen Xer, “Remember Doom, how good I was”- … type of shooter player. But we had, you know, a couple of these QA specialists who, like, they could just snipe from 100 meters out and hit the shot every time and tell us if there was a frame of input delay, you know? And then you sit that person with an engineer and say, “Hey, I think there’s some input lag here.”
And then as a developer interacting with QA, you would find that there were certain specialists whether like, like for example, on Overwatch, there were a couple of players that… Like, we all were shooter players when we were making Overwatch, but I’m not like esports level shooter player. I’m like, you know, Gen Xer, “Remember Doom, how good I was”- … type of shooter player. But we had, you know, a couple of these QA specialists who, like, they could just snipe from 100 meters out and hit the shot every time and tell us if there was a frame of input delay, you know? And then you sit that person with an engineer and say, “Hey, I think there’s some input lag here.”
Lex Fridman
That’s amazing.
That’s amazing.
Jeff Kaplan
And sure enough, they’d be right. But you have to have that relationship where the devs trust QA. Or just even on, like World of Warcraft, they had a great relationship with QA in that they built out a full raid team to do the raids. And then you’re not only, like, looking for bugs, like, “Hey, the dragon was supposed to fly and instead it just, like, sunk through the world and the game crashed,” which would happen. But, like, if you really value QA, you’re asking them, “What do you… Dude, what do you think? You’re…” You know? Like, “10 million people are gonna see this. Your opinion, multiply it, you know? It matters. What do you think? You know? Are you having fun? Oh, yeah, this is cool. This isn’t cool.” So QA was important.
And sure enough, they’d be right. But you have to have that relationship where the devs trust QA. Or just even on, like World of Warcraft, they had a great relationship with QA in that they built out a full raid team to do the raids. And then you’re not only, like, looking for bugs, like, “Hey, the dragon was supposed to fly and instead it just, like, sunk through the world and the game crashed,” which would happen. But, like, if you really value QA, you’re asking them, “What do you… Dude, what do you think? You’re…” You know? Like, “10 million people are gonna see this. Your opinion, multiply it, you know? It matters. What do you think? You know? Are you having fun? Oh, yeah, this is cool. This isn’t cool.” So QA was important.
Jeff Kaplan
The other thing that was important is the Blizzard engineering, which you have to architect your game to be hotfixable.
The other thing that was important is the Blizzard engineering, which you have to architect your game to be hotfixable.
Jeff Kaplan
And what a hotfix is, games, there’s a couple ways to fix ’em. The way most of us know, ’cause all the software we have gets a patch, you know? You have to update it. You have to download a new version of it. Windows, you know, you get that annoying message, like, “There’s a new version of Windows.” And it takes, you know, a few minutes and you update it. You know, obviously, we patch our games and that’s where we fix a lot of bugs, but if you really wanna run a game like Overwatch or World of Warcraft successfully, you need master level engineers who have architected the client and server in such a way that you can hotfix the game on a dime. And what a hotfix is, is a server patch that no one’s client has to go down for.
And what a hotfix is, games, there’s a couple ways to fix ’em. The way most of us know, ’cause all the software we have gets a patch, you know? You have to update it. You have to download a new version of it. Windows, you know, you get that annoying message, like, “There’s a new version of Windows.” And it takes, you know, a few minutes and you update it. You know, obviously, we patch our games and that’s where we fix a lot of bugs, but if you really wanna run a game like Overwatch or World of Warcraft successfully, you need master level engineers who have architected the client and server in such a way that you can hotfix the game on a dime. And what a hotfix is, is a server patch that no one’s client has to go down for.
Lex Fridman
Mm-hmm. That’s because you’re dealing with a huge number of players and you discover an issue and you want to respond to that issue really quickly.
Mm-hmm. That’s because you’re dealing with a huge number of players and you discover an issue and you want to respond to that issue really quickly.
Jeff Kaplan
Yeah. There’s emergency issues like something’s crashing. Like, the worst case scenario is anytime the server’s crashing. Or in Overwatch, like, a really catastrophic bug would be something where you have to disable a hero. Like, someone found an exploit and you have to disable a hero from the lineup. You want to turn around that hotfix if you can in a half an hour, get that hero back live. You might have somebody who only plays that hero, and the only reason they’re gonna play Overwatch is because that hero’s active. You don’t want to wait for patches and you want to hotfix- … as fast as you can.
Yeah. There’s emergency issues like something’s crashing. Like, the worst case scenario is anytime the server’s crashing. Or in Overwatch, like, a really catastrophic bug would be something where you have to disable a hero. Like, someone found an exploit and you have to disable a hero from the lineup. You want to turn around that hotfix if you can in a half an hour, get that hero back live. You might have somebody who only plays that hero, and the only reason they’re gonna play Overwatch is because that hero’s active. You don’t want to wait for patches and you want to hotfix- … as fast as you can.
Lex Fridman
And then also to improve the game quickly to just even settle stuff to do that.
And then also to improve the game quickly to just even settle stuff to do that.
Jeff Kaplan
Yeah. Players feel it. Like, they… That’s where there’s this idea of, like, the love and the craftsmanship of the developer that you can feel. Like, any product, you know your iPhone or Android or, like, any computer or consumer product, you can feel when there are people who loved it behind it and aren’t just putting it out on a shelf. And games have that as well, where you can feel the heart and soul of the developer in the thing. And some of that’s, like, the joy and delight of, like, that there’s a Cow Level, right? That that’s… You know, you can feel the humanity of the development team-
Yeah. Players feel it. Like, they… That’s where there’s this idea of, like, the love and the craftsmanship of the developer that you can feel. Like, any product, you know your iPhone or Android or, like, any computer or consumer product, you can feel when there are people who loved it behind it and aren’t just putting it out on a shelf. And games have that as well, where you can feel the heart and soul of the developer in the thing. And some of that’s, like, the joy and delight of, like, that there’s a Cow Level, right? That that’s… You know, you can feel the humanity of the development team-
Jeff Kaplan
… through that. But another part of that is, like, do they clean up their fucking yard, you know? Does this game work? Is it… And it’s not just the bugs and the crashes. It’s, like, when balance gets wacky and stupid and, you know, suddenly everybody’s a Barbarian and whirlwinding and no one else will play anything else. You’re like, “We should probably fix that,” you know?
… through that. But another part of that is, like, do they clean up their fucking yard, you know? Does this game work? Is it… And it’s not just the bugs and the crashes. It’s, like, when balance gets wacky and stupid and, you know, suddenly everybody’s a Barbarian and whirlwinding and no one else will play anything else. You’re like, “We should probably fix that,” you know?
Lex Fridman
Oh, those were the days. I sadly was the Barbarian Whirlwind guy.
Oh, those were the days. I sadly was the Barbarian Whirlwind guy.
Jeff Kaplan
One-handed.
One-handed.
Lex Fridman
It was… Yeah, it brought so much joy. So a lot of people modern day think of you as Jeff from the Overwatch team.
It was… Yeah, it brought so much joy. So a lot of people modern day think of you as Jeff from the Overwatch team.
Jeff Kaplan
My name is Jeff from the Overwatch team. I’m Jeff from the Overwatch team. I’m Jeff from the Overwatch team.
My name is Jeff from the Overwatch team. I’m Jeff from the Overwatch team. I’m Jeff from the Overwatch team.
Lex Fridman
But y’all must have forgot, you were the game director of WoW in an era when WoW was one of the biggest games in the world. Just, you know, looking back, what wisdom can you draw from that time when you got to experience this era of gaming that changed gaming forever, where it’s millions of people playing this video game?
But y’all must have forgot, you were the game director of WoW in an era when WoW was one of the biggest games in the world. Just, you know, looking back, what wisdom can you draw from that time when you got to experience this era of gaming that changed gaming forever, where it’s millions of people playing this video game?
Jeff Kaplan
It was my first game I worked on, and I joined it as this entry level dude. I still have my offer letter from Blizzard, which was for 35K a year.
It was my first game I worked on, and I joined it as this entry level dude. I still have my offer letter from Blizzard, which was for 35K a year.
Jeff Kaplan
You know, that’s what I was making. And very shortly after WoW shipped, you know, Allen left as lead before the beta, or like right around the beta, and then Rob took over as the lead designer, and then he left the team very shortly after WoW shipped to go start StarCraft II. And he put myself and Tom Chilton in charge. Tom is a designer who… He was a great partner of mine and a great leader and he actually came from Ultima Online. And so I always looked up to Tom because he had a lot more experience than I did. And this is like early 2005, the world was on fire, the servers were barely running… WoW was just, had taken off like gangbusters, and they basically put me and Tom in charge of WoW. And at the time they promoted me, my title…
You know, that’s what I was making. And very shortly after WoW shipped, you know, Allen left as lead before the beta, or like right around the beta, and then Rob took over as the lead designer, and then he left the team very shortly after WoW shipped to go start StarCraft II. And he put myself and Tom Chilton in charge. Tom is a designer who… He was a great partner of mine and a great leader and he actually came from Ultima Online. And so I always looked up to Tom because he had a lot more experience than I did. And this is like early 2005, the world was on fire, the servers were barely running… WoW was just, had taken off like gangbusters, and they basically put me and Tom in charge of WoW. And at the time they promoted me, my title…
Jeff Kaplan
I didn’t even have a lead title, my title was Senior Game Designer. And Tom and I were running the design of WoW at that time. So I thought it was totally normal, and I thought what we were experiencing with WoW was just normal for making a video game because it was the first video game that I had worked on. I thought it was the funnest joyride because we were working on WoW, we were still working insane hours and then I’d get home, eat dinner, and then me and my wife would log in and play WoW, you know, for four hours, and then I’d go in the next day and I’d work… And it was just this… My whole life was World of Warcraft. And I loved it.
I didn’t even have a lead title, my title was Senior Game Designer. And Tom and I were running the design of WoW at that time. So I thought it was totally normal, and I thought what we were experiencing with WoW was just normal for making a video game because it was the first video game that I had worked on. I thought it was the funnest joyride because we were working on WoW, we were still working insane hours and then I’d get home, eat dinner, and then me and my wife would log in and play WoW, you know, for four hours, and then I’d go in the next day and I’d work… And it was just this… My whole life was World of Warcraft. And I loved it.
Jeff Kaplan
Like I loved everything from, you know, the creative meetings with Chris Metzen and just what an inspiration and muse he was, down to the simplest, dumbest design stuff that like we as game designers, like, you wanna talk about why a button is in the lower-left versus lower-right and what does that mean? That’s like two hours of discussion. And is there a better way? Like the 10,000 minutiae problems were thrilling to me. And then also the big disasters. Like the big… I had in the early days of WoW, we didn’t really have all the processes in place for, like, how to deal with being a successful online game, and I literally had GMs, like game masters, these are customer support guys, calling my home phone at 3:00 in the morning.
Like I loved everything from, you know, the creative meetings with Chris Metzen and just what an inspiration and muse he was, down to the simplest, dumbest design stuff that like we as game designers, like, you wanna talk about why a button is in the lower-left versus lower-right and what does that mean? That’s like two hours of discussion. And is there a better way? Like the 10,000 minutiae problems were thrilling to me. And then also the big disasters. Like the big… I had in the early days of WoW, we didn’t really have all the processes in place for, like, how to deal with being a successful online game, and I literally had GMs, like game masters, these are customer support guys, calling my home phone at 3:00 in the morning.
Jeff Kaplan
Like, I remember this one time there was some faction token in Stranglethorn Vale and they figured out a way to exploit it, and this GM calls me panicked, it’s 3:00 in the morning. He’s like, “I’m just spawning…” Uh, what, what did we call ’em? Guardians of Blizzard. They were these giant infernals that we just made that instantly death touched anything. We used to have them when we were in the beta, like off in the distance of places players weren’t supposed to get in case they cheated their way there. And this GM is just spawning them all over Stranglethorn Vale because he’s worried because the players are exploiting. Yeah. It’s like 3:00 in the morning and I’m talking in hushed tones because my wife is sleeping right next to the bed.
Like, I remember this one time there was some faction token in Stranglethorn Vale and they figured out a way to exploit it, and this GM calls me panicked, it’s 3:00 in the morning. He’s like, “I’m just spawning…” Uh, what, what did we call ’em? Guardians of Blizzard. They were these giant infernals that we just made that instantly death touched anything. We used to have them when we were in the beta, like off in the distance of places players weren’t supposed to get in case they cheated their way there. And this GM is just spawning them all over Stranglethorn Vale because he’s worried because the players are exploiting. Yeah. It’s like 3:00 in the morning and I’m talking in hushed tones because my wife is sleeping right next to the bed.
Jeff Kaplan
I’m doing this ’cause it was actually like before the cell phone days when I actually had a landline. But that’s just how… And I loved it. I loved the thrill of those big moments, the minutiae. And I felt like through the running WoW Live, which was me and Tom together with an amazing team, we kind of learned how to be the WoW team. And putting WoW in a box and shipping it was like only chapter one in a 12-chapter book essentially. And that first how to run the game, how to patch it, what type of content, how to deal with emergencies, what should our customer support be like. I mean, we would debate should we have a launcher or not. You know, in the early days, the only reason the launcher existed in WoW was to run anti-cheat on your machine.
I’m doing this ’cause it was actually like before the cell phone days when I actually had a landline. But that’s just how… And I loved it. I loved the thrill of those big moments, the minutiae. And I felt like through the running WoW Live, which was me and Tom together with an amazing team, we kind of learned how to be the WoW team. And putting WoW in a box and shipping it was like only chapter one in a 12-chapter book essentially. And that first how to run the game, how to patch it, what type of content, how to deal with emergencies, what should our customer support be like. I mean, we would debate should we have a launcher or not. You know, in the early days, the only reason the launcher existed in WoW was to run anti-cheat on your machine.
Jeff Kaplan
And we had a moment where we figured out how to put that into the game and out of the launcher. And it was the first time I ever really had an in-depth conversation with Mike Morhaime. He’s like, “You gotta bring the launcher back, guys.” We’re like, “Why?” He’s like, “There’s no better way for us to talk to our players.” Um, and I remember trying to hide the launcher. And to this day, Mike was right. Like, that launcher turned out to be the best thing we ever had. That’s essentially what Battle.net has morphed into these days. But all those decisions and when it came time to make Burning Crusade, you know, at that point, Tom and I were leads. We were full, they had actually promoted us.
And we had a moment where we figured out how to put that into the game and out of the launcher. And it was the first time I ever really had an in-depth conversation with Mike Morhaime. He’s like, “You gotta bring the launcher back, guys.” We’re like, “Why?” He’s like, “There’s no better way for us to talk to our players.” Um, and I remember trying to hide the launcher. And to this day, Mike was right. Like, that launcher turned out to be the best thing we ever had. That’s essentially what Battle.net has morphed into these days. But all those decisions and when it came time to make Burning Crusade, you know, at that point, Tom and I were leads. We were full, they had actually promoted us.
Jeff Kaplan
There was, there were two big exoduses of groups that quit Blizzard; they were disenfranchised if you can believe it. Like we just shipped World of Warcraft and this whole group just walked out the door. I was actually sitting, my desk faced Morhaime’s office, and I watched them all go in and quit, and they were the group that formed Carbine… which made the game WildStar. Ended up taking them 10 years to make, and they were just really unhappy with World of Warcraft, and they were unhappy with… I don’t know what they were unhappy with. They were unhappy enough to walk out the door right after we had shipped WoW.
There was, there were two big exoduses of groups that quit Blizzard; they were disenfranchised if you can believe it. Like we just shipped World of Warcraft and this whole group just walked out the door. I was actually sitting, my desk faced Morhaime’s office, and I watched them all go in and quit, and they were the group that formed Carbine… which made the game WildStar. Ended up taking them 10 years to make, and they were just really unhappy with World of Warcraft, and they were unhappy with… I don’t know what they were unhappy with. They were unhappy enough to walk out the door right after we had shipped WoW.
Lex Fridman
That’s incredible. Like, what, what is it? Just because they put their heart and soul into the game and they maybe get exhausted in a certain kind of way?
That’s incredible. Like, what, what is it? Just because they put their heart and soul into the game and they maybe get exhausted in a certain kind of way?
Jeff Kaplan
Yeah, and I don’t want to… It’s not fair of me to speak on their behalf. I think they were promised some compensation that they didn’t immediately see. I don’t know if the game… Like, here’s the weird part when you make a game. When you come up with the idea and you start pitching it to people, that’s the best the game is ever gonna be, and then you work on it. Like, you know, games I worked on take five years, you know? Overwatch was two and a half, three years. Every day you get close to ship, the imagination of the ideal game gets farther and farther from the reality, and you’re always shipping this, like, greatly sacrificed thing that nowhere near matches the imagination- … of the inception of the idea, so you become disenfranchised with the concept.
Yeah, and I don’t want to… It’s not fair of me to speak on their behalf. I think they were promised some compensation that they didn’t immediately see. I don’t know if the game… Like, here’s the weird part when you make a game. When you come up with the idea and you start pitching it to people, that’s the best the game is ever gonna be, and then you work on it. Like, you know, games I worked on take five years, you know? Overwatch was two and a half, three years. Every day you get close to ship, the imagination of the ideal game gets farther and farther from the reality, and you’re always shipping this, like, greatly sacrificed thing that nowhere near matches the imagination- … of the inception of the idea, so you become disenfranchised with the concept.
Lex Fridman
So in some sense, you’re shipping… You’re constantly in a state of disappointment. You’re basically shipping a lesser thing than you’ve been dreaming about.
So in some sense, you’re shipping… You’re constantly in a state of disappointment. You’re basically shipping a lesser thing than you’ve been dreaming about.
Jeff Kaplan
Yes.
Yes.
Lex Fridman
You’re doing less and less and less, saying no and no, and cutting, and all that kind of stuff. Yeah, it’s difficult, psychologically difficult, but nevertheless, the result when you zoom out, it’s one of the greatest games of all time that millions of people played for thousands of hours. It’s just… Did you ever have an experience, a realization how huge WoW was in terms of not, like, statistics on the server and so on, but the cultural impact it had?
You’re doing less and less and less, saying no and no, and cutting, and all that kind of stuff. Yeah, it’s difficult, psychologically difficult, but nevertheless, the result when you zoom out, it’s one of the greatest games of all time that millions of people played for thousands of hours. It’s just… Did you ever have an experience, a realization how huge WoW was in terms of not, like, statistics on the server and so on, but the cultural impact it had?
Jeff Kaplan
The first time was the first BlizzCon, which was in 2005. So when WoW shipped—and this is so weird to tell people—but on the team, not everyone, but a lot of us were very demoralized after WoW shipped. There were all sorts of issues with the servers because the game did way more successful than we expected it to do and the server load was just nuts. Like, we were just… We were doing our best to hire database programmers, you know, ’cause we just didn’t know how to deal with the sheer scope of the game. But when you’re an individual like… And at that time, like I mentioned, there were multiple exoduses of people who quit Blizzard. They went and formed a couple notable studios. One was Carbine, the other was Red 5.
The first time was the first BlizzCon, which was in 2005. So when WoW shipped—and this is so weird to tell people—but on the team, not everyone, but a lot of us were very demoralized after WoW shipped. There were all sorts of issues with the servers because the game did way more successful than we expected it to do and the server load was just nuts. Like, we were just… We were doing our best to hire database programmers, you know, ’cause we just didn’t know how to deal with the sheer scope of the game. But when you’re an individual like… And at that time, like I mentioned, there were multiple exoduses of people who quit Blizzard. They went and formed a couple notable studios. One was Carbine, the other was Red 5.
Jeff Kaplan
And we lost, like, kind of our core people. Like, when Red 5 started, that was our team leader, that was Mark Kern, and our art director, Bill Petris, they quit. When Carbine started, it was, I think, all of our animators and some of our best programmers and… Like, it’s really demoralizing when you lose team members like that, but then we were also underwater. Like, the servers aren’t running, we’re not able to keep up with demand, and we had to start putting patches out, and now we’re making patches like… For a while we had one animator who stuck around, and then eventually he left also, but you’re doing like, okay, we gotta now do a patch without an animator.
And we lost, like, kind of our core people. Like, when Red 5 started, that was our team leader, that was Mark Kern, and our art director, Bill Petris, they quit. When Carbine started, it was, I think, all of our animators and some of our best programmers and… Like, it’s really demoralizing when you lose team members like that, but then we were also underwater. Like, the servers aren’t running, we’re not able to keep up with demand, and we had to start putting patches out, and now we’re making patches like… For a while we had one animator who stuck around, and then eventually he left also, but you’re doing like, okay, we gotta now do a patch without an animator.
Jeff Kaplan
A lot of our art team was gone at that point, and you’re trying to keep the ship afloat and the morale was just in the shitter. Like, everybody felt very down on Team Two, the WoW team was called Team Two, and that we had somehow failed. And during that time, there was this idea to do BlizzCon, and the way that started was EverQuest had done these, like, meetups because they knew it was, like, a big guild social game, and people would get together at like some hotel ballroom and you’d sit with your guild at like a banquet room table.
A lot of our art team was gone at that point, and you’re trying to keep the ship afloat and the morale was just in the shitter. Like, everybody felt very down on Team Two, the WoW team was called Team Two, and that we had somehow failed. And during that time, there was this idea to do BlizzCon, and the way that started was EverQuest had done these, like, meetups because they knew it was, like, a big guild social game, and people would get together at like some hotel ballroom and you’d sit with your guild at like a banquet room table.
Jeff Kaplan
And to give credit where credit’s due, I remember sitting in the meeting for what was to become BlizzCon, it was Pardo who said, “Blizzard’s bigger than that. We’re not just one game, and I know everybody’s focused on World of Warcraft right now, we should do BlizzCon.” And at the time, we had a game called StarCraft: Ghost was in development, and that was getting ready to show, and there was Frozen Throne, which was the expansion to Warcraft III, but, like, we knew we were gonna make StarCraft II. And then there was a lot of motion happening with Blizzard North, which is a whole separate story, but there was like, hey, we could really do a cool show-
And to give credit where credit’s due, I remember sitting in the meeting for what was to become BlizzCon, it was Pardo who said, “Blizzard’s bigger than that. We’re not just one game, and I know everybody’s focused on World of Warcraft right now, we should do BlizzCon.” And at the time, we had a game called StarCraft: Ghost was in development, and that was getting ready to show, and there was Frozen Throne, which was the expansion to Warcraft III, but, like, we knew we were gonna make StarCraft II. And then there was a lot of motion happening with Blizzard North, which is a whole separate story, but there was like, hey, we could really do a cool show-
Jeff Kaplan
… that’s this BlizzCon thing. And at first, we kind of announced it and it just was crickets. You know when you’re, like, excited about something, you’re like, “Man, everybody’s gonna love. Like, we’re doing BlizzCon,” and everybody’s kinda like, “Crickets. What’s BlizzCon? Who cares?” And we’re, we’re idiots, we’re reading the forums, and the forums are just flaming us all the time, like, “There’s lag on this server and can’t log into that server.” And that was our perspective of what was happening. And then, like I said, give Mike Morhaime credit where credit’s due. He kept us committed to that launcher, and they put the BlizzCon tickets on the launcher, which they hadn’t done before. It was on the website.
… that’s this BlizzCon thing. And at first, we kind of announced it and it just was crickets. You know when you’re, like, excited about something, you’re like, “Man, everybody’s gonna love. Like, we’re doing BlizzCon,” and everybody’s kinda like, “Crickets. What’s BlizzCon? Who cares?” And we’re, we’re idiots, we’re reading the forums, and the forums are just flaming us all the time, like, “There’s lag on this server and can’t log into that server.” And that was our perspective of what was happening. And then, like I said, give Mike Morhaime credit where credit’s due. He kept us committed to that launcher, and they put the BlizzCon tickets on the launcher, which they hadn’t done before. It was on the website.
Jeff Kaplan
And so everybody who logged into World of Warcraft suddenly got this like, “Hey, we’re doing BlizzCon in Anaheim, do you wanna come?” Sold out instant. Like, instantly sold out. And when I showed up at that show, it… One of the most emotional things in my life. It was nothing but an outpouring of love. And up until that point, your perception was, because you’re just reading online and it was… The perception is such hatred, because people who are passionate online, they express themselves in the harshest ways ’cause it gets attention. You know, that’s the lesson I should’ve learned from my early days. And it’s such an unfortunate thing, because then you met these people in person and they loved World of Warcraft.
And so everybody who logged into World of Warcraft suddenly got this like, “Hey, we’re doing BlizzCon in Anaheim, do you wanna come?” Sold out instant. Like, instantly sold out. And when I showed up at that show, it… One of the most emotional things in my life. It was nothing but an outpouring of love. And up until that point, your perception was, because you’re just reading online and it was… The perception is such hatred, because people who are passionate online, they express themselves in the harshest ways ’cause it gets attention. You know, that’s the lesson I should’ve learned from my early days. And it’s such an unfortunate thing, because then you met these people in person and they loved World of Warcraft.
Jeff Kaplan
And all they wanted to do was talk about World of Warcraft and hear about what was coming next and be around other people who loved World of Warcraft, and-
And all they wanted to do was talk about World of Warcraft and hear about what was coming next and be around other people who loved World of Warcraft, and-
Lex Fridman
It’s incredible. It’s a fascinating theme, to me, about human nature, and it’s absolutely true, and I wish there was a thing that could be solved. But then again, maybe not. Maybe that’s just the way it is. But in person, all of the people that are passionate about a particular topic, and whatever that topic is, it could be games, it could be at conferences, technical conferences, they are all mostly full of love. And just the way they talk about stuff, they nerd out. Even the disagreements are drenched in this respect and appreciation and love for the game, for the topic. And online, you’re right, I don’t know if it’s because of popularity or clicks or so on, but it’s just the way of speaking on the internet is more mockery and-
It’s incredible. It’s a fascinating theme, to me, about human nature, and it’s absolutely true, and I wish there was a thing that could be solved. But then again, maybe not. Maybe that’s just the way it is. But in person, all of the people that are passionate about a particular topic, and whatever that topic is, it could be games, it could be at conferences, technical conferences, they are all mostly full of love. And just the way they talk about stuff, they nerd out. Even the disagreements are drenched in this respect and appreciation and love for the game, for the topic. And online, you’re right, I don’t know if it’s because of popularity or clicks or so on, but it’s just the way of speaking on the internet is more mockery and-
Jeff Kaplan
Cynical.
Cynical.
Lex Fridman
If you say, “I love this thing. Here’s an apple. I love apples,” or, “I love bananas. I love fruit…” Like, “I love X,” whatever. You just get made fun of. You get… And then so what the lesson you learn from that is, “Well, I’m just not going to speak up when I love something. I’m going to instead speak up when I, maybe how much I hate another thing that’s similar to it.” Or maybe join in when we’re making fun of a particular quirky thing, about, “Don’t you hate it when bananas are too ripe or too…” Versus like not saying the, calling out the elephant in the room is, “We’re all gathered here today ’cause we love the thing.”
If you say, “I love this thing. Here’s an apple. I love apples,” or, “I love bananas. I love fruit…” Like, “I love X,” whatever. You just get made fun of. You get… And then so what the lesson you learn from that is, “Well, I’m just not going to speak up when I love something. I’m going to instead speak up when I, maybe how much I hate another thing that’s similar to it.” Or maybe join in when we’re making fun of a particular quirky thing, about, “Don’t you hate it when bananas are too ripe or too…” Versus like not saying the, calling out the elephant in the room is, “We’re all gathered here today ’cause we love the thing.”
Lex Fridman
It’s interesting. It’s that aspect of the internet that I think is jarring to a lot of people depending on the game, but if you go to Discord or Reddit or so on, in the communities that love a particular video game, there’s a… If you’re not used to it, and I don’t often go, so when I go it’s like, “Wow, there’s a lot of, like, pretty intense kinda mockery and derision and so on.” But you get used to it pretty quick and you understand it. I just, I wish there was more love.
It’s interesting. It’s that aspect of the internet that I think is jarring to a lot of people depending on the game, but if you go to Discord or Reddit or so on, in the communities that love a particular video game, there’s a… If you’re not used to it, and I don’t often go, so when I go it’s like, “Wow, there’s a lot of, like, pretty intense kinda mockery and derision and so on.” But you get used to it pretty quick and you understand it. I just, I wish there was more love.
Online toxicity
Jeff Kaplan
I feel bad because I played a role in the earliest development of some of that online culture. It really was social media before it was called social media. You know, I ran a… I actually had this reputation for being edgier than I really was. There were a couple notable posts that survived 30 years that people like to look back on but they don’t look back on the ones where I’m just being chill. And that’s unfortunate. I think a lot as a game designer about the design of social media. And unfortunately, social media in general is designed in such a way where the maximum hyperbole works, and that’s how you get the most points is by being max hyperbolic.
I feel bad because I played a role in the earliest development of some of that online culture. It really was social media before it was called social media. You know, I ran a… I actually had this reputation for being edgier than I really was. There were a couple notable posts that survived 30 years that people like to look back on but they don’t look back on the ones where I’m just being chill. And that’s unfortunate. I think a lot as a game designer about the design of social media. And unfortunately, social media in general is designed in such a way where the maximum hyperbole works, and that’s how you get the most points is by being max hyperbolic.
Jeff Kaplan
And usually, unfortunately, it’s more in the negative direction than the positive direction. You know, if I say, “That’s, that’s a pretty nice mug. I’ve seen nicer, but I like this one,” no one’s interested in that. I have to either love this thing, or better, this thing’s a crime against humanity- … in some way. And it’s very self-reinforcing and everybody sort of feeds into it and-
And usually, unfortunately, it’s more in the negative direction than the positive direction. You know, if I say, “That’s, that’s a pretty nice mug. I’ve seen nicer, but I like this one,” no one’s interested in that. I have to either love this thing, or better, this thing’s a crime against humanity- … in some way. And it’s very self-reinforcing and everybody sort of feeds into it and-
Lex Fridman
Especially when you’re young. I got to see this kinda interesting thing. So I was at I spent, that’s what we’re talking about, you’re from Pasadena, so I’ve been spending a lot of time in Caltech and working on robots, and we get to see students come in from high school. Undergraduates come in and, like a tour, hang out with the robots. And middle school also. And the interesting thing you see, the younger that they are, the more prevalent this effect, which is all of them are kind of afraid to show that they think a thing is awesome. They’re all… You could just feel they’re checking, “Is it okay?”
Especially when you’re young. I got to see this kinda interesting thing. So I was at I spent, that’s what we’re talking about, you’re from Pasadena, so I’ve been spending a lot of time in Caltech and working on robots, and we get to see students come in from high school. Undergraduates come in and, like a tour, hang out with the robots. And middle school also. And the interesting thing you see, the younger that they are, the more prevalent this effect, which is all of them are kind of afraid to show that they think a thing is awesome. They’re all… You could just feel they’re checking, “Is it okay?”
Lex Fridman
So they’re kinda like the default mode is whatever, this, everything is stupid, this is stupid. You know, ’cause that’s the safe place to be. It’s a real act of vulnerability. I would say it’s an act of courage, especially for a young person, to be like, “Holy shit, that’s awesome.” Like, I’m gonna, if I think this is awesome, I’m gonna be the nerd, I’m gonna take the risk and be made fun of for saying, “I love this,” in that case, it’s, “I love this robot.” So that’s an actual psychological effect that also young people are dealing with, in-person also. So I think, I just wanna say, for young people listening to this, be vulnerable, be courageous and say you love a thing if you love a thing. And do more of that on the internet, I think.
So they’re kinda like the default mode is whatever, this, everything is stupid, this is stupid. You know, ’cause that’s the safe place to be. It’s a real act of vulnerability. I would say it’s an act of courage, especially for a young person, to be like, “Holy shit, that’s awesome.” Like, I’m gonna, if I think this is awesome, I’m gonna be the nerd, I’m gonna take the risk and be made fun of for saying, “I love this,” in that case, it’s, “I love this robot.” So that’s an actual psychological effect that also young people are dealing with, in-person also. So I think, I just wanna say, for young people listening to this, be vulnerable, be courageous and say you love a thing if you love a thing. And do more of that on the internet, I think.
Lex Fridman
I think people make up the internet, people build the internet, and young people, more than anybody else, define the future of the internet. So put more love out there in the world. If you love a video game, if you love Overwatch, say you love it.
I think people make up the internet, people build the internet, and young people, more than anybody else, define the future of the internet. So put more love out there in the world. If you love a video game, if you love Overwatch, say you love it.
Jeff Kaplan
I couldn’t agree more. You know, as somebody who’s taken a lot of heat online, like any game developer, you just get destroyed. Doing what you do, you must get destroyed, you know? And it doesn’t matter, you get 100 compliments, it’s the one, you know, you’re… And you’re supposed to read it and supposed to be fine with it and have it not affect you. It’ll stay with you for years, you know? I have those. And I think of it, like the cheesy, the cheesy way I think about it is like, is there some kind of social Darwinism going on? And my big worry is that there are creators… Like, now being a creator of anything, writer, musician, you know, make online videos, whatever creator means to you, make games.
I couldn’t agree more. You know, as somebody who’s taken a lot of heat online, like any game developer, you just get destroyed. Doing what you do, you must get destroyed, you know? And it doesn’t matter, you get 100 compliments, it’s the one, you know, you’re… And you’re supposed to read it and supposed to be fine with it and have it not affect you. It’ll stay with you for years, you know? I have those. And I think of it, like the cheesy, the cheesy way I think about it is like, is there some kind of social Darwinism going on? And my big worry is that there are creators… Like, now being a creator of anything, writer, musician, you know, make online videos, whatever creator means to you, make games.
Jeff Kaplan
Now part of the skillset is being able to weather like a fire hose of criticism like the world has never seen. And I make up these scenarios in my head of like, would Van Gogh have existed if, you know, Reddit and all these things were out there commenting on… Like, how many people were able to communicate with Beethoven in his lifetime, or in a week? Like, how many influences could comment on his music directly to him? Versus like if I want to insult Brad Pitt right now, I can just go on 10 different devices and do it. And it’s like that level of access is very dangerous, and I worry that there is a whole group of people who’s receding from us that will never see the brilliance, and they’re being shut out by the negativity.
Now part of the skillset is being able to weather like a fire hose of criticism like the world has never seen. And I make up these scenarios in my head of like, would Van Gogh have existed if, you know, Reddit and all these things were out there commenting on… Like, how many people were able to communicate with Beethoven in his lifetime, or in a week? Like, how many influences could comment on his music directly to him? Versus like if I want to insult Brad Pitt right now, I can just go on 10 different devices and do it. And it’s like that level of access is very dangerous, and I worry that there is a whole group of people who’s receding from us that will never see the brilliance, and they’re being shut out by the negativity.
Jeff Kaplan
There- there’s a very real example, was Jay Wilson, who I think is one of the great design minds, who was the game director of Diablo III. And he took so much heat, it just affected him to the point where he essentially retired from making games. Went and, you know, wrote novels. I was very happy for him because, you know, I’m glad he found his place, and I think he’s getting back into making games now. But we lost, we essentially… Like, think how many people loved Diablo III and played the shit out of Diablo III. And Jay is one of the people you have to thank for that. And yet that community basically removed him from making games for like 10, 15 years, and it feels criminal to me.
There- there’s a very real example, was Jay Wilson, who I think is one of the great design minds, who was the game director of Diablo III. And he took so much heat, it just affected him to the point where he essentially retired from making games. Went and, you know, wrote novels. I was very happy for him because, you know, I’m glad he found his place, and I think he’s getting back into making games now. But we lost, we essentially… Like, think how many people loved Diablo III and played the shit out of Diablo III. And Jay is one of the people you have to thank for that. And yet that community basically removed him from making games for like 10, 15 years, and it feels criminal to me.
Lex Fridman
Yeah, absolutely. They… So this is a call to action, again. People out there, support, support, especially young creators, support them. They need it. Like you think negativity has no cost, but it does. You’re robbing the world of some of the great creations. And also, allow creators to suck and to improve. Because that’s what the process of creation is like, is to take risks. To take risks meaning being vulnerable, being cringe.
Yeah, absolutely. They… So this is a call to action, again. People out there, support, support, especially young creators, support them. They need it. Like you think negativity has no cost, but it does. You’re robbing the world of some of the great creations. And also, allow creators to suck and to improve. Because that’s what the process of creation is like, is to take risks. To take risks meaning being vulnerable, being cringe.
Lex Fridman
To doing the thing that like, the embarrassing failure where you’re standing there in a silly clown outfit, on stage, dancing, and nobody’s laughing. And it’s a… Comedians go through this all the time, when… They talk about this all the time, when they bomb, right? They, the act just doesn’t work, and you have to go through that. And you have to, you have to support the creators through that journey. In order to have great things, we need to support those folks. So, after shipping WoW, Wrath of the Lich King, again, many consider it to be one of the great expansions for WoW, you stepped down as WoW’s game director and switched to developing Titan.
To doing the thing that like, the embarrassing failure where you’re standing there in a silly clown outfit, on stage, dancing, and nobody’s laughing. And it’s a… Comedians go through this all the time, when… They talk about this all the time, when they bomb, right? They, the act just doesn’t work, and you have to go through that. And you have to, you have to support the creators through that journey. In order to have great things, we need to support those folks. So, after shipping WoW, Wrath of the Lich King, again, many consider it to be one of the great expansions for WoW, you stepped down as WoW’s game director and switched to developing Titan.
Why Titan failed
Lex Fridman
This epic huge game that promised to be the, sort of the MMO to end all MMOs. I mean, it’s kind of a legendary vision for a game, right? It’s gigantic. With a lot of, like you said, a brilliant team, a team that’s now hardened and knows how to do a great game. But it was canceled after seven years in development. So, tell me, what was the vision of the game and what happened?
This epic huge game that promised to be the, sort of the MMO to end all MMOs. I mean, it’s kind of a legendary vision for a game, right? It’s gigantic. With a lot of, like you said, a brilliant team, a team that’s now hardened and knows how to do a great game. But it was canceled after seven years in development. So, tell me, what was the vision of the game and what happened?
Jeff Kaplan
Sure. So, as we were experiencing success with World of Warcraft, there was this concept in the studio that WoW wasn’t gonna last forever.
Sure. So, as we were experiencing success with World of Warcraft, there was this concept in the studio that WoW wasn’t gonna last forever.
Jeff Kaplan
WoW would be maybe successful for five years, and eventually kind of age out. And the studio would be in real trouble if we didn’t have another massively multiplayer online game sort of waiting in the wings. So starting around, I wanna say 2006, maybe 2005 the talk of starting a team really picked up momentum, and we were working on Burning Crusade. Rob Pardo took the helm to start sort of Titan development. We didn’t even really have a team then. And I remember being embroiled in Burning Crusade and going to Titan meetings, and Rob pulled a group from kind of across the company, and we started talking about what this next MMO could be and when it would get going. And eventually, it started in earnest, like real development, around 2007.
WoW would be maybe successful for five years, and eventually kind of age out. And the studio would be in real trouble if we didn’t have another massively multiplayer online game sort of waiting in the wings. So starting around, I wanna say 2006, maybe 2005 the talk of starting a team really picked up momentum, and we were working on Burning Crusade. Rob Pardo took the helm to start sort of Titan development. We didn’t even really have a team then. And I remember being embroiled in Burning Crusade and going to Titan meetings, and Rob pulled a group from kind of across the company, and we started talking about what this next MMO could be and when it would get going. And eventually, it started in earnest, like real development, around 2007.
Jeff Kaplan
The first team members joined, and it was a real ambitious project, including like building a new engine from scratch. I think maybe the first team member was a guy named John LaFleur, who was just a stellar game programmer, and the engine which ultimately failed for Titan ended up becoming the engine for Overwatch, which is a great success story for him. And the idea behind the game, it was gonna take place in future Earth, and the players played as secret agents. And by day, they all had day jobs, and by night, they went off and did cool secret agent stuff. And the secret agent stuff was very first-person shooter, but over-the-top abilities like you would see in Overwatch, because that’s where they came from. And the by day stuff, we were gonna let you run businesses.
The first team members joined, and it was a real ambitious project, including like building a new engine from scratch. I think maybe the first team member was a guy named John LaFleur, who was just a stellar game programmer, and the engine which ultimately failed for Titan ended up becoming the engine for Overwatch, which is a great success story for him. And the idea behind the game, it was gonna take place in future Earth, and the players played as secret agents. And by day, they all had day jobs, and by night, they went off and did cool secret agent stuff. And the secret agent stuff was very first-person shooter, but over-the-top abilities like you would see in Overwatch, because that’s where they came from. And the by day stuff, we were gonna let you run businesses.
Jeff Kaplan
We took a lot of influence from games like Animal Crossing, Harvest Moon, the Sims. We had a brilliant game designer and game director named Matt Brown, who was the creative director on The Sims. He came over. And so we had this vision that there was gonna be all this like daytime business house stuff. You could build a house. You could live in a neighborhood. And beyond that, there was also a vision on the technical side, game design and technical side, that unlike World of Warcraft, which the modern day term for it is that it’s sharded. Mm-hmm. So meaning people play on different realms or servers.
We took a lot of influence from games like Animal Crossing, Harvest Moon, the Sims. We had a brilliant game designer and game director named Matt Brown, who was the creative director on The Sims. He came over. And so we had this vision that there was gonna be all this like daytime business house stuff. You could build a house. You could live in a neighborhood. And beyond that, there was also a vision on the technical side, game design and technical side, that unlike World of Warcraft, which the modern day term for it is that it’s sharded. Mm-hmm. So meaning people play on different realms or servers.
Jeff Kaplan
In a WoW server, I don’t, I haven’t been on that team in a very long time, but back in the day, you might have 5,000 people on a WoW server before they’d have to spin up another WoW server. The big idea behind Titan is that everybody would play on one server. It was a one server, one world game, and the world was massive. It was gonna take place in future Earth, and we were literally building like, we had what we called Bay City, which was San Francisco. We had, you know, Hollywood, and then we had to build all of California between that, and we also wanted to build like Cairo and London. And there’s this realization of like, how do we connect all of these? The game had driving in it, like full-blown, like GTA-style driving.
In a WoW server, I don’t, I haven’t been on that team in a very long time, but back in the day, you might have 5,000 people on a WoW server before they’d have to spin up another WoW server. The big idea behind Titan is that everybody would play on one server. It was a one server, one world game, and the world was massive. It was gonna take place in future Earth, and we were literally building like, we had what we called Bay City, which was San Francisco. We had, you know, Hollywood, and then we had to build all of California between that, and we also wanted to build like Cairo and London. And there’s this realization of like, how do we connect all of these? The game had driving in it, like full-blown, like GTA-style driving.
Jeff Kaplan
It was such a gargantuan, huge undertaking with a, with a brand new engine, a brand new team, a brand new IP, intellectual property, you know, setting, which we really wrestled over. Like, the amount that the IP just, you know, trying to figure out, like, are there aliens or not aliens, you know? Like, all that sounds kinda dumb and fun, but when you’re building a game, like you, especially world-building, you have to have rules. That’s what makes world-building work, is that like, this exists in this world, and this doesn’t, and you know, why? It’s like, ’cause someone said so, and just the way it needs to be. But that development started in 2007, kind of as ideation, brainstorming, early work.
It was such a gargantuan, huge undertaking with a, with a brand new engine, a brand new team, a brand new IP, intellectual property, you know, setting, which we really wrestled over. Like, the amount that the IP just, you know, trying to figure out, like, are there aliens or not aliens, you know? Like, all that sounds kinda dumb and fun, but when you’re building a game, like you, especially world-building, you have to have rules. That’s what makes world-building work, is that like, this exists in this world, and this doesn’t, and you know, why? It’s like, ’cause someone said so, and just the way it needs to be. But that development started in 2007, kind of as ideation, brainstorming, early work.
Jeff Kaplan
Really got going in late 2007, and then I had to ship Wrath of the Lich King and it was… We had the like, we always did like a champagne toast. I still remember it because it was Election Day. I think it was like Election Day and my birthday, and the day Obama got elected, and then I left the WoW team on that day. It was like memorable in all those ways. And then I joined the Titan team, and that game, we went on, like the fast-forward part of that is we shut it down in 2013.
Really got going in late 2007, and then I had to ship Wrath of the Lich King and it was… We had the like, we always did like a champagne toast. I still remember it because it was Election Day. I think it was like Election Day and my birthday, and the day Obama got elected, and then I left the WoW team on that day. It was like memorable in all those ways. And then I joined the Titan team, and that game, we went on, like the fast-forward part of that is we shut it down in 2013.
Jeff Kaplan
That was one of the most painful development processes that I’ve ever been a part of, and probably, probably deep into 2009, I knew that the game in its current form could never ship and would never exist, and by 2010—like after numerous times trying to convince the powers that be that, like this game is not gonna happen, it’s in trouble. I remember going to Mike Morhaime in 2010, and, like, you’re going to the CEO of… You know, at that time, Blizzard was a big company, and I’m like, “You gotta shut us down. We’re just gonna burn money.”
That was one of the most painful development processes that I’ve ever been a part of, and probably, probably deep into 2009, I knew that the game in its current form could never ship and would never exist, and by 2010—like after numerous times trying to convince the powers that be that, like this game is not gonna happen, it’s in trouble. I remember going to Mike Morhaime in 2010, and, like, you’re going to the CEO of… You know, at that time, Blizzard was a big company, and I’m like, “You gotta shut us down. We’re just gonna burn money.”
Lex Fridman
What was your intuition about why? So like from my understanding, there was a few issues. So one, with such a gigantic world, which by the way, is a beautiful dream, this kind of universe simulator, because I love… Every game you mentioned there is great. I empathize with the dream. I would love to play that game. But one of the issues, as I understand, was it was unclear what, like, the quest flow is. Like what are you supposed to really do in this game? What’s the thing that connects all of the pieces together?
What was your intuition about why? So like from my understanding, there was a few issues. So one, with such a gigantic world, which by the way, is a beautiful dream, this kind of universe simulator, because I love… Every game you mentioned there is great. I empathize with the dream. I would love to play that game. But one of the issues, as I understand, was it was unclear what, like, the quest flow is. Like what are you supposed to really do in this game? What’s the thing that connects all of the pieces together?
Jeff Kaplan
So it was a multifaceted failure for many reasons. Ultimately, the failure of Titan lies with leadership, team leadership, myself included. Like, there’s just no getting around that. And then on top of that, like, a lot of games you can point to as being like an engineering failure, like the, you know, the servers didn’t work— … or like an art failure, like no one responded to the look of the game, or a design failure, like the… it’s just not fun or it’s tuned poorly. We failed on art, engineering, and design, and I’m cautious about calling out art because some of the best art ever made at Blizzard was made for Titan. My criticism isn’t of the art that was created. My criticism is that we never had any art cohesion, so the art looked like it could’ve come from 10 different games.
So it was a multifaceted failure for many reasons. Ultimately, the failure of Titan lies with leadership, team leadership, myself included. Like, there’s just no getting around that. And then on top of that, like, a lot of games you can point to as being like an engineering failure, like the, you know, the servers didn’t work— … or like an art failure, like no one responded to the look of the game, or a design failure, like the… it’s just not fun or it’s tuned poorly. We failed on art, engineering, and design, and I’m cautious about calling out art because some of the best art ever made at Blizzard was made for Titan. My criticism isn’t of the art that was created. My criticism is that we never had any art cohesion, so the art looked like it could’ve come from 10 different games.
Lex Fridman
Mm-hmm. And we should say it cost $83 million across those years. So a large team doing a lot of stuff, but not converging towards a game that could actually ship.
Mm-hmm. And we should say it cost $83 million across those years. So a large team doing a lot of stuff, but not converging towards a game that could actually ship.
Jeff Kaplan
Correct. As, like, a game designer, I use semantics a lot and I like to define my semantics so people know where I’m coming from. Talking about ideas versus vision for a second, ideas are easy. Ideas, you know, I can have 10 in 10 seconds. You know, let’s make a 2D platformer about a mouse, you know, whatever. Like, you can… I want to… A secret agent by day is, you know, doing all this cool shooting stuff, by night is running a flower shop. You know, ideas are just infinite. At least on creative teams, you know, you have no shortage of ideas. What I call vision is the ability to not only take a great idea, but shepherd it into existence, and you’re doing that through inspiration first and foremost.
Correct. As, like, a game designer, I use semantics a lot and I like to define my semantics so people know where I’m coming from. Talking about ideas versus vision for a second, ideas are easy. Ideas, you know, I can have 10 in 10 seconds. You know, let’s make a 2D platformer about a mouse, you know, whatever. Like, you can… I want to… A secret agent by day is, you know, doing all this cool shooting stuff, by night is running a flower shop. You know, ideas are just infinite. At least on creative teams, you know, you have no shortage of ideas. What I call vision is the ability to not only take a great idea, but shepherd it into existence, and you’re doing that through inspiration first and foremost.
Jeff Kaplan
If you need a team to make it, you need a team to believe in the vision of the idea. And then there also has to be a technological plan for the idea. There has to be a design plan. There has to be an art style for the plan. There has to be a pragmatic production reality to the plan. And Titan kind of was like that was the hubris of Blizzard in that era at its height of, you know, we were over being hurt about, you know, World of Warcraft. I don’t know if people are gonna like it. And we were now in the era of, like, we made World of Warcraft. We can do no wrong. This next thing is gonna be the best ever. And there was also a lot of what I call anticipatory hiring-
If you need a team to make it, you need a team to believe in the vision of the idea. And then there also has to be a technological plan for the idea. There has to be a design plan. There has to be an art style for the plan. There has to be a pragmatic production reality to the plan. And Titan kind of was like that was the hubris of Blizzard in that era at its height of, you know, we were over being hurt about, you know, World of Warcraft. I don’t know if people are gonna like it. And we were now in the era of, like, we made World of Warcraft. We can do no wrong. This next thing is gonna be the best ever. And there was also a lot of what I call anticipatory hiring-
Jeff Kaplan
… or, like, there’s opportunity hiring and then there’s also anticipatory hiring. I have the exact opposite hiring philosophy. I won’t hire anybody on any team until, like, we’re feeling like we gotta work overtime or, like, we might not ship if we don’t get, you know, somebody else in here. And Titan kinda had that hubris of like, well, we’re gonna build a really big world. We don’t know the story of the world yet. We don’t really have it mapped out what it should be like. We don’t have the art style really defined. We don’t know technically how we’re gonna make the art or what the constraints of it are, but we know we’re gonna build a really big world, so let’s just start hiring environmental artists.
… or, like, there’s opportunity hiring and then there’s also anticipatory hiring. I have the exact opposite hiring philosophy. I won’t hire anybody on any team until, like, we’re feeling like we gotta work overtime or, like, we might not ship if we don’t get, you know, somebody else in here. And Titan kinda had that hubris of like, well, we’re gonna build a really big world. We don’t know the story of the world yet. We don’t really have it mapped out what it should be like. We don’t have the art style really defined. We don’t know technically how we’re gonna make the art or what the constraints of it are, but we know we’re gonna build a really big world, so let’s just start hiring environmental artists.
Jeff Kaplan
And, like, in one year, we would hire, like, 70 environmental artists from all over the world. You know, we’re getting visas and, like, the top tier talent ’cause at the height of World of Warcraft and nobody knew the team that they were coming on. It was Blizzard’s next MMO top secret and they, you know, their first day at work, like some, you know, poor guy from Belgium just shows up and he’s at his first day at work and he’s like, “Oh, are we making World of StarCraft? Is that…” And they’re like, “No, dude. Let me show you it.” And he’s like, “What is this game?” You know? We were in that world, and we hired way too many people.
And, like, in one year, we would hire, like, 70 environmental artists from all over the world. You know, we’re getting visas and, like, the top tier talent ’cause at the height of World of Warcraft and nobody knew the team that they were coming on. It was Blizzard’s next MMO top secret and they, you know, their first day at work, like some, you know, poor guy from Belgium just shows up and he’s at his first day at work and he’s like, “Oh, are we making World of StarCraft? Is that…” And they’re like, “No, dude. Let me show you it.” And he’s like, “What is this game?” You know? We were in that world, and we hired way too many people.
Jeff Kaplan
The right way to incubate a video game is you have the smallest group possible and you try to get the idea across with whatever technology you can get your hands on, using other engines, using art from whatever. You prove out that idea, and once you know what you’re doing, then you expand the team. You know the cliche of “idle hands is the devil’s work,” or whatever.
The right way to incubate a video game is you have the smallest group possible and you try to get the idea across with whatever technology you can get your hands on, using other engines, using art from whatever. You prove out that idea, and once you know what you’re doing, then you expand the team. You know the cliche of “idle hands is the devil’s work,” or whatever.
Lex Fridman
Yeah, yeah.
Yeah, yeah.
Jeff Kaplan
You have these, like, brilliant team, huge, and we don’t have a road map for what we’re making or how we’re gonna make it. And now you’re having to deal with all these people. Like, they’re coming into your office, you know, you’re trying to figure out what is the quest flow, how do I design the quest system for Titan, how can we prototype it? And we’re like, “Oh, this prop artist over here is running out of stuff to do. What props should he make? Should he work on Chinatown or the Hollywood set?” And you’re just making up busy work. The engine didn’t work.
You have these, like, brilliant team, huge, and we don’t have a road map for what we’re making or how we’re gonna make it. And now you’re having to deal with all these people. Like, they’re coming into your office, you know, you’re trying to figure out what is the quest flow, how do I design the quest system for Titan, how can we prototype it? And we’re like, “Oh, this prop artist over here is running out of stuff to do. What props should he make? Should he work on Chinatown or the Hollywood set?” And you’re just making up busy work. The engine didn’t work.
Jeff Kaplan
When we would run play tests on Titan, we would have to tell the team, “Stop checking in because it slows us down.” We had this really great technical artist, a guy named Dylan Jones, and he was on Titan with us, and I remember in, like, the last days, we asked him, because he was a very active user—Titan editor was called Titan Edit or TED which is, to this day, TED is the proprietary tool for Overwatch, since Overwatch came from the Titan engine—
When we would run play tests on Titan, we would have to tell the team, “Stop checking in because it slows us down.” We had this really great technical artist, a guy named Dylan Jones, and he was on Titan with us, and I remember in, like, the last days, we asked him, because he was a very active user—Titan editor was called Titan Edit or TED which is, to this day, TED is the proprietary tool for Overwatch, since Overwatch came from the Titan engine—
Jeff Kaplan
… which was Tank. And we said to Dylan, “I want you to log your uptime in the editor, in TED.” And in a 40-hour week, he was only able to work for 20 hours. And you can imagine, you’re building a team of the best and the—like, the best in the industry, and they can’t work. So not only are you just burning cash faster than anybody on the planet, it’s also, like, imagine having fighter pilots, but we don’t let them fly. Like, the creative frustration and the way that that manifested itself, and how demoralized the team got, it was a disaster.
… which was Tank. And we said to Dylan, “I want you to log your uptime in the editor, in TED.” And in a 40-hour week, he was only able to work for 20 hours. And you can imagine, you’re building a team of the best and the—like, the best in the industry, and they can’t work. So not only are you just burning cash faster than anybody on the planet, it’s also, like, imagine having fighter pilots, but we don’t let them fly. Like, the creative frustration and the way that that manifested itself, and how demoralized the team got, it was a disaster.
Lex Fridman
And so many elements of that were done completely differently for Overwatch, which turned out to be this incredibly masterful execution on a short timescale with a small team with a clear vision. I read that sort of if you—if you were to compare Overwatch and Titan, sort of the defining characteristic for the Titan team, they said yes to everything, and the Overwatch team said no to everything. Meaning focus, like deep, deep focus on the execution of a very clear vision. And maybe that’s the process of designing games, like you said, is, you know on a team that’s full of incredible ideas because it’s creative minds, it’s constantly saying no. It’s a really painful process, but perhaps it—it is the responsibility of leadership to just keep saying no. Which sucks.
And so many elements of that were done completely differently for Overwatch, which turned out to be this incredibly masterful execution on a short timescale with a small team with a clear vision. I read that sort of if you—if you were to compare Overwatch and Titan, sort of the defining characteristic for the Titan team, they said yes to everything, and the Overwatch team said no to everything. Meaning focus, like deep, deep focus on the execution of a very clear vision. And maybe that’s the process of designing games, like you said, is, you know on a team that’s full of incredible ideas because it’s creative minds, it’s constantly saying no. It’s a really painful process, but perhaps it—it is the responsibility of leadership to just keep saying no. Which sucks.
Lex Fridman
I guess it sucks to be a leader on a team in that sense, because you’re constantly saying no.
I guess it sucks to be a leader on a team in that sense, because you’re constantly saying no.
Jeff Kaplan
Being a creative leader, you’re in two modes. You’re pushing or you’re pulling, and whatever mode you’re in is the exact opposite of the team. When they’re not thinking outside the box enough or, like, elevating the vision enough, that’s when you’re pushing them. Like, “Come on guys,” you know, “don’t worry about the schedule. We got—” you know, “capture hearts and minds, inspire people.” And when they’re going a little crazy and they… Endless source of great ideas and really fun development, that’s when you gotta pull and say, “Guys, we need to ship this. The best feature we can add for the player is shipping.” That was a common phrase that we had.
Being a creative leader, you’re in two modes. You’re pushing or you’re pulling, and whatever mode you’re in is the exact opposite of the team. When they’re not thinking outside the box enough or, like, elevating the vision enough, that’s when you’re pushing them. Like, “Come on guys,” you know, “don’t worry about the schedule. We got—” you know, “capture hearts and minds, inspire people.” And when they’re going a little crazy and they… Endless source of great ideas and really fun development, that’s when you gotta pull and say, “Guys, we need to ship this. The best feature we can add for the player is shipping.” That was a common phrase that we had.
Overwatch in six weeks
Lex Fridman
So when Titan was canceled, I mean that must’ve been a gigantic heartbreak for everybody. And there was this moment when the plan was for the Titan team to be disbanded and moved elsewhere, but you fought for keeping some part of the Titan team, the core of the team together, and Mike Morhaime gave you six weeks to come up with a pitch for a new game. And you’ve talked about this process, and you’ve mentioned that there were three possible ideas, directions you were thinking about. A StarCraft MMO, maybe an MMO in a new IP called Crossworlds, and then the third idea was Overwatch. Can you take me through those six weeks?
So when Titan was canceled, I mean that must’ve been a gigantic heartbreak for everybody. And there was this moment when the plan was for the Titan team to be disbanded and moved elsewhere, but you fought for keeping some part of the Titan team, the core of the team together, and Mike Morhaime gave you six weeks to come up with a pitch for a new game. And you’ve talked about this process, and you’ve mentioned that there were three possible ideas, directions you were thinking about. A StarCraft MMO, maybe an MMO in a new IP called Crossworlds, and then the third idea was Overwatch. Can you take me through those six weeks?
Jeff Kaplan
Yeah, the six weeks, it’s… It was supposed to be the greatest time ever if you think about it. Because you’re a game developer at Blizzard, and you get to come up with a new idea. So that sounds awesome, like, to everybody at Blizzard, to all game developers, it sounds great. But we were probably the most demoralized we’d ever been in our careers. At least I was, you know? I didn’t know if I was gonna be fired. I didn’t know if that was the end of my career at that point. And so it was like a really serious, kind of dire environment that this was happening in. And we were given two criteria that we had to hit for these pitches. The first one was that we had to ship within two years. And that is a very ambitious timeframe for any game.
Yeah, the six weeks, it’s… It was supposed to be the greatest time ever if you think about it. Because you’re a game developer at Blizzard, and you get to come up with a new idea. So that sounds awesome, like, to everybody at Blizzard, to all game developers, it sounds great. But we were probably the most demoralized we’d ever been in our careers. At least I was, you know? I didn’t know if I was gonna be fired. I didn’t know if that was the end of my career at that point. And so it was like a really serious, kind of dire environment that this was happening in. And we were given two criteria that we had to hit for these pitches. The first one was that we had to ship within two years. And that is a very ambitious timeframe for any game.
Lex Fridman
Yeah, crazy. That’s crazy.
Yeah, crazy. That’s crazy.
Jeff Kaplan
But for a Blizzard game, it’s kind of insane. And then the second… Okay, the second is even more ambitious and crazy, was whatever we made, whatever we pitched had to have the potential to have World of Warcraft-like revenue.
But for a Blizzard game, it’s kind of insane. And then the second… Okay, the second is even more ambitious and crazy, was whatever we made, whatever we pitched had to have the potential to have World of Warcraft-like revenue.
Lex Fridman
Yeah. Right.
Yeah. Right.
Jeff Kaplan
And to date, at that point, there was one game that had World of Warcraft-like revenue, which was World of Warcraft, so… Immediately, I just threw out the revenue thing ’cause it’s all fucking Monopoly money to me. Like, this game money is… It’s insane, and I just don’t think about it. That’s someone else’s problem. But I did want to be as realistic as I could about the schedule part of it. So most of our team, the Titan team, was 140-some people. Most of that team got moved to go work on Heroes of the Storm, the D3 expansion, World of Warcraft, Hearthstone. So immediately, a large number of the team was gone. Then we had a bunch of, like, what we called temp loans-
And to date, at that point, there was one game that had World of Warcraft-like revenue, which was World of Warcraft, so… Immediately, I just threw out the revenue thing ’cause it’s all fucking Monopoly money to me. Like, this game money is… It’s insane, and I just don’t think about it. That’s someone else’s problem. But I did want to be as realistic as I could about the schedule part of it. So most of our team, the Titan team, was 140-some people. Most of that team got moved to go work on Heroes of the Storm, the D3 expansion, World of Warcraft, Hearthstone. So immediately, a large number of the team was gone. Then we had a bunch of, like, what we called temp loans-
Jeff Kaplan
… people that someday were gonna come back to us, but we loaned off for, like, six-month tour of duty. And then there was a very small team. There was a group of engineers that was mothballing Titan, so it exists somewhere at Blizzard at that point. And they were also deconstructing the engine because they knew it didn’t work anymore, and to make a new game, it had to be way reconsidered to sort of what it is today. And then there was a very small creative group that was supposed to come up with these three pitches and given six weeks. And we just sort of arbitrarily decided, like, let’s spend two weeks on each pitch. The ground rules that I sort of led with is you have to be all in for the two weeks on the pitch. So if we’re…
… people that someday were gonna come back to us, but we loaned off for, like, six-month tour of duty. And then there was a very small team. There was a group of engineers that was mothballing Titan, so it exists somewhere at Blizzard at that point. And they were also deconstructing the engine because they knew it didn’t work anymore, and to make a new game, it had to be way reconsidered to sort of what it is today. And then there was a very small creative group that was supposed to come up with these three pitches and given six weeks. And we just sort of arbitrarily decided, like, let’s spend two weeks on each pitch. The ground rules that I sort of led with is you have to be all in for the two weeks on the pitch. So if we’re…
Jeff Kaplan
You know, pitch one was a StarCraft MMO, and we have to live and breathe and want it more than anything. And I kind of warned everybody. I said, “At the end of this two weeks, you’re going to think this is the only game idea, and you’re not going to be invested in the next, but we’re going to throw it out as soon as we finish it and do the next one.” And the StarCraft MMO, I actually really loved that pitch. It was called StarCraft Frontiers. And the concept was, like, less of you’re playing, like, space marine. Like, it was less armies. StarCraft the RTS is always about the three races and the giant armies.
You know, pitch one was a StarCraft MMO, and we have to live and breathe and want it more than anything. And I kind of warned everybody. I said, “At the end of this two weeks, you’re going to think this is the only game idea, and you’re not going to be invested in the next, but we’re going to throw it out as soon as we finish it and do the next one.” And the StarCraft MMO, I actually really loved that pitch. It was called StarCraft Frontiers. And the concept was, like, less of you’re playing, like, space marine. Like, it was less armies. StarCraft the RTS is always about the three races and the giant armies.
Jeff Kaplan
And kind of what made WoW wow and separate from the Warcraft RTS series was that instead of being, like, a footman in the army in World of Warcraft, you were like a lone adventurer, you know, make your mark on the world. So we had this idea, it was this old Chris Metzen drawing of a space prospector. And I love that idea that, like, somewhere out in, like, where all the giant StarCraft battles were happening, you know, thousands of Zerg and Protoss and Terran, there’s, like, this, like, lone prospector on some planet, like, going through, like, a mysterious dungeon- … you know, looking for minerals but finding monsters. Like, it was that kind of spirit of-
And kind of what made WoW wow and separate from the Warcraft RTS series was that instead of being, like, a footman in the army in World of Warcraft, you were like a lone adventurer, you know, make your mark on the world. So we had this idea, it was this old Chris Metzen drawing of a space prospector. And I love that idea that, like, somewhere out in, like, where all the giant StarCraft battles were happening, you know, thousands of Zerg and Protoss and Terran, there’s, like, this, like, lone prospector on some planet, like, going through, like, a mysterious dungeon- … you know, looking for minerals but finding monsters. Like, it was that kind of spirit of-
Lex Fridman
That’s awesome.
That’s awesome.
Jeff Kaplan
… more on the ground level.
… more on the ground level.
Lex Fridman
I didn’t even think about that because my intuition with a StarCraft MMO would be the soldier as part of the army, right? The prospector. That’s such a beautiful vision. Yeah.
I didn’t even think about that because my intuition with a StarCraft MMO would be the soldier as part of the army, right? The prospector. That’s such a beautiful vision. Yeah.
Jeff Kaplan
Yeah, I-
Yeah, I-
Lex Fridman
Looking for the resources and on the way finding the monsters.
Looking for the resources and on the way finding the monsters.
Jeff Kaplan
You want to be on the ground f- Like, what’s it like on the ground floor? And I don’t want to be a minion in a giant army. I want to be Indiana Jones in space, you know?
You want to be on the ground f- Like, what’s it like on the ground floor? And I don’t want to be a minion in a giant army. I want to be Indiana Jones in space, you know?
Lex Fridman
Nice.
Nice.
Jeff Kaplan
So then there was this Metzen picture of the prospector, and then two of the most amazing artists, Arnold Tsang and Peter Lee. Arnold’s the great character artist. Peter Lee’s the great environment artist. They did this concept art for Frontiers that was Metzen’s space prospector. He’s smoking a cigar-… and he’s got his foot on a Hydralisk skull.
So then there was this Metzen picture of the prospector, and then two of the most amazing artists, Arnold Tsang and Peter Lee. Arnold’s the great character artist. Peter Lee’s the great environment artist. They did this concept art for Frontiers that was Metzen’s space prospector. He’s smoking a cigar-… and he’s got his foot on a Hydralisk skull.
Lex Fridman
Nice.
Nice.
Jeff Kaplan
And then there’s, like, a Medivac in the background, and they’re on this, like, big alien planet. And, like, that picture, you just wanted to like, “Here’s my money. I’ll pre-order now. Like, sign me up for that game.” That picture ended up being McCree from Overwatch. We redid it.
And then there’s, like, a Medivac in the background, and they’re on this, like, big alien planet. And, like, that picture, you just wanted to like, “Here’s my money. I’ll pre-order now. Like, sign me up for that game.” That picture ended up being McCree from Overwatch. We redid it.
Lex Fridman
Nice.
Nice.
Jeff Kaplan
But, but yeah, that’s, that was where McCree actually came from. So that was the StarCraft Frontiers idea. We kind of, we, we went all in on the design. We had a world design. We had class design, like how, how the classes would work, what progression might look like. And you also have to think when you’re trying to design an MMO, like, what could expansions and live content be like? And we put together a really good pitch. We all knew there’s no way you can make this game. Like, this, even though it was more focused than Titan, it’s five years on Blizzard’s best day with nothing going wrong, in a perfect scenario, five years to make that game probably with, you know, 150 to 200 people.
But, but yeah, that’s, that was where McCree actually came from. So that was the StarCraft Frontiers idea. We kind of, we, we went all in on the design. We had a world design. We had class design, like how, how the classes would work, what progression might look like. And you also have to think when you’re trying to design an MMO, like, what could expansions and live content be like? And we put together a really good pitch. We all knew there’s no way you can make this game. Like, this, even though it was more focused than Titan, it’s five years on Blizzard’s best day with nothing going wrong, in a perfect scenario, five years to make that game probably with, you know, 150 to 200 people.
Jeff Kaplan
Like, these 40 people are not making that game in two years. So as much as I… Like, again, that was an idea, not a vision, ’cause it lacked, it lacked the path to reality, you know? There-
Like, these 40 people are not making that game in two years. So as much as I… Like, again, that was an idea, not a vision, ’cause it lacked, it lacked the path to reality, you know? There-
Lex Fridman
‘Cause that’s a legit large-scale MMO in a, in a world that you haven’t quite developed in the way that an MMO needs that was really crafted for the arts or the real-time strategy formulation of StarCraft. And it’s in space. It’s-
‘Cause that’s a legit large-scale MMO in a, in a world that you haven’t quite developed in the way that an MMO needs that was really crafted for the arts or the real-time strategy formulation of StarCraft. And it’s in space. It’s-
Jeff Kaplan
Yeah.
Yeah.
Lex Fridman
It’s… It would, it would take… I mean, it would be incredible, but it would be a five-year and realistically even more.
It’s… It would, it would take… I mean, it would be incredible, but it would be a five-year and realistically even more.
Jeff Kaplan
Like, an endless thing that you’d spin on on that team. You’re making the StarCraft game. How do you get from planet to planet?
Like, an endless thing that you’d spin on on that team. You’re making the StarCraft game. How do you get from planet to planet?
Lex Fridman
Yeah.
Yeah.
Jeff Kaplan
Is it a cut scene? No one’s going to want a cut scene, but we should probably make it a cut scene because that’s easy. But well, we gotta have space flight. That… You’re adding, like, three years just by saying, “We gotta have space flight.”
Is it a cut scene? No one’s going to want a cut scene, but we should probably make it a cut scene because that’s easy. But well, we gotta have space flight. That… You’re adding, like, three years just by saying, “We gotta have space flight.”
Lex Fridman
You are. Yeah.
You are. Yeah.
Jeff Kaplan
And then how do you make a space game without space flight? We’ve all played them. We know, we know those games, so.
And then how do you make a space game without space flight? We’ve all played them. We know, we know those games, so.
Lex Fridman
So are you essentially, when you’re brainstorming like that, and by the way, such an incredible thing, for two weeks, you’re just really falling in love with the game altogether and trying to figure out if it’s actually possible. So if you’re developing that, are you just constantly trying to say, like, “What is the simplest possible thing we can do that’s a complete world?” Like, are you constantly trying to simplify or you’re allowing yourself to go big?
So are you essentially, when you’re brainstorming like that, and by the way, such an incredible thing, for two weeks, you’re just really falling in love with the game altogether and trying to figure out if it’s actually possible. So if you’re developing that, are you just constantly trying to say, like, “What is the simplest possible thing we can do that’s a complete world?” Like, are you constantly trying to simplify or you’re allowing yourself to go big?
Jeff Kaplan
So when you’re brainstorming and you’re with the team and you’re the creative leader, it’s, “Guys, what’s fucking amazing?” What’s big? What do players need? There’s a Blizzard design value called “what is the fantasy?” What is the fantasy? You want to be in space. You want to be in the StarCraft universe, and then your job as the game director, and if you have a great creative director, art director, tech director, the director should be scoping it back into reality. The mistake I see on a lot of game teams is scope becomes a production problem. You give it to the project managers or the executives or the producers to say, “No, there’s not enough time.” Or, “You guys should hire more,” ’cause-
So when you’re brainstorming and you’re with the team and you’re the creative leader, it’s, “Guys, what’s fucking amazing?” What’s big? What do players need? There’s a Blizzard design value called “what is the fantasy?” What is the fantasy? You want to be in space. You want to be in the StarCraft universe, and then your job as the game director, and if you have a great creative director, art director, tech director, the director should be scoping it back into reality. The mistake I see on a lot of game teams is scope becomes a production problem. You give it to the project managers or the executives or the producers to say, “No, there’s not enough time.” Or, “You guys should hire more,” ’cause-
Lex Fridman
Right.
Right.
Jeff Kaplan
Like, what do executives, what do those types have at their disposal that they can hit you with meetings in Outlook and tell you that you can hire more people? That’s not really how you get the game made.
Like, what do executives, what do those types have at their disposal that they can hit you with meetings in Outlook and tell you that you can hire more people? That’s not really how you get the game made.
Lex Fridman
That’s why they get paid the big bucks.
That’s why they get paid the big bucks.
Jeff Kaplan
The scoping, your best-case scenario is when your tech director, art director, and game director are doing the scoping. Because then you know, like, this part we gotta spend big bucks on. There’s no getting around it. This part we can cheat. If you have a giant team and one guy’s job is just to make props, you know, crates and chairs, that guy’s going to make the… You know, that’s a AAA awesome developer who’s going to put his heart and soul into it. If you let him, he’ll take, you know, six weeks to make a crate. You have to have that moment where you’re like, “I kind of need 200 crates. So just spend, like, a couple hours on that one.” And that’s a hard thing to say to somebody.
The scoping, your best-case scenario is when your tech director, art director, and game director are doing the scoping. Because then you know, like, this part we gotta spend big bucks on. There’s no getting around it. This part we can cheat. If you have a giant team and one guy’s job is just to make props, you know, crates and chairs, that guy’s going to make the… You know, that’s a AAA awesome developer who’s going to put his heart and soul into it. If you let him, he’ll take, you know, six weeks to make a crate. You have to have that moment where you’re like, “I kind of need 200 crates. So just spend, like, a couple hours on that one.” And that’s a hard thing to say to somebody.
Lex Fridman
You’re doing this kind of scope carving while also talking about “what is the fantasy.” So you’re, there’s a tension there that you’re constantly dancing with. So you’re, you’re allowing yourself to think big, but then scoping it down, and doing that, what, on a scale of days in this case, like?
You’re doing this kind of scope carving while also talking about “what is the fantasy.” So you’re, there’s a tension there that you’re constantly dancing with. So you’re, you’re allowing yourself to think big, but then scoping it down, and doing that, what, on a scale of days in this case, like?
Jeff Kaplan
Yeah. We had two weeks, so, and I don’t think we were… I was working on weekends, but we weren’t getting the group together. So it’s, you know, like 10 working days.
Yeah. We had two weeks, so, and I don’t think we were… I was working on weekends, but we weren’t getting the group together. So it’s, you know, like 10 working days.
Lex Fridman
And then you, like, shut it off and go to idea number two?
And then you, like, shut it off and go to idea number two?
Jeff Kaplan
Yeah. Idea number two was CrossWorlds. That was a Metzen vision for a universe, and, like, I’m glad Metzen’s back at Blizzard, and I hope they make this game someday. The way Chris described it was there’s a planet on the edge of the universe that’s like the Mos Eisley space port with all these, you know, freakish aliens and people from all walks of life-
Yeah. Idea number two was CrossWorlds. That was a Metzen vision for a universe, and, like, I’m glad Metzen’s back at Blizzard, and I hope they make this game someday. The way Chris described it was there’s a planet on the edge of the universe that’s like the Mos Eisley space port with all these, you know, freakish aliens and people from all walks of life-
Lex Fridman
Nice.
Nice.
Jeff Kaplan
… and it’s kind of seedy and criminal. And there’s traders and smugglers and diplomats and… But this one planet is sort of the planet that they’ve agreed to like meet on, and this is like the neutral place, and then the game was going to take place on that planet, so-
… and it’s kind of seedy and criminal. And there’s traders and smugglers and diplomats and… But this one planet is sort of the planet that they’ve agreed to like meet on, and this is like the neutral place, and then the game was going to take place on that planet, so-
Lex Fridman
This is awesome.
This is awesome.
Jeff Kaplan
Yeah. So that was more of like a world IP driven one that was really inspired by Chris.
Yeah. So that was more of like a world IP driven one that was really inspired by Chris.
Lex Fridman
And that allows you to play with different characters, different… I like that, I like that idea a lot, because it’s the meeting place of different worlds, and then you can allow your imagination to drive what the worlds from which they came from are like. So you don’t have to design those worlds.
And that allows you to play with different characters, different… I like that, I like that idea a lot, because it’s the meeting place of different worlds, and then you can allow your imagination to drive what the worlds from which they came from are like. So you don’t have to design those worlds.
Jeff Kaplan
No, you don’t have to design them, but then they’re yours. Like, if the players really are reacting to, like, the Green People planet- … or whatever, and someday you’re like, “Hey, what expansion should we make?” “I don’t know. Green People planet.”
No, you don’t have to design them, but then they’re yours. Like, if the players really are reacting to, like, the Green People planet- … or whatever, and someday you’re like, “Hey, what expansion should we make?” “I don’t know. Green People planet.”
Lex Fridman
Green People, yeah.
Green People, yeah.
Jeff Kaplan
Like, “Let’s do it.”
Like, “Let’s do it.”
Lex Fridman
I like it.
I like it.
Jeff Kaplan
So it was actually that, it was CrossWorlds. We were working on CrossWorlds, and like the StarCraft Frontiers, you know, for Frontiers, we were having the class meetings, you know, how class progression work, like, the game designery stuff. And on CrossWorlds, we were having a class meeting of, like a big decision in, like, RPG type games is always: are you doing, like, skill based or class based?
So it was actually that, it was CrossWorlds. We were working on CrossWorlds, and like the StarCraft Frontiers, you know, for Frontiers, we were having the class meetings, you know, how class progression work, like, the game designery stuff. And on CrossWorlds, we were having a class meeting of, like a big decision in, like, RPG type games is always: are you doing, like, skill based or class based?
Jeff Kaplan
And it’s usually some combination of those, but class based, you’re like choosing, “I’m going to be a warrior, therefore I use sword and shield, and I do these things.” Where more of a skill base is everybody’s kind of an avatar, and then the skills that you pick define, so I might take that I know how to use swords. So you’re kind of making those decisions, and with all things game design, there’s no right or wrong. It’s all trade-offs. So the trade-off decision we were making is like, “Oh, I think we want to be class based with this CrossWorlds thing,” and we were in a design meeting and one of my favorite designers of all time is a guy named Geoff Goodman. He was one of the original WoW encounter designers; he designed like Onyxia and all the big raid bosses.
And it’s usually some combination of those, but class based, you’re like choosing, “I’m going to be a warrior, therefore I use sword and shield, and I do these things.” Where more of a skill base is everybody’s kind of an avatar, and then the skills that you pick define, so I might take that I know how to use swords. So you’re kind of making those decisions, and with all things game design, there’s no right or wrong. It’s all trade-offs. So the trade-off decision we were making is like, “Oh, I think we want to be class based with this CrossWorlds thing,” and we were in a design meeting and one of my favorite designers of all time is a guy named Geoff Goodman. He was one of the original WoW encounter designers; he designed like Onyxia and all the big raid bosses.
Jeff Kaplan
Like, if someone has a favorite raid boss, Geoff probably designed it. And he just kind of off the cuff said in this meeting, he said, “I wish instead of making, like, six classes, I wish we could make 50 classes. And I wish instead of having, like, you know, 100 abilities on the classes, the 50 classes all just had, like, one or two things that was really interesting about them.” And then the class meeting ended. Like, we designed our six classes in that meeting, and then the meeting ended. And I was back at my desk, and it just stuck with me what Geoff had said about the way he wished he could design the classes. And then I also had… We had this directory of all the amazing Titan art.
Like, if someone has a favorite raid boss, Geoff probably designed it. And he just kind of off the cuff said in this meeting, he said, “I wish instead of making, like, six classes, I wish we could make 50 classes. And I wish instead of having, like, you know, 100 abilities on the classes, the 50 classes all just had, like, one or two things that was really interesting about them.” And then the class meeting ended. Like, we designed our six classes in that meeting, and then the meeting ended. And I was back at my desk, and it just stuck with me what Geoff had said about the way he wished he could design the classes. And then I also had… We had this directory of all the amazing Titan art.
Jeff Kaplan
And I started pulling up Arnold Tsang’s characters. Arnold’s vision and his art is second to none. And I started taking some of the old Titan characters that we had designed. We had a class called the Jumper, and the Jumper could, like, teleport forward and rewind time and come back. And the Jumper used dual-wield pistols, which was, at the time, designed after my dual G18s from Modern Warfare 2. It was my favorite loadout. I was just cribbing Infinity Ward. That’s where Tracer’s guns came from.
And I started pulling up Arnold Tsang’s characters. Arnold’s vision and his art is second to none. And I started taking some of the old Titan characters that we had designed. We had a class called the Jumper, and the Jumper could, like, teleport forward and rewind time and come back. And the Jumper used dual-wield pistols, which was, at the time, designed after my dual G18s from Modern Warfare 2. It was my favorite loadout. I was just cribbing Infinity Ward. That’s where Tracer’s guns came from.
Jeff Kaplan
And we had all these, like, different guns, like, some that bloomed and some that, you know, had this, like, really crazy recoil, and we had other types of guns. And I took every version of, like, the Titan Jumper, and I just distilled it into what I thought was the best version of the Jumper, which was, you know, the dual-wield pistols, the blink, the recall, and time bomb. And then I took Arnold Tsang art, and I went, you know, to Arnold, and I’m like, “What if this wasn’t, like, a class? You know, who is this as a person, not a class?” And Arnold, “What if she’s British, and her name’s Tracer?” And, like, that was the origin of Overwatch.
And we had all these, like, different guns, like, some that bloomed and some that, you know, had this, like, really crazy recoil, and we had other types of guns. And I took every version of, like, the Titan Jumper, and I just distilled it into what I thought was the best version of the Jumper, which was, you know, the dual-wield pistols, the blink, the recall, and time bomb. And then I took Arnold Tsang art, and I went, you know, to Arnold, and I’m like, “What if this wasn’t, like, a class? You know, who is this as a person, not a class?” And Arnold, “What if she’s British, and her name’s Tracer?” And, like, that was the origin of Overwatch.
Jeff Kaplan
And some of the pragmatic part of that was I knew that Geoff Goodman was gonna be on this team, and I knew that Arnold Tsang was gonna be on this team. And it’s a play to your strengths moment. Like, what could we make in two years with the talent we have, and what is realistic? Like, what could we realistically make? And so then I just sat there, and I sort of I went through a bunch of Titan classes with a guy named The Gunjack, who was… became Reaper. We had… Actually, the Ranger got split out and became 76 and became Bastion, of all things.
And some of the pragmatic part of that was I knew that Geoff Goodman was gonna be on this team, and I knew that Arnold Tsang was gonna be on this team. And it’s a play to your strengths moment. Like, what could we make in two years with the talent we have, and what is realistic? Like, what could we realistically make? And so then I just sat there, and I sort of I went through a bunch of Titan classes with a guy named The Gunjack, who was… became Reaper. We had… Actually, the Ranger got split out and became 76 and became Bastion, of all things.
Lex Fridman
You’re describing the game of Overwatch where exactly that vision from that meeting- … came to life for you. As opposed to having a small number of classes with a large number of skills, you have a large number of heroes with each their distinct look, distinct set of skills.
You’re describing the game of Overwatch where exactly that vision from that meeting- … came to life for you. As opposed to having a small number of classes with a large number of skills, you have a large number of heroes with each their distinct look, distinct set of skills.
Jeff Kaplan
Yeah, and the personality was a big part of it, like capturing… This isn’t some generic, the Jumper. It’s this person, Lena Oxton. You know? And she has a life, and we’re gonna, you know, make you interested in her.
Yeah, and the personality was a big part of it, like capturing… This isn’t some generic, the Jumper. It’s this person, Lena Oxton. You know? And she has a life, and we’re gonna, you know, make you interested in her.
Lex Fridman
Yeah, there’s, like, a deep backstory. And that’s also what’s interesting about Overwatch, is that backstory is not, like, revealed in a direct way. It’s, it sort of, like, seeps in indirectly throughout the game. So, the backstory is implied almost. And it’s told not directly. So, there’s a lot of ideas like this. And so you’re… This is the thing that the team converged to.
Yeah, there’s, like, a deep backstory. And that’s also what’s interesting about Overwatch, is that backstory is not, like, revealed in a direct way. It’s, it sort of, like, seeps in indirectly throughout the game. So, the backstory is implied almost. And it’s told not directly. So, there’s a lot of ideas like this. And so you’re… This is the thing that the team converged to.
Jeff Kaplan
Yeah. Well, and it was funny because, like, we’re having these CrossWorlds. Like, people are, you know, writing design docs and doing concept art for CrossWorlds. And, you know, we’d have some brainstorm meetings every day, and I put together… It was a seven-page deck, Overwatch deck. And it was called Monetized Shooter at the time. And it just said, “Monetized Shooter.” And then the first slide was League of Legends plus Team Fortress 2 logos.
Yeah. Well, and it was funny because, like, we’re having these CrossWorlds. Like, people are, you know, writing design docs and doing concept art for CrossWorlds. And, you know, we’d have some brainstorm meetings every day, and I put together… It was a seven-page deck, Overwatch deck. And it was called Monetized Shooter at the time. And it just said, “Monetized Shooter.” And then the first slide was League of Legends plus Team Fortress 2 logos.
Jeff Kaplan
And then I had, like, six heroes, like, sloppily designed. And as everybody was working on CrossWorlds there were two, you know, co-leaders of that team for… There was, you know… Chris Metzen was there, and Ray Gresko. And I remember Ray coming over. Ray is, like, a phenomenal game developer of all time. He, like, wrote the Dark Forces engine, was the production director on Diablo III. He and I killed Titan. And then he’s at my desk looking over my shoulder, and he’s like, “Well, what are you working on? Is this the CrossWorlds pitch?” I’m like, “No, this is, like, another idea that I’m just working on on the side.” And I show him the seven slides, and he just looks at me, and he says, “Go show Metzen this.”
And then I had, like, six heroes, like, sloppily designed. And as everybody was working on CrossWorlds there were two, you know, co-leaders of that team for… There was, you know… Chris Metzen was there, and Ray Gresko. And I remember Ray coming over. Ray is, like, a phenomenal game developer of all time. He, like, wrote the Dark Forces engine, was the production director on Diablo III. He and I killed Titan. And then he’s at my desk looking over my shoulder, and he’s like, “Well, what are you working on? Is this the CrossWorlds pitch?” I’m like, “No, this is, like, another idea that I’m just working on on the side.” And I show him the seven slides, and he just looks at me, and he says, “Go show Metzen this.”
Jeff Kaplan
This is what we should make instead.” And then I went and I showed Metzen, like, “Hey, this is just an idea.” And then Metzen was like, “Yes. You know, this is what we should make.” And I showed Arnold, and it was Arnold’s art. And then Ray tells me, he’s like… ‘Cause we would- Every morning, we’d get the team together ’cause we were in this dire, you know, dire straits, and we’re midway through at that point. And Ray and a producer named Matt Hawley said, “Tomorrow morning at the meeting, you’re gonna pitch this Monetized Shooter idea.” It was called Monetized Shooter because originally when I pitched it, it was free to play and you had to buy the heroes, which is fucking terrible, but at the time, I actually thought that was a good idea.
This is what we should make instead.” And then I went and I showed Metzen, like, “Hey, this is just an idea.” And then Metzen was like, “Yes. You know, this is what we should make.” And I showed Arnold, and it was Arnold’s art. And then Ray tells me, he’s like… ‘Cause we would- Every morning, we’d get the team together ’cause we were in this dire, you know, dire straits, and we’re midway through at that point. And Ray and a producer named Matt Hawley said, “Tomorrow morning at the meeting, you’re gonna pitch this Monetized Shooter idea.” It was called Monetized Shooter because originally when I pitched it, it was free to play and you had to buy the heroes, which is fucking terrible, but at the time, I actually thought that was a good idea.
Jeff Kaplan
And I’m walking down the hall with Matt Hawley to go, like, pitch this to this group, you know, we’re supposed to be working on CrossWorlds, and they’re like, “You gotta pitch this idea to them.” And Matt Hawley stops me in the hall and says, “You, Jeff, you cannot go into that meeting. I refuse to put up a deck in front of the team where the first slide says, ‘Monetized Shooter.'” “They’ll hate that, and that’s not the spirit of who we are-” “… as, you know, creative devel-” And I’m like, “Yeah, you’re right.” Like, well no one was supposed to see his deck anyway.
And I’m walking down the hall with Matt Hawley to go, like, pitch this to this group, you know, we’re supposed to be working on CrossWorlds, and they’re like, “You gotta pitch this idea to them.” And Matt Hawley stops me in the hall and says, “You, Jeff, you cannot go into that meeting. I refuse to put up a deck in front of the team where the first slide says, ‘Monetized Shooter.'” “They’ll hate that, and that’s not the spirit of who we are-” “… as, you know, creative devel-” And I’m like, “Yeah, you’re right.” Like, well no one was supposed to see his deck anyway.
Jeff Kaplan
You guys are all looking over my shoulder. He’s like, “You need to put a name on it.” I’m like, “It’s Overwatch.” Like, right on the spot, I said the name was Overwatch. And where that had come from was when we were working on Titan, I was really angry about this. We did this fake… I did not do this, another leader on the team did this, of this fake, like, we’re gonna put up whiteboards and everyone gets to vote for their favorite name for Titan. But the person who did it already had a name in mind- … for the game. And just kept pushing towards that name.
You guys are all looking over my shoulder. He’s like, “You need to put a name on it.” I’m like, “It’s Overwatch.” Like, right on the spot, I said the name was Overwatch. And where that had come from was when we were working on Titan, I was really angry about this. We did this fake… I did not do this, another leader on the team did this, of this fake, like, we’re gonna put up whiteboards and everyone gets to vote for their favorite name for Titan. But the person who did it already had a name in mind- … for the game. And just kept pushing towards that name.
Jeff Kaplan
And the thing that got the most votes was Overwatch. Overwatch in Titan was, like, a police group, essentially. But somebody had written Overwatch on that board and it got the most votes. So I basically named the game Overwatch to, like, high five my team- … and kind of middle finger. Like- Don’t act like it’s a democracy when it’s not.
And the thing that got the most votes was Overwatch. Overwatch in Titan was, like, a police group, essentially. But somebody had written Overwatch on that board and it got the most votes. So I basically named the game Overwatch to, like, high five my team- … and kind of middle finger. Like- Don’t act like it’s a democracy when it’s not.
Lex Fridman
Yeah, yeah.
Yeah, yeah.
Jeff Kaplan
You know? So…
You know? So…
Lex Fridman
So it’s a middle finger. So Overwatch, and then the, I mean, the rest is history. So what, in that slide deck, did you already have a kind of crawl, walk, run idea of the way this would be developed?
So it’s a middle finger. So Overwatch, and then the, I mean, the rest is history. So what, in that slide deck, did you already have a kind of crawl, walk, run idea of the way this would be developed?
Jeff Kaplan
So my, my deck was terrible. People actually- … there’s a thing called the Jeff Deck, which is: it’s always gray with black writing and then the default, like, PowerPoint blue shapes, because I just don’t bother making it look good-
So my, my deck was terrible. People actually- … there’s a thing called the Jeff Deck, which is: it’s always gray with black writing and then the default, like, PowerPoint blue shapes, because I just don’t bother making it look good-
Jeff Kaplan
… Besides dragging Arnold Tsang’s art, you know, desecrating it into my deck. We put together… We had this amazing game designer on the Overwatch team, a guy named Jeremy Craig who’s now actually game directing a game over at Bonfire. Jeremy, not only was he a great game designer, but he had the ability to sell things better than anybody else, visually. So Jeremy took my shitty deck, and then we had lots more, like, creative brainstorms and we thought through the game of Overwatch a lot more, and then he made this gorgeous pitch deck that we pitched. We first had to go through the Blizzard production and game directors for them to approve it and give it their thumbs up, then we had to go through the Blizzard executives, then we had to go through Activision.
… Besides dragging Arnold Tsang’s art, you know, desecrating it into my deck. We put together… We had this amazing game designer on the Overwatch team, a guy named Jeremy Craig who’s now actually game directing a game over at Bonfire. Jeremy, not only was he a great game designer, but he had the ability to sell things better than anybody else, visually. So Jeremy took my shitty deck, and then we had lots more, like, creative brainstorms and we thought through the game of Overwatch a lot more, and then he made this gorgeous pitch deck that we pitched. We first had to go through the Blizzard production and game directors for them to approve it and give it their thumbs up, then we had to go through the Blizzard executives, then we had to go through Activision.
Jeff Kaplan
And in that deck, because we had to speak to schedule, we had to speak to two things that were tough to speak to. One, we had to speak to schedule, and we came up with this concept of crawl, walk, run. We had identified the reason Titan failed is we just tried to run; we tried to come up with the next World of Warcraft. But if you think about World of Warcraft, it had Warcraft I, II and III to build upon to even get to the point where people gave a shit enough about that world to want to live in the world of Warcraft. So the idea was that instead of trying to cut right to World of Warcraft, let’s try to honor Warcraft I, essentially. So this first game is just to establish that there’s a universe you might give a shit about.
And in that deck, because we had to speak to schedule, we had to speak to two things that were tough to speak to. One, we had to speak to schedule, and we came up with this concept of crawl, walk, run. We had identified the reason Titan failed is we just tried to run; we tried to come up with the next World of Warcraft. But if you think about World of Warcraft, it had Warcraft I, II and III to build upon to even get to the point where people gave a shit enough about that world to want to live in the world of Warcraft. So the idea was that instead of trying to cut right to World of Warcraft, let’s try to honor Warcraft I, essentially. So this first game is just to establish that there’s a universe you might give a shit about.
Jeff Kaplan
We also knew that the timeframe we were given of two years, there was no way to create a compelling PvE experience, so we just kinda randomly put dates in a slide of crawl, walk, run, thinking it was aspirational, and really, we were just trying to save ourselves. Like, don’t cancel us. You know, this team can make something great. The other part that we had to talk to too was, like, a mobile strategy. Like, at that time, it was like, everything has to be also on mobile, which I think is the dumbest thing ever. And so literally what we did is, this was Jeremy’s brilliant part, we had a picture with all the boxes and then one of them is, like, a tablet with just a fucking Photoshop of, you know, Arnold’s art on it. We’re like, “And also-“
We also knew that the timeframe we were given of two years, there was no way to create a compelling PvE experience, so we just kinda randomly put dates in a slide of crawl, walk, run, thinking it was aspirational, and really, we were just trying to save ourselves. Like, don’t cancel us. You know, this team can make something great. The other part that we had to talk to too was, like, a mobile strategy. Like, at that time, it was like, everything has to be also on mobile, which I think is the dumbest thing ever. And so literally what we did is, this was Jeremy’s brilliant part, we had a picture with all the boxes and then one of them is, like, a tablet with just a fucking Photoshop of, you know, Arnold’s art on it. We’re like, “And also-“
Lex Fridman
Mobile
Mobile
Jeff Kaplan
“… it’ll be on mobile.”
“… it’ll be on mobile.”
Lex Fridman
Brilliant. But I think this crawl, walk, run idea is really nice. So the, the, the initial idea is you would have basically a shooter with all these different characters, all these heroes, and then the, the walk would be the PvE version of that, co-op. And then if people really fall in love with the world, then you build a big MMO around it. Quick pause for a bathroom break. Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Fin for customer service AI agents, Blitzy for code generation in large code bases, BetterHelp for mental health, Shopify for selling stuff online, CodeRabbit for AI-powered code review, and Perplexity for curiosity-driven knowledge exploration.
Brilliant. But I think this crawl, walk, run idea is really nice. So the, the, the initial idea is you would have basically a shooter with all these different characters, all these heroes, and then the, the walk would be the PvE version of that, co-op. And then if people really fall in love with the world, then you build a big MMO around it. Quick pause for a bathroom break. Quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Fin for customer service AI agents, Blitzy for code generation in large code bases, BetterHelp for mental health, Shopify for selling stuff online, CodeRabbit for AI-powered code review, and Perplexity for curiosity-driven knowledge exploration.
Best Overwatch heroes
Lex Fridman
Choose wisely, my friends. And now, back to my conversation with Jeff Kaplan. And we should also say that there’s a whole world that was built around Overwatch. And one of the ideas was… So, Warcraft is a very particular kind of world. StarCraft is a particular kind of world. Diablo is a particular kind of world. And you wanted to bring Overwatch to Earth and make it positive. You give this talk where there was a lot of respect paid to the sort of dark, gritty, post-apocalyptic games on Earth. Also gave a lot of respect to the ultra-realistic first-person shooter games like Call of Duty. And you wanted to create something more that paints a vision of a near-term hopeful future, and fun, and more sort of surreal, versus like ultra-real.
Choose wisely, my friends. And now, back to my conversation with Jeff Kaplan. And we should also say that there’s a whole world that was built around Overwatch. And one of the ideas was… So, Warcraft is a very particular kind of world. StarCraft is a particular kind of world. Diablo is a particular kind of world. And you wanted to bring Overwatch to Earth and make it positive. You give this talk where there was a lot of respect paid to the sort of dark, gritty, post-apocalyptic games on Earth. Also gave a lot of respect to the ultra-realistic first-person shooter games like Call of Duty. And you wanted to create something more that paints a vision of a near-term hopeful future, and fun, and more sort of surreal, versus like ultra-real.
Lex Fridman
So it’s interesting to talk through how a world comes to life. How you think about that world, how you create the tone of the game, how you think, how you craft in this vision. And not just, like, different characters like Tracer and so on, like what the personality is, but, like, bringing the world to life in which they will be. What was that process like?
So it’s interesting to talk through how a world comes to life. How you think about that world, how you create the tone of the game, how you think, how you craft in this vision. And not just, like, different characters like Tracer and so on, like what the personality is, but, like, bringing the world to life in which they will be. What was that process like?
Jeff Kaplan
The process was a blast. And, like, the goal was that bright, hopeful future. And the other phrase we used all the time on the team was, “A future worth fighting for.”
The process was a blast. And, like, the goal was that bright, hopeful future. And the other phrase we used all the time on the team was, “A future worth fighting for.”
Lex Fridman
Mm-hmm, yes.
Mm-hmm, yes.
Jeff Kaplan
You know, if there’s gonna be all this fighting, like the… it kinda has to be worth it for something. Picking the locations in the world was the funnest thing. You know, there’s just a group of us who would sit around, and be like, “Where do you wanna go?” You know, “Santorini looks amazing.” And you’re looking at pictures, and like, “Let’s make that place.” You know in a video game people are gonna spend hours and hours in a location. Resist the urge to do the common, I call them the cargo container mazes, that you see in every game. And I know why they exist, they’re easy to make, but we kinda wanted Overwatch to be this world tour of great places that you’d wanna go to.
You know, if there’s gonna be all this fighting, like the… it kinda has to be worth it for something. Picking the locations in the world was the funnest thing. You know, there’s just a group of us who would sit around, and be like, “Where do you wanna go?” You know, “Santorini looks amazing.” And you’re looking at pictures, and like, “Let’s make that place.” You know in a video game people are gonna spend hours and hours in a location. Resist the urge to do the common, I call them the cargo container mazes, that you see in every game. And I know why they exist, they’re easy to make, but we kinda wanted Overwatch to be this world tour of great places that you’d wanna go to.
Jeff Kaplan
Or in the case of like Oasis, it’s like, okay, maybe Iraq, back when we were making this game, wasn’t the top of people’s list, but what is the bright, hopeful version of what that could look like? So we just really tried to sell this idea of these aspirational locations. One, just to get people thinking about different places on Planet Earth and how awesome they all are. But also, from like a pure game design standpoint, you’re gonna spend a lot of time in the environment, so the environment should be pleasing and not oppressive.
Or in the case of like Oasis, it’s like, okay, maybe Iraq, back when we were making this game, wasn’t the top of people’s list, but what is the bright, hopeful version of what that could look like? So we just really tried to sell this idea of these aspirational locations. One, just to get people thinking about different places on Planet Earth and how awesome they all are. But also, from like a pure game design standpoint, you’re gonna spend a lot of time in the environment, so the environment should be pleasing and not oppressive.
Lex Fridman
Can you go through some of the heroes that you ended up putting in the game? Maybe a good way to do it is, which are your favorites? And what’s from the best of your knowledge of the internet, favorites?
Can you go through some of the heroes that you ended up putting in the game? Maybe a good way to do it is, which are your favorites? And what’s from the best of your knowledge of the internet, favorites?
Jeff Kaplan
My favorite… I have a couple favorite heroes. Obviously, Tracer.
My favorite… I have a couple favorite heroes. Obviously, Tracer.
Lex Fridman
She’s the OG.
She’s the OG.
Jeff Kaplan
The OG, the cornerstone. You know, we put her on the front of the box. She was that moment of, “We should just take the best of the best,” and we know this gameplay is good and solid. And it’s so simple. Like, the mechanics are very easy to explain to somebody. It’s very easy to pick up. The first time anybody hits Recall for the first time and they try to wrap their mind around like, “Wait, does that mean if I…” You know, and they’re mapping out the possibilities.
The OG, the cornerstone. You know, we put her on the front of the box. She was that moment of, “We should just take the best of the best,” and we know this gameplay is good and solid. And it’s so simple. Like, the mechanics are very easy to explain to somebody. It’s very easy to pick up. The first time anybody hits Recall for the first time and they try to wrap their mind around like, “Wait, does that mean if I…” You know, and they’re mapping out the possibilities.
Lex Fridman
And by the way, we should say that it’s a PvP game with six versus six at first, and where there’s three distinct roles that people take on on a team. And those roles, at first, I guess were not required. Like, you can reallocate those roles as you wanted. And then to maximize the fun, you add a little bit of structure. You enforce two per role, and the role being Tank, Support, and Damage. So, that. And then there’s all the kinds of heroes that are associated with the different roles, and people pick and there’s lore. And some people are probably like hardcore just one particular hero. And so there’s a lot of personality and story and community that builds around each of the heroes. And, but at the end of the day, it is just a fun shooter.
And by the way, we should say that it’s a PvP game with six versus six at first, and where there’s three distinct roles that people take on on a team. And those roles, at first, I guess were not required. Like, you can reallocate those roles as you wanted. And then to maximize the fun, you add a little bit of structure. You enforce two per role, and the role being Tank, Support, and Damage. So, that. And then there’s all the kinds of heroes that are associated with the different roles, and people pick and there’s lore. And some people are probably like hardcore just one particular hero. And so there’s a lot of personality and story and community that builds around each of the heroes. And, but at the end of the day, it is just a fun shooter.
Jeff Kaplan
Yeah. Our goal was to pay homage to the shooters before us that we loved. There’s no way you can talk about Overwatch without talking about Team Fortress 2. Uh, Team Fortress started as a Quake mod, which was brilliant and I played tons of. Then there was Team Fortress Classic that came out with Half-Life 1.
Yeah. Our goal was to pay homage to the shooters before us that we loved. There’s no way you can talk about Overwatch without talking about Team Fortress 2. Uh, Team Fortress started as a Quake mod, which was brilliant and I played tons of. Then there was Team Fortress Classic that came out with Half-Life 1.
Jeff Kaplan
And then Team Fortress 2, I think everything about it blew everybody away when it came out in 2007. And there’s obviously just huge influence there. But the shooter mechanics of Overwatch are… They hearken back to what people call the arcade or arena shooter genre. Which pains me ’cause I never… Back in the day, I didn’t think of Quake as an arcade shooter. It was almost an insulting way of saying it. But just the fast movement, really epic, over-the-top weapons. You have a low time to kill, or TTK, that players call it. Meaning you’re very survivable; you can take a few hits. Where, in a game like Call of Duty or Counter-Strike, if you get shot in the head, you’re just dead right away. So it was supposed to be this explosive, larger than life, fun, arcade-y shooter-
And then Team Fortress 2, I think everything about it blew everybody away when it came out in 2007. And there’s obviously just huge influence there. But the shooter mechanics of Overwatch are… They hearken back to what people call the arcade or arena shooter genre. Which pains me ’cause I never… Back in the day, I didn’t think of Quake as an arcade shooter. It was almost an insulting way of saying it. But just the fast movement, really epic, over-the-top weapons. You have a low time to kill, or TTK, that players call it. Meaning you’re very survivable; you can take a few hits. Where, in a game like Call of Duty or Counter-Strike, if you get shot in the head, you’re just dead right away. So it was supposed to be this explosive, larger than life, fun, arcade-y shooter-
Jeff Kaplan
With a lot of teamwork involved.
With a lot of teamwork involved.
Lex Fridman
And so you said Tracer up there? She’s the OG. Who else?
And so you said Tracer up there? She’s the OG. Who else?
Jeff Kaplan
McCree. McCree is another, like, I’m somebody who’s attracted to the simplicity and design. And I did not design McCree’s six shooter. The way that gun feels is phenomenal, and to capture the spirit of that, we had a designer named Mike Heiberg design the High Noon ultimate. And then just all the care and love the team put in, like when he does the ultimate, we roll a tumbleweed across the screen like every time. It’s a very simple hero, but the simplicity is what I like best in design.
McCree. McCree is another, like, I’m somebody who’s attracted to the simplicity and design. And I did not design McCree’s six shooter. The way that gun feels is phenomenal, and to capture the spirit of that, we had a designer named Mike Heiberg design the High Noon ultimate. And then just all the care and love the team put in, like when he does the ultimate, we roll a tumbleweed across the screen like every time. It’s a very simple hero, but the simplicity is what I like best in design.
Jeff Kaplan
I’m not a fan of when somebody starts explaining, you know, in any of these games, whether they’re MOBAs or hero shooters, and they start, like, “This guy throws orbs, and he throws three orbs, and then he runs out of his orb bank, and then he can call the orbs back, or he can catch the orbs.” And my head is spinning, and I’m like, “Just give me a fucking good gun.” You know? And I’m done.
I’m not a fan of when somebody starts explaining, you know, in any of these games, whether they’re MOBAs or hero shooters, and they start, like, “This guy throws orbs, and he throws three orbs, and then he runs out of his orb bank, and then he can call the orbs back, or he can catch the orbs.” And my head is spinning, and I’m like, “Just give me a fucking good gun.” You know? And I’m done.
Lex Fridman
Simplicity is everything. What about Reinhardt, the tank?
Simplicity is everything. What about Reinhardt, the tank?
Jeff Kaplan
Reinhardt was actually my main. So I played the most of Reinhardt. That was another amazing Geoff Goodman design of this guy who just has a shield. As soon as you give somebody a shield, they know what to do. They go into protector mode. The shield was designed to shoot through. The shield has since been copied by like every hero shooter since, and even non-hero shooters. And then he just has a giant rocket hammer. And he does a charge ability. It’s really interesting where the charge ability came from. I was playing a ton of Left 4 Dead 2, and you could play in versus mode where you could be the enemy zombie guys.
Reinhardt was actually my main. So I played the most of Reinhardt. That was another amazing Geoff Goodman design of this guy who just has a shield. As soon as you give somebody a shield, they know what to do. They go into protector mode. The shield was designed to shoot through. The shield has since been copied by like every hero shooter since, and even non-hero shooters. And then he just has a giant rocket hammer. And he does a charge ability. It’s really interesting where the charge ability came from. I was playing a ton of Left 4 Dead 2, and you could play in versus mode where you could be the enemy zombie guys.
Jeff Kaplan
And there was an enemy boss zombie called The Charger who had that charge ability. And I thought, the reason that ability was so cool is because it’s a commit. Once you press the button, you’re a runaway train. And watching Reinhardts charge to their deaths is kind of hilarious, and it’s what separates a great Rein from a shitty one.
And there was an enemy boss zombie called The Charger who had that charge ability. And I thought, the reason that ability was so cool is because it’s a commit. Once you press the button, you’re a runaway train. And watching Reinhardts charge to their deaths is kind of hilarious, and it’s what separates a great Rein from a shitty one.
The challenge of matchmaking
Lex Fridman
You’ve explained that the Overwatch matchmaker process is designed to keep players at a 50% win rate. I think it’s just a fascinating topic. Not to get too philosophical, but you can’t have the up without the down, hence the 50%. Can, can you speak to the complexity of like what makes a good matchmaker?
You’ve explained that the Overwatch matchmaker process is designed to keep players at a 50% win rate. I think it’s just a fascinating topic. Not to get too philosophical, but you can’t have the up without the down, hence the 50%. Can, can you speak to the complexity of like what makes a good matchmaker?
Jeff Kaplan
The matchmaking systems are some of the most complex design and engineering tasks you’re ever gonna tackle. And they’re thankless. It’s, it’s very hard, too, because I think most people, and they’re not being disingenuous, like if you ask a gamer, “What do you want?” They’re like, “I just want a fair match. Like, just make it even.” And the reality of what they want is they want a match where they’re slightly better than the other guy.
The matchmaking systems are some of the most complex design and engineering tasks you’re ever gonna tackle. And they’re thankless. It’s, it’s very hard, too, because I think most people, and they’re not being disingenuous, like if you ask a gamer, “What do you want?” They’re like, “I just want a fair match. Like, just make it even.” And the reality of what they want is they want a match where they’re slightly better than the other guy.
Jeff Kaplan
Like, they want it to feel like it was close but then win. And you can’t architect that. Like, there, it’s, you know, it’s a zero-sum situation, so there’s gotta be winners and there’s gotta be losers. The other really core problem, and we would study this all the time when people would complain. You know, you see a Reddit post, and somebody would say, “I had a six game losing streak. This is so fucked. It’s the worst matchmaker ever.”
Like, they want it to feel like it was close but then win. And you can’t architect that. Like, there, it’s, you know, it’s a zero-sum situation, so there’s gotta be winners and there’s gotta be losers. The other really core problem, and we would study this all the time when people would complain. You know, you see a Reddit post, and somebody would say, “I had a six game losing streak. This is so fucked. It’s the worst matchmaker ever.”
Lex Fridman
Oh, Reddit.
Oh, Reddit.
Jeff Kaplan
Yeah, right? I love Reddit.
Yeah, right? I love Reddit.
Lex Fridman
Me too.
Me too.
Jeff Kaplan
But we would look up that person’s account. I would do that all the time. I love looking up people’s accounts and seeing- … what would happen. It’s like, yeah, he had the six-game losing streak. He had an eight-game winning streak before that. There was no post about how awesome is this. And the human psychology doesn’t allow for that. One of my hindsight regrets about Overwatch, and this is, I think we did the right thing in the moment. It’s you know, like, I wouldn’t go back and redo it, but if I was making a hero shooter from scratch today, I would make it less team-focused. And we put all of our eggs in you noticing if the team won or lost.
But we would look up that person’s account. I would do that all the time. I love looking up people’s accounts and seeing- … what would happen. It’s like, yeah, he had the six-game losing streak. He had an eight-game winning streak before that. There was no post about how awesome is this. And the human psychology doesn’t allow for that. One of my hindsight regrets about Overwatch, and this is, I think we did the right thing in the moment. It’s you know, like, I wouldn’t go back and redo it, but if I was making a hero shooter from scratch today, I would make it less team-focused. And we put all of our eggs in you noticing if the team won or lost.
Jeff Kaplan
And we downplayed your individual contribution as much as possible. There wasn’t a scoreboard. We had a medal system, but the medal system was, in my opinion, it was not good because the losing team got medals and the winning team got medals. And on the losing team, they would use that. They would weaponize it against their teammate. “Well, I’m the top kills, and all you guys are making us lose.” And it’s like, “Okay, you’re the top kills by like one, and you guys still lost.” So I would, if I was to redo it today, or for any aspiring hero shooter makers out there, I would actually downplay the team factor, and try to put more focus on individual contribution.
And we downplayed your individual contribution as much as possible. There wasn’t a scoreboard. We had a medal system, but the medal system was, in my opinion, it was not good because the losing team got medals and the winning team got medals. And on the losing team, they would use that. They would weaponize it against their teammate. “Well, I’m the top kills, and all you guys are making us lose.” And it’s like, “Okay, you’re the top kills by like one, and you guys still lost.” So I would, if I was to redo it today, or for any aspiring hero shooter makers out there, I would actually downplay the team factor, and try to put more focus on individual contribution.
Jeff Kaplan
Because that’s just how people play. They’re, they’re selfish. And I don’t mean that in a bad way. It’s just, it’s that human nature, they can’t help.
Because that’s just how people play. They’re, they’re selfish. And I don’t mean that in a bad way. It’s just, it’s that human nature, they can’t help.
Rust
Lex Fridman
And in terms of how they experience the game, in terms of how they derive joy from it, or how they see the challenge of the game is individual. Even when you’re on a team, you’re still feeling- … it’s individual, a fundamental individual experience. Let me, as a small aside, before I forget, since we mentioned first-person shooters so much, outside of Overwatch, what are some of the great shooters of all time that you’ve played?
And in terms of how they experience the game, in terms of how they derive joy from it, or how they see the challenge of the game is individual. Even when you’re on a team, you’re still feeling- … it’s individual, a fundamental individual experience. Let me, as a small aside, before I forget, since we mentioned first-person shooters so much, outside of Overwatch, what are some of the great shooters of all time that you’ve played?
Jeff Kaplan
Quake is the greatest.
Quake is the greatest.
Lex Fridman
Quake is GOAT.
Quake is GOAT.
Jeff Kaplan
Yeah. Quake is GOAT. There’s a lot of contenders up there.
Yeah. Quake is GOAT. There’s a lot of contenders up there.
Lex Fridman
What have you logged the most hours in outside of the games?
What have you logged the most hours in outside of the games?
Jeff Kaplan
Rust.
Rust.
Lex Fridman
Okay. Can you… Okay. A lot of folks have written to me that I need to play Rust, the video game. I have not even looked into it. Somebody on Reddit said it has a steep learning curve. I would like to give it a chance because you have to me spoken so highly of it. So can you explain Rust?
Okay. Can you… Okay. A lot of folks have written to me that I need to play Rust, the video game. I have not even looked into it. Somebody on Reddit said it has a steep learning curve. I would like to give it a chance because you have to me spoken so highly of it. So can you explain Rust?
Jeff Kaplan
Yeah. Rust is an open world game. It’s a procedural map, so it means that every time it’s different. You’re always on an island, and it resets every month. So-
Yeah. Rust is an open world game. It’s a procedural map, so it means that every time it’s different. You’re always on an island, and it resets every month. So-
Lex Fridman
Is it PvP?
Is it PvP?
Jeff Kaplan
It’s all PvP. In fact, Rust is the most PvP thing in all of PvP.
It’s all PvP. In fact, Rust is the most PvP thing in all of PvP.
Lex Fridman
Well, I don’t know what that means, but-
Well, I don’t know what that means, but-
Jeff Kaplan
Rust players know what that means.
Rust players know what that means.
Lex Fridman
Everybody who plays Rust and loves it sounds to me like they’re in a cult. So I with all due respect, please don’t write me letters.
Everybody who plays Rust and loves it sounds to me like they’re in a cult. So I with all due respect, please don’t write me letters.
Jeff Kaplan
They’re too busy playing Rust. They’re too busy checking on their base, making sure it’s not raided, to write you letters.
They’re too busy playing Rust. They’re too busy checking on their base, making sure it’s not raided, to write you letters.
Lex Fridman
Oh, good.
Oh, good.
Jeff Kaplan
It takes place… It’s basically… It’s open world. You can do whatever you want. There’s not really any directed gameplay to it, but at any time, any other player can kill you and take anything that’s on you.
It takes place… It’s basically… It’s open world. You can do whatever you want. There’s not really any directed gameplay to it, but at any time, any other player can kill you and take anything that’s on you.
Lex Fridman
Oh, wow.
Oh, wow.
Jeff Kaplan
Yeah, and then you build what Rust players call bases, and you upgrade the base, and you try to make the base as safe as possible to store your stuff, and then you can make explosives and blow up other people’s walls to get into their base where they’re keeping all their best stuff and take all their shit.
Yeah, and then you build what Rust players call bases, and you upgrade the base, and you try to make the base as safe as possible to store your stuff, and then you can make explosives and blow up other people’s walls to get into their base where they’re keeping all their best stuff and take all their shit.
Lex Fridman
Like, permanently?
Like, permanently?
Jeff Kaplan
Permanently. Like-
Permanently. Like-
Lex Fridman
Oh, I see.
Oh, I see.
Jeff Kaplan
… it would be like PvPing in WoW. Imagine in World of Warcraft- … if somebody could not only kill you but take everything that’s in your bank and make you level one the next time you log in.
… it would be like PvPing in WoW. Imagine in World of Warcraft- … if somebody could not only kill you but take everything that’s in your bank and make you level one the next time you log in.
Lex Fridman
Wow. That’s very stressful.
Wow. That’s very stressful.
Jeff Kaplan
The beauty of Rust, and why it’s so good, is you can’t have the high highs without the low lows. And-
The beauty of Rust, and why it’s so good, is you can’t have the high highs without the low lows. And-
Lex Fridman
Like, real low lows.
Like, real low lows.
Jeff Kaplan
Real low lows.
Real low lows.
Lex Fridman
Wow. All right.
Wow. All right.
Jeff Kaplan
Like, debilitating, like, “am I ever gonna play this game” lows.
Like, debilitating, like, “am I ever gonna play this game” lows.
Lex Fridman
Right.
Right.
Jeff Kaplan
You know, like, you spend a week building the world’s most perfect base and getting tons of loot, and then it… There’s what’s called online raiding and offline raiding. Online raiding means that my enemy is… I can see that they’re in their base right now, and I’m gonna try to attack them while they’re in their base. Offlining, which is, like, all Rust players will say you’re the scum of the Earth if you offline someone, and then all Rust players also offline people all the time. Yeah. It’s-
You know, like, you spend a week building the world’s most perfect base and getting tons of loot, and then it… There’s what’s called online raiding and offline raiding. Online raiding means that my enemy is… I can see that they’re in their base right now, and I’m gonna try to attack them while they’re in their base. Offlining, which is, like, all Rust players will say you’re the scum of the Earth if you offline someone, and then all Rust players also offline people all the time. Yeah. It’s-
Lex Fridman
Yes,
Yes,
Jeff Kaplan
… gamer etiquette.
… gamer etiquette.
Lex Fridman
Yes.
Yes.
Jeff Kaplan
Offlining’s when, like, “Hey, I think that my neighbor logged off for the night. You know, they just played six hours. I’ve been watching them, and now there’s no activity in their base, so I’m gonna, like, blow up their walls and take up all their stuff when they’re not here.”
Offlining’s when, like, “Hey, I think that my neighbor logged off for the night. You know, they just played six hours. I’ve been watching them, and now there’s no activity in their base, so I’m gonna, like, blow up their walls and take up all their stuff when they’re not here.”
Lex Fridman
Mm-hmm. Yeah. So Rust, because real life is not hard enough, is what it sounds like. Just, I want… If I want-
Mm-hmm. Yeah. So Rust, because real life is not hard enough, is what it sounds like. Just, I want… If I want-
Jeff Kaplan
That’d be a great tag.
That’d be a great tag.
Lex Fridman
If I want more stress in my life, I’ll play Rust. Yeah. I can’t wait. So okay, so that’s one. That sounds like a unique experience and a great joy. So quick number one, Rust in up there.
If I want more stress in my life, I’ll play Rust. Yeah. I can’t wait. So okay, so that’s one. That sounds like a unique experience and a great joy. So quick number one, Rust in up there.
Jeff Kaplan
Call of Duty.
Call of Duty.
Lex Fridman
Call of Duty just has its own-
Call of Duty just has its own-
Jeff Kaplan
You know, there’s a lot of haters. Like, Call of Duty 4 and Modern Warfare 2 were the pinnacle of Call of Duty, with Black Ops being a very respectable, you know, third. But you’re never gonna get a better gun feel from a game than Call… Like, just study the visual effects, the animation, the modeling, the sounds. Every aspect of shooting a gun in Call of Duty is so masterfully done. And then the maps, like, the flow of the multiplayer is just great. Like, there’s… There’s a map called Crash from Call of Duty 4 that Aaron Keller and I… Aaron’s now the game director on Overwatch. We just sat and studied that map, or Terminal from Modern Warfare 2. Just studied the maps of just, like, this map design is off the hook. So Call of Duty is definitely up there.
You know, there’s a lot of haters. Like, Call of Duty 4 and Modern Warfare 2 were the pinnacle of Call of Duty, with Black Ops being a very respectable, you know, third. But you’re never gonna get a better gun feel from a game than Call… Like, just study the visual effects, the animation, the modeling, the sounds. Every aspect of shooting a gun in Call of Duty is so masterfully done. And then the maps, like, the flow of the multiplayer is just great. Like, there’s… There’s a map called Crash from Call of Duty 4 that Aaron Keller and I… Aaron’s now the game director on Overwatch. We just sat and studied that map, or Terminal from Modern Warfare 2. Just studied the maps of just, like, this map design is off the hook. So Call of Duty is definitely up there.
Lex Fridman
So even though you were not thinking about it, Overwatch ended up being a gigantic success. So did you start thinking about, in this framework of crawl, walk, run, about the walk, the PvE piece?
So even though you were not thinking about it, Overwatch ended up being a gigantic success. So did you start thinking about, in this framework of crawl, walk, run, about the walk, the PvE piece?
Jeff Kaplan
Yes. So the PvE piece was what Overwatch 2 was supposed to be. And I don’t know if people know this or not, but we started working on Overwatch 2 in 2015.
Yes. So the PvE piece was what Overwatch 2 was supposed to be. And I don’t know if people know this or not, but we started working on Overwatch 2 in 2015.
Lex Fridman
♪ Over- ♪
♪ Over- ♪
Jeff Kaplan
So, Overwatch 1 didn’t ship until 2016. So before Overwatch… And it wasn’t like work in earnest. It was like pitching the game. I remember I spent a lot of time… It was myself, Chris Metzen, and Michael Chu sort of brainstorming a framework for what, like, a campaign could look like. And we had this idea of, like, a cooperative PvE shooter. And we actually pitched it to the team before we launched because we were trying to put a bunch of runway in front of us. That worked against us, and it’s one of my biggest mistakes I’ve made as a creative leader in my career, was Overwatch 2. There were two points of failure for me.
So, Overwatch 1 didn’t ship until 2016. So before Overwatch… And it wasn’t like work in earnest. It was like pitching the game. I remember I spent a lot of time… It was myself, Chris Metzen, and Michael Chu sort of brainstorming a framework for what, like, a campaign could look like. And we had this idea of, like, a cooperative PvE shooter. And we actually pitched it to the team before we launched because we were trying to put a bunch of runway in front of us. That worked against us, and it’s one of my biggest mistakes I’ve made as a creative leader in my career, was Overwatch 2. There were two points of failure for me.
Jeff Kaplan
The first was, I had people on the game team who didn’t like PvP or competitive shooters, and they really loved the Overwatch universe and wanted to play these characters and heroes, but they wanted to kind of do it on their own terms in like a PvE setting. So even though Overwatch is this like runaway success and everybody’s talking about it, they felt like they couldn’t really engage with it. And so like people on the dev team are like, “Okay, thank God we, you know, shipped that PvP thing-“
The first was, I had people on the game team who didn’t like PvP or competitive shooters, and they really loved the Overwatch universe and wanted to play these characters and heroes, but they wanted to kind of do it on their own terms in like a PvE setting. So even though Overwatch is this like runaway success and everybody’s talking about it, they felt like they couldn’t really engage with it. And so like people on the dev team are like, “Okay, thank God we, you know, shipped that PvP thing-“
Jeff Kaplan
“… When do we start work on this other thing?” So that came from a genuine place of excitement. And then the other point of pressure was from the executive team, and this was both the Blizzard and more so the Activision executive teams, and they started really putting the heat on, “Well, you said Overwatch 2 was gonna be out in 2019.” And they’re referring back to these slides that were just crazy dates. Like- … it was… You never want to put a PowerPoint deck in front of a corporate executive. Like, you might as well etch it in stone and come down from the mountain on it.
“… When do we start work on this other thing?” So that came from a genuine place of excitement. And then the other point of pressure was from the executive team, and this was both the Blizzard and more so the Activision executive teams, and they started really putting the heat on, “Well, you said Overwatch 2 was gonna be out in 2019.” And they’re referring back to these slides that were just crazy dates. Like- … it was… You never want to put a PowerPoint deck in front of a corporate executive. Like, you might as well etch it in stone and come down from the mountain on it.
Lex Fridman
So you just threw some dates because the layout looked good.
So you just threw some dates because the layout looked good.
Jeff Kaplan
Yeah. This is just all bullshit. This is just… In the same way we put, like, the tablet, you know? We just put Overwatch, like put Tracer on a tablet and say we have a mobile strategy.
Yeah. This is just all bullshit. This is just… In the same way we put, like, the tablet, you know? We just put Overwatch, like put Tracer on a tablet and say we have a mobile strategy.
Jeff Kaplan
So the executives started getting really angry at us that Overwatch 2 was slipping, slipping. And so when Overwatch 1 took off, I remember very early, we were in like May of 2016, and that year the Olympics were gonna be in Rio, I think. And, you know, I always like to pay, pay respects to, like, when a big event is happening, I’m like, “Hey, we should do, like, an event for the Olympics.” You can’t call it the Olympics or else they sue you, so you just… Even though you’re advertising for them to a bunch of kids who want to play video games and not watch the Olympics. But we also had like these two developers, Mike Heiberg and Dave Adams, like worked on this quirky… Like, they made soccer in Overwatch. We called it Lúcioball.
So the executives started getting really angry at us that Overwatch 2 was slipping, slipping. And so when Overwatch 1 took off, I remember very early, we were in like May of 2016, and that year the Olympics were gonna be in Rio, I think. And, you know, I always like to pay, pay respects to, like, when a big event is happening, I’m like, “Hey, we should do, like, an event for the Olympics.” You can’t call it the Olympics or else they sue you, so you just… Even though you’re advertising for them to a bunch of kids who want to play video games and not watch the Olympics. But we also had like these two developers, Mike Heiberg and Dave Adams, like worked on this quirky… Like, they made soccer in Overwatch. We called it Lúcioball.
Jeff Kaplan
Like, they made a map and they made these mechanics. We’re like, “Yeah, let’s do an event called the Summer Games.”
Like, they made a map and they made these mechanics. We’re like, “Yeah, let’s do an event called the Summer Games.”
Jeff Kaplan
And we do a live patch that’s the Summer Games. It’s extremely successful. And then after that, we’re like, “Yeah, let’s do… Halloween’s coming up. Let’s do a Halloween event. How cool will that be?” And our fans just loved these events, but there were two groups that were struggling with it. One was that group I told you on the dev team who was like, “Oh my God, you guys are over-scoping the patches. Why are we doing this Halloween event? We should be doing… We should start work on Overwatch 2. We shouldn’t be this focused on the live game,” which was fucking nuts. Like, that was just crazy. There’s this phrase of catch the wave, ride the wave. Most games fall off the back of the wave. They don’t catch the wave. No one plays it or plays it for two weeks.
And we do a live patch that’s the Summer Games. It’s extremely successful. And then after that, we’re like, “Yeah, let’s do… Halloween’s coming up. Let’s do a Halloween event. How cool will that be?” And our fans just loved these events, but there were two groups that were struggling with it. One was that group I told you on the dev team who was like, “Oh my God, you guys are over-scoping the patches. Why are we doing this Halloween event? We should be doing… We should start work on Overwatch 2. We shouldn’t be this focused on the live game,” which was fucking nuts. Like, that was just crazy. There’s this phrase of catch the wave, ride the wave. Most games fall off the back of the wave. They don’t catch the wave. No one plays it or plays it for two weeks.
Jeff Kaplan
If you’re lucky enough to have caught the wave- … ride it till the end. And my instincts at that point were like, “Let’s just keep… How many more of these live events can we do?”
If you’re lucky enough to have caught the wave- … ride it till the end. And my instincts at that point were like, “Let’s just keep… How many more of these live events can we do?”
Why Jeff left Blizzard
Lex Fridman
So yeah. So now there’s this wave in the live game and events, but the pressure on creating Overwatch 2 was building.
So yeah. So now there’s this wave in the live game and events, but the pressure on creating Overwatch 2 was building.
Jeff Kaplan
Yeah. We had a coalition on the team that was… really wanted Overwatch 2 built instead of the live events. And then the executive pressure became monumental. And what would have been correct was to do more world events, like keep it going, but the major derail was Overwatch League. And we really like… The, the weirdest part about Overwatch League is I believe in it. You know, I helped pitch it along with some other people. We thought it was like the future of esports and doing regional-based teams, ensuring minimum player salaries and player protections. Like, there was a lot of very good about Overwatch League.
Yeah. We had a coalition on the team that was… really wanted Overwatch 2 built instead of the live events. And then the executive pressure became monumental. And what would have been correct was to do more world events, like keep it going, but the major derail was Overwatch League. And we really like… The, the weirdest part about Overwatch League is I believe in it. You know, I helped pitch it along with some other people. We thought it was like the future of esports and doing regional-based teams, ensuring minimum player salaries and player protections. Like, there was a lot of very good about Overwatch League.
Lex Fridman
And there would be teams associated with particular cities.
And there would be teams associated with particular cities.
Jeff Kaplan
Yes.
Yes.
Lex Fridman
And it would be international. It would be real competition. So the dream, the ambition was really huge there.
And it would be international. It would be real competition. So the dream, the ambition was really huge there.
Jeff Kaplan
Yeah. The teams part of the dream was more of like regional based, player protection, try to make esports more of a first class citizen, because there were all these stories about like shady teams, you know, screwing their players over. Where it got away from us was there was a lot of excitement about Overwatch League, like too much so, and then it got over marketed to the people buying the teams. They went on this road show where they had a deck basically, and like you could put anything in a deck and sell anything, and they were pretty much selling the Brooklyn Bridge, that Overwatch League was going to be more popular than the NFL.
Yeah. The teams part of the dream was more of like regional based, player protection, try to make esports more of a first class citizen, because there were all these stories about like shady teams, you know, screwing their players over. Where it got away from us was there was a lot of excitement about Overwatch League, like too much so, and then it got over marketed to the people buying the teams. They went on this road show where they had a deck basically, and like you could put anything in a deck and sell anything, and they were pretty much selling the Brooklyn Bridge, that Overwatch League was going to be more popular than the NFL.
Jeff Kaplan
And we got a bunch of…… billionaire investors in these teams. And when 2018 started, like for example the day I got back, they said, “We signed this huge deal with Twitch for streaming of Overwatch League,” like a media rights deal. And that means that here’s all these commitments we made for Overwatch League of like in-game stuff that had to exist. Like a lot of it was integration with Twitch and camera control and that kind of stuff. The other part of it was a bunch of skins and you know, uniforms for all the teams, which was not just getting the art in the game, but there was huge technical challenges to, like, how all that worked and was efficient and hit the right, you know, memory footprint and all of that kind of stuff.
And we got a bunch of…… billionaire investors in these teams. And when 2018 started, like for example the day I got back, they said, “We signed this huge deal with Twitch for streaming of Overwatch League,” like a media rights deal. And that means that here’s all these commitments we made for Overwatch League of like in-game stuff that had to exist. Like a lot of it was integration with Twitch and camera control and that kind of stuff. The other part of it was a bunch of skins and you know, uniforms for all the teams, which was not just getting the art in the game, but there was huge technical challenges to, like, how all that worked and was efficient and hit the right, you know, memory footprint and all of that kind of stuff.
Jeff Kaplan
And so all of your plans at that point kind of go out the window. Like you’re not gonna work on new world events. You’re not really even focused on Overwatch 2, you’re just kind of treading water. There was a lot of talk of like, “Oh God, you know, the deal, like, the deal didn’t go well and we’ve got to do make goods to make the deal better for them.” I’m like, “Just give them some money back, you know?” Like, if you… The deal isn’t what people wanted, like, putting it on us, the Overwatch team, to, like, support this beast.
And so all of your plans at that point kind of go out the window. Like you’re not gonna work on new world events. You’re not really even focused on Overwatch 2, you’re just kind of treading water. There was a lot of talk of like, “Oh God, you know, the deal, like, the deal didn’t go well and we’ve got to do make goods to make the deal better for them.” I’m like, “Just give them some money back, you know?” Like, if you… The deal isn’t what people wanted, like, putting it on us, the Overwatch team, to, like, support this beast.
Jeff Kaplan
And it was a great idea that the wrong instincts and sort of, I don’t know how to phrase this in a way that’s not damning, but there was too much focus on, “Let’s make lots of money really fast.” And a lot of people got dragged into it. And while Overwatch League was great for Overwatch in terms of the players that it brought in, and the Overwatch League players, they were awesome. I love them. The Overwatch League staff at Blizzard, some of the nicest, most motivated, great creative people- … like all of these organizations got built and they were all great, but it was a house of cards waiting to fall.
And it was a great idea that the wrong instincts and sort of, I don’t know how to phrase this in a way that’s not damning, but there was too much focus on, “Let’s make lots of money really fast.” And a lot of people got dragged into it. And while Overwatch League was great for Overwatch in terms of the players that it brought in, and the Overwatch League players, they were awesome. I love them. The Overwatch League staff at Blizzard, some of the nicest, most motivated, great creative people- … like all of these organizations got built and they were all great, but it was a house of cards waiting to fall.
Lex Fridman
And when it became more about the money versus the quality of the experience of the different teams playing together and actually building this ecosystem of esports…
And when it became more about the money versus the quality of the experience of the different teams playing together and actually building this ecosystem of esports…
Jeff Kaplan
The financial reality kicked in, where these teams now, we didn’t just have, you know, executives at Activision and Blizzard who cared about the bottom line of Overwatch. We had all these people who basically invested in the game, and then they started to express their opinions. Originally, the business model was going to be that they were going to do in-person events and there’s going to be big ticket sales and then merch, you know, and all of that. And I think really quickly everybody learned like, yeah, we can’t do in-game events when you have a London team and a Shanghai team and, like, how does this work? So that fell apart super quickly. The merch was good, but it wasn’t going to be making NFL level money-
The financial reality kicked in, where these teams now, we didn’t just have, you know, executives at Activision and Blizzard who cared about the bottom line of Overwatch. We had all these people who basically invested in the game, and then they started to express their opinions. Originally, the business model was going to be that they were going to do in-person events and there’s going to be big ticket sales and then merch, you know, and all of that. And I think really quickly everybody learned like, yeah, we can’t do in-game events when you have a London team and a Shanghai team and, like, how does this work? So that fell apart super quickly. The merch was good, but it wasn’t going to be making NFL level money-
Jeff Kaplan
… whatever insanity anybody thought that was going to be. So everybody quickly defaulted back to, “Hey, didn’t Overwatch make like $500 million just in the live game last year? What can we sell and what can you give us?” That pressure comes onto the team, and then the pressure to ship Overwatch 2 and all the care and love that we had for, like, the live game and the live server, “Let’s just make events and new heroes and new maps,” we’re losing all these resources. And it got to the point, you know, my exit at Blizzard, I believed in Overwatch 2. I think we could have made a great game. I have a lot of hindsight of, like, how I would have designed that game differently with what I know now versus what ultimately we didn’t ship.
… whatever insanity anybody thought that was going to be. So everybody quickly defaulted back to, “Hey, didn’t Overwatch make like $500 million just in the live game last year? What can we sell and what can you give us?” That pressure comes onto the team, and then the pressure to ship Overwatch 2 and all the care and love that we had for, like, the live game and the live server, “Let’s just make events and new heroes and new maps,” we’re losing all these resources. And it got to the point, you know, my exit at Blizzard, I believed in Overwatch 2. I think we could have made a great game. I have a lot of hindsight of, like, how I would have designed that game differently with what I know now versus what ultimately we didn’t ship.
Jeff Kaplan
And Overwatch 2 is out now, but it’s not the Overwatch 2 that we planned and announced.
And Overwatch 2 is out now, but it’s not the Overwatch 2 that we planned and announced.
Lex Fridman
So when you’re referring to Overwatch 2 in this conversation, you’re referring to the PvE version?
So when you’re referring to Overwatch 2 in this conversation, you’re referring to the PvE version?
Jeff Kaplan
The PvE version.
The PvE version.
Lex Fridman
Which, by the way, I would have loved to play. I’m one of the people that were… Overwatch is great, but the PvP, but I would have loved to play the PvE version.
Which, by the way, I would have loved to play. I’m one of the people that were… Overwatch is great, but the PvP, but I would have loved to play the PvE version.
Jeff Kaplan
I think everybody would have loved to have played it. And there’s a misconception online that all I cared about was PvE and I didn’t care about PvP. All of the Overwatch 2 PvP maps were something that I said to the team over and over, “We have a PvP audience. If we get anything right, it has to be the PvP.” We would be lucky to welcome these PvE players, but that’s not guaranteed. So it was never a PvE only focus.
I think everybody would have loved to have played it. And there’s a misconception online that all I cared about was PvE and I didn’t care about PvP. All of the Overwatch 2 PvP maps were something that I said to the team over and over, “We have a PvP audience. If we get anything right, it has to be the PvP.” We would be lucky to welcome these PvE players, but that’s not guaranteed. So it was never a PvE only focus.
Lex Fridman
It’s just almost expanding it to also the E.
It’s just almost expanding it to also the E.
Jeff Kaplan
Yeah. And what eventually broke me was it used to be like in 2016 and 2017, I felt very in control of the Overwatch team and the direction of the game as a game director, you know, working with Ray Gresko as the production director, it felt like we were running Overwatch.
Yeah. And what eventually broke me was it used to be like in 2016 and 2017, I felt very in control of the Overwatch team and the direction of the game as a game director, you know, working with Ray Gresko as the production director, it felt like we were running Overwatch.
Jeff Kaplan
And we were very, very successful and doing a good job. And I think the fans were happy. And then as we transitioned, you know, Overwatch League was the best intention. You know, my parents always say, “The road to hell is paved with good intentions.” That was the Overwatch League, and it ended up being an albatross. And then Overwatch 2 is the same thing. And what it boiled down for me, like what sort of ultimately broke me in my Blizzard career was I got called in the CFO’s office, and he sits me down and he says, he gives me a date, which at the time was 2020 and was going to slip to 2021, but at the time, it was 2020.
And we were very, very successful and doing a good job. And I think the fans were happy. And then as we transitioned, you know, Overwatch League was the best intention. You know, my parents always say, “The road to hell is paved with good intentions.” That was the Overwatch League, and it ended up being an albatross. And then Overwatch 2 is the same thing. And what it boiled down for me, like what sort of ultimately broke me in my Blizzard career was I got called in the CFO’s office, and he sits me down and he says, he gives me a date, which at the time was 2020 and was going to slip to 2021, but at the time, it was 2020.
Jeff Kaplan
And he said, “Overwatch has to make in 2020, and then every year after that, it needs a recurring revenue of…” And then he says to me, “If it doesn’t do dollars, we’re gonna lay off a thousand people, and that’s gonna be on you.” And that was just the biggest fuck you moment I had in my career. It felt surreal to be in that condition. And as somebody who’s worked on a lot of games, made a lot of games, you get in these meetings where they’re like, “There’s Fortnite, has 1,400 people working on it. If you just hire 1,400 people and make it free-to-play, we’ll make that money, right?” And that was… I had believed I would never work any place but Blizzard. I loved it. It was a part of who I was and I felt I was a part of it, and I literally thought I would retire from the place.
And he said, “Overwatch has to make in 2020, and then every year after that, it needs a recurring revenue of…” And then he says to me, “If it doesn’t do dollars, we’re gonna lay off a thousand people, and that’s gonna be on you.” And that was just the biggest fuck you moment I had in my career. It felt surreal to be in that condition. And as somebody who’s worked on a lot of games, made a lot of games, you get in these meetings where they’re like, “There’s Fortnite, has 1,400 people working on it. If you just hire 1,400 people and make it free-to-play, we’ll make that money, right?” And that was… I had believed I would never work any place but Blizzard. I loved it. It was a part of who I was and I felt I was a part of it, and I literally thought I would retire from the place.
Jeff Kaplan
I never thought the day would come, and that was it. I was like, we’re done here. Luckily for Blizzard, that CFO is no longer there.
I never thought the day would come, and that was it. I was like, we’re done here. Luckily for Blizzard, that CFO is no longer there.
Lex Fridman
I mean, Blizzard is one of the greatest companies in the history of Earth. They’ve created so many incredible video games. It’s so difficult to create so many hits, and they were done not by chasing money. They’re done by small incredible teams, the hodgepodge that you describe taking big risks and falling in love with the thing they do and then just chasing it, working extremely hard. And just because you figured out a way how to make a lot of money doesn’t mean it’s not at the core this incredible creative journey that’s incredibly difficult to pull off. And just because you got a bunch of really smart creative people who have somehow figured out how to pull it off multiple times in a row doesn’t mean you can just treat it like a machine.
I mean, Blizzard is one of the greatest companies in the history of Earth. They’ve created so many incredible video games. It’s so difficult to create so many hits, and they were done not by chasing money. They’re done by small incredible teams, the hodgepodge that you describe taking big risks and falling in love with the thing they do and then just chasing it, working extremely hard. And just because you figured out a way how to make a lot of money doesn’t mean it’s not at the core this incredible creative journey that’s incredibly difficult to pull off. And just because you got a bunch of really smart creative people who have somehow figured out how to pull it off multiple times in a row doesn’t mean you can just treat it like a machine.
Lex Fridman
Every single time, it’s this beautiful journey of hodgepodge of weirdos working together, and weirdos have to run that thing. If you have, ever have a chance to create something special, you have to have weirdos at the helm. And the degree to which you don’t have weirdos at the helm, creative minds at the helm and you’re a businessperson at the helm, get out of their way, right? You can’t, you cannot have the meetings like you’re describing. And I don’t just speak about this particular company. It’s just the entire industry. I just, there’s so much joy to be had if we keep creating great games, and I just hope we get to see those great games.
Every single time, it’s this beautiful journey of hodgepodge of weirdos working together, and weirdos have to run that thing. If you have, ever have a chance to create something special, you have to have weirdos at the helm. And the degree to which you don’t have weirdos at the helm, creative minds at the helm and you’re a businessperson at the helm, get out of their way, right? You can’t, you cannot have the meetings like you’re describing. And I don’t just speak about this particular company. It’s just the entire industry. I just, there’s so much joy to be had if we keep creating great games, and I just hope we get to see those great games.
Jeff Kaplan
I think there’s a message to creative people out there and people who make stuff. We’re generally—we’re so focused on the love of the craft that we get lost in it and we love doing it and we’re not cutthroat and we don’t have that kind of ambition. We have a different kind of ambition. But there’s this whole world, especially as soon as you’re lucky enough to have success, that are very cutthroat and very ambitious. And for whatever reason, we keep giving ourselves to them, and we need to stop giving our so… World of Warcraft, when we made it, there was no CFO at Blizzard. You don’t need a CFO to make World of Warcraft. You need artists, engineers, designers, producers, and an audio team.
I think there’s a message to creative people out there and people who make stuff. We’re generally—we’re so focused on the love of the craft that we get lost in it and we love doing it and we’re not cutthroat and we don’t have that kind of ambition. We have a different kind of ambition. But there’s this whole world, especially as soon as you’re lucky enough to have success, that are very cutthroat and very ambitious. And for whatever reason, we keep giving ourselves to them, and we need to stop giving our so… World of Warcraft, when we made it, there was no CFO at Blizzard. You don’t need a CFO to make World of Warcraft. You need artists, engineers, designers, producers, and an audio team.
Lex Fridman
You don’t need to bring in… Just because you’re making a lot of money doesn’t mean you need to now start adulting by bringing in a CFO. You can figure it out.
You don’t need to bring in… Just because you’re making a lot of money doesn’t mean you need to now start adulting by bringing in a CFO. You can figure it out.
Jeff Kaplan
And there are great finance guys. Like I’ve worked with finance guys who get it and get out of the way and respect, and they’re gamers, and they sort of understand, but like, I wish developers would understand their own value more and stop handing the golden goose to people who don’t deserve it.
And there are great finance guys. Like I’ve worked with finance guys who get it and get out of the way and respect, and they’re gamers, and they sort of understand, but like, I wish developers would understand their own value more and stop handing the golden goose to people who don’t deserve it.
Lex Fridman
How painful was it to say goodbye?
How painful was it to say goodbye?
Jeff Kaplan
It broke me. I think after you’ve been at a place like Blizzard, which I love Blizzard. To this day, I have nothing but warm, fond memories. I mean, there’s those moments where you’re like, “I wish that hadn’t happened,” but on the whole, that place is mecca for game development, and everything I have is due to Blizzard. They provided for me and my family, made me the person I am, so separating from Blizzard was one of the most painful things. And I was very sad when I resigned, and I didn’t realize how broken I was until recently, like the mourning, grieving I had gone through of like… I think I’m a little fucked in the head for not being there any… How could I give that up? How could I not be there anymore? It was—it was really, really painful leaving.
It broke me. I think after you’ve been at a place like Blizzard, which I love Blizzard. To this day, I have nothing but warm, fond memories. I mean, there’s those moments where you’re like, “I wish that hadn’t happened,” but on the whole, that place is mecca for game development, and everything I have is due to Blizzard. They provided for me and my family, made me the person I am, so separating from Blizzard was one of the most painful things. And I was very sad when I resigned, and I didn’t realize how broken I was until recently, like the mourning, grieving I had gone through of like… I think I’m a little fucked in the head for not being there any… How could I give that up? How could I not be there anymore? It was—it was really, really painful leaving.
Lex Fridman
Can we just speak to, I don’t know, I don’t think we can give enough love to Blizzard. It’s a legendary company. For me personally, for everybody, for millions of people, created some of the greatest games ever, Warcraft, StarCraft Universe, Diablo, WoW, Overwatch. What made it such a legendary game company? Just looking back at the whole of it?
Can we just speak to, I don’t know, I don’t think we can give enough love to Blizzard. It’s a legendary company. For me personally, for everybody, for millions of people, created some of the greatest games ever, Warcraft, StarCraft Universe, Diablo, WoW, Overwatch. What made it such a legendary game company? Just looking back at the whole of it?
Jeff Kaplan
The start is Mike, Allen, and Frank. It was run by three gamers. They were, all three of them, programmers. They made the games before they just ran the company, so they knew what each of us as developers beneath them were going through, and they protected us. They shielded us from all of the nonsense, and even when they would align with a businessperson, they had a COO in the early days named Paul Sams, and Paul protected us.
The start is Mike, Allen, and Frank. It was run by three gamers. They were, all three of them, programmers. They made the games before they just ran the company, so they knew what each of us as developers beneath them were going through, and they protected us. They shielded us from all of the nonsense, and even when they would align with a businessperson, they had a COO in the early days named Paul Sams, and Paul protected us.
Jeff Kaplan
You know, they just, they found great people who got it. The company when I joined was, like, 95% developers and, like, 5% operations. It’s, when I left, it was, you know, 50/50, and that’s like a 4,500-person company. That love of the games and the respect and good treatment for game developers really turned it into the place that it was, just the commitment to excellence, the high-quality bar and then finding these passionate people like Chris Metzen or Sam Didier, they were, like, the visionaries of early Blizzard, Allen Adham, of just these worlds that we’re still making and we’re still playing in today. It was infectious and it was inspirational, and you wore the Blizzard blue with an esprit de corps.
You know, they just, they found great people who got it. The company when I joined was, like, 95% developers and, like, 5% operations. It’s, when I left, it was, you know, 50/50, and that’s like a 4,500-person company. That love of the games and the respect and good treatment for game developers really turned it into the place that it was, just the commitment to excellence, the high-quality bar and then finding these passionate people like Chris Metzen or Sam Didier, they were, like, the visionaries of early Blizzard, Allen Adham, of just these worlds that we’re still making and we’re still playing in today. It was infectious and it was inspirational, and you wore the Blizzard blue with an esprit de corps.
Jeff Kaplan
Like, you felt proud to be part of it and you felt like you had made it to be there, and everything you did, you did wanting to respect and honor those who had come before you. I know that sounds almost cheesy saying it that way, but it really had that sense of reverence, like you knew you were part of something special. You didn’t take it for granted.
Like, you felt proud to be part of it and you felt like you had made it to be there, and everything you did, you did wanting to respect and honor those who had come before you. I know that sounds almost cheesy saying it that way, but it really had that sense of reverence, like you knew you were part of something special. You didn’t take it for granted.
Lex Fridman
Yeah. That’s the sense. Reading everything, that’s the sense I got. Everybody there was a part of it that truly, truly, truly honored that time. Just to, just to take a small slice, what were some of the brain… So you mentioned Chris Metzen. You gave so much love to so many people on the team, but I gotta ask about Chris Metzen, who I would, by the way, love to do a podcast with at some point. What were the brainstorming sessions with him like? It seems like those are pretty awesome.
Yeah. That’s the sense. Reading everything, that’s the sense I got. Everybody there was a part of it that truly, truly, truly honored that time. Just to, just to take a small slice, what were some of the brain… So you mentioned Chris Metzen. You gave so much love to so many people on the team, but I gotta ask about Chris Metzen, who I would, by the way, love to do a podcast with at some point. What were the brainstorming sessions with him like? It seems like those are pretty awesome.
Jeff Kaplan
They were the best. Like, you could walk into a room. Like, the way I would work with Chris is early on when I was more junior, it was just sort of getting creative direction from him. “Hey, Chris, I’m about to work on this zone called Westfall. What are your ideas? You know, how could I capture them in gameplay? Well, that won’t quite work. How about like this?” It was more like that. Later on, like, I, I still remember the first discussion I ever had with Chris about Wrath of the Lich King, I went up to his office like, “Hey, we’re, we’re finally doing it. We’re doing the Northrend expansion. You know, what excites you about Northrend?” And that’s all you had to say. And he would draw a map and he’d start pulling up old, like, Warcraft II and Warcraft I manuals- …
They were the best. Like, you could walk into a room. Like, the way I would work with Chris is early on when I was more junior, it was just sort of getting creative direction from him. “Hey, Chris, I’m about to work on this zone called Westfall. What are your ideas? You know, how could I capture them in gameplay? Well, that won’t quite work. How about like this?” It was more like that. Later on, like, I, I still remember the first discussion I ever had with Chris about Wrath of the Lich King, I went up to his office like, “Hey, we’re, we’re finally doing it. We’re doing the Northrend expansion. You know, what excites you about Northrend?” And that’s all you had to say. And he would draw a map and he’d start pulling up old, like, Warcraft II and Warcraft I manuals- …
Jeff Kaplan
and, you know, showing you, like, pictures he and Sammy had drawn and, like, maps and, and he, all of it, he would just go on for an hour and then I would sort of digest. I’d just listen, taking constant notes. I’m photographing his whiteboards all the time, and then I go back and start to put those into design flow of, like, “Okay. What, what’s a zone? What’s a dungeon? What could be cool? What should come first? What should come last?” You know, Lich King, for example, we wanted to try a very specific design to counter a problem we had in Burning Crusade, which is everybody entered through the Dark Portal through Hellfire Peninsula, all the server programmers hate you because everybody loads into the same zone at the same time. Lich King, we split them up for better player flow.
and, you know, showing you, like, pictures he and Sammy had drawn and, like, maps and, and he, all of it, he would just go on for an hour and then I would sort of digest. I’d just listen, taking constant notes. I’m photographing his whiteboards all the time, and then I go back and start to put those into design flow of, like, “Okay. What, what’s a zone? What’s a dungeon? What could be cool? What should come first? What should come last?” You know, Lich King, for example, we wanted to try a very specific design to counter a problem we had in Burning Crusade, which is everybody entered through the Dark Portal through Hellfire Peninsula, all the server programmers hate you because everybody loads into the same zone at the same time. Lich King, we split them up for better player flow.
Jeff Kaplan
Plus, it’s more interesting the more choice you have. You know, Sid Meier says, “Games are a series of interesting choices,” so we give them two starting zones, but that was the flow with Chris. And so often we would just, like, okay, in that first meeting, Chris had put a zone called Grizzly Hills on the board. Well, I don’t know anything about Grizzly Hills. “Hey, Chris? Talk about Grizzly Hills.” If you didn’t interrupt him, he’d just go for an hour. And you have no idea how much of it, like, he had pre-thought about or had existed in previous lore and how much of it he was just making up on the spot. He’s just that charismatic and captivating.
Plus, it’s more interesting the more choice you have. You know, Sid Meier says, “Games are a series of interesting choices,” so we give them two starting zones, but that was the flow with Chris. And so often we would just, like, okay, in that first meeting, Chris had put a zone called Grizzly Hills on the board. Well, I don’t know anything about Grizzly Hills. “Hey, Chris? Talk about Grizzly Hills.” If you didn’t interrupt him, he’d just go for an hour. And you have no idea how much of it, like, he had pre-thought about or had existed in previous lore and how much of it he was just making up on the spot. He’s just that charismatic and captivating.
Lex Fridman
Creating these worlds and being able to- … brainstorm through them and together, I mean, that is what you’re doing. As a consumer of those worlds, you kind of take it for granted that they’re incredible, but, like, you’re crafting them. Like, you’re looking at a blank sheet of paper and then together coming up…
Creating these worlds and being able to- … brainstorm through them and together, I mean, that is what you’re doing. As a consumer of those worlds, you kind of take it for granted that they’re incredible, but, like, you’re crafting them. Like, you’re looking at a blank sheet of paper and then together coming up…
Jeff Kaplan
My job, as I saw it working with Chris, was I had to on World of Warcraft specifically working with Chris, is I was like the translator into gameplay of what Chris wanted, how to get it to play like how Chris wanted. So my favorite story is we’re working on Burning Crusade and we’re in this meeting and Chris is like… He’s the gentlest, sweetest guy, but because he carries himself with such confidence and everybody’s in awe of him, the junior developers get kind of intimidated by him. So we’re in this meeting and we’re talking about Silvermoon City because we’re introducing the Blood Elves, and Chris is like, “And Silvermoon City’s got the tallest fucking tower in all of Azeroth. I mean, it is the tallest thing. You know, it’s mind-blowing, the awe of it.”
My job, as I saw it working with Chris, was I had to on World of Warcraft specifically working with Chris, is I was like the translator into gameplay of what Chris wanted, how to get it to play like how Chris wanted. So my favorite story is we’re working on Burning Crusade and we’re in this meeting and Chris is like… He’s the gentlest, sweetest guy, but because he carries himself with such confidence and everybody’s in awe of him, the junior developers get kind of intimidated by him. So we’re in this meeting and we’re talking about Silvermoon City because we’re introducing the Blood Elves, and Chris is like, “And Silvermoon City’s got the tallest fucking tower in all of Azeroth. I mean, it is the tallest thing. You know, it’s mind-blowing, the awe of it.”
Jeff Kaplan
Only the blood elves could build it.” Fast-forward like two weeks later. I’m walking through the hall and I see a bunch of level designers and artists are all like crowded around the screen, and on the screen they’ve dragged Blackrock Mountain and Karazhan and the Stormwind Cathedral. I’m like, “What the fuck are you guys doing?” And they’re like, “Well, Chris said that the Silvermoon Tower had to be the tallest thing in World of Warcraft-” “… and so we’re measuring how tall all of these other things are so we can make the tower taller.” And I’m like, “Guys, Chris doesn’t know how tall the Burning Steppes, you know- … and the cathedral in Stormwind- … is. What Chris means is just make the tower really fucking tall.”
Only the blood elves could build it.” Fast-forward like two weeks later. I’m walking through the hall and I see a bunch of level designers and artists are all like crowded around the screen, and on the screen they’ve dragged Blackrock Mountain and Karazhan and the Stormwind Cathedral. I’m like, “What the fuck are you guys doing?” And they’re like, “Well, Chris said that the Silvermoon Tower had to be the tallest thing in World of Warcraft-” “… and so we’re measuring how tall all of these other things are so we can make the tower taller.” And I’m like, “Guys, Chris doesn’t know how tall the Burning Steppes, you know- … and the cathedral in Stormwind- … is. What Chris means is just make the tower really fucking tall.”
Jeff Kaplan
“You don’t need to measure it.” And they’re, “Oh, okay. That’s okay?” Like, “Are you willing to take the heat if he—” I’m like, “I’m willing to take the heat on this one, guys.”
“You don’t need to measure it.” And they’re, “Oh, okay. That’s okay?” Like, “Are you willing to take the heat if he—” I’m like, “I’m willing to take the heat on this one, guys.”
Lex Fridman
Yeah. It’s just a feeling. It’s a vibe. It’s-
Yeah. It’s just a feeling. It’s a vibe. It’s-
Jeff Kaplan
It’s a vibe.
It’s a vibe.
Diablo IV
Lex Fridman
Yeah. And I also just personally have to give all the love in the world for the current Diablo IV team, because I’ve spent, most recently out of the Blizzard games, I’ve spent a huge amount of time in Diablo, and they’ve created some… And it’s not just the loot, all right? It’s the, the whole experience, the art, everything together. And the seasons they’ve created, they’ve created a really wonderful world. So I can, I could see, I could feel how much effort goes into that.
Yeah. And I also just personally have to give all the love in the world for the current Diablo IV team, because I’ve spent, most recently out of the Blizzard games, I’ve spent a huge amount of time in Diablo, and they’ve created some… And it’s not just the loot, all right? It’s the, the whole experience, the art, everything together. And the seasons they’ve created, they’ve created a really wonderful world. So I can, I could see, I could feel how much effort goes into that.
Jeff Kaplan
They’re crushing it. And I think Diablo IV in like modern times is one of the best worlds that they’ve built. And they know, they understand Diablo players. Like that community is so hard and so demanding, and that team is amazing.
They’re crushing it. And I think Diablo IV in like modern times is one of the best worlds that they’ve built. And they know, they understand Diablo players. Like that community is so hard and so demanding, and that team is amazing.
Lex Fridman
Yeah, there’s a lot of richness. It’s like there’s this really… I mean, I don’t know how often you get that, but it’s really the perfect Diablo game. They’ve really like evolved a lot, grew a lot. So there’s this whole mathematical component of just so many numbers everywhere and it’s all balanced really masterfully. And then, of course, you have to come up with new content with the seasons and they figure out ways to do that, and at a crazy pace. And still make it super fun.
Yeah, there’s a lot of richness. It’s like there’s this really… I mean, I don’t know how often you get that, but it’s really the perfect Diablo game. They’ve really like evolved a lot, grew a lot. So there’s this whole mathematical component of just so many numbers everywhere and it’s all balanced really masterfully. And then, of course, you have to come up with new content with the seasons and they figure out ways to do that, and at a crazy pace. And still make it super fun.
Jeff Kaplan
They’re a great live team, yeah.
They’re a great live team, yeah.
Getting back to making video games
Lex Fridman
And for me personally, like I said, the co-op, the couch co-op experience has been really… like that aspect of it is really great, just all of it. It’s one of the greatest games in recent history. One of the things I wanted to mention, ’cause this is a powerful speech, is sort of instead of doing some kind of a corporate goodbye as you were leaving Blizzard, you allegedly shared with your team a video of David Bowie giving advice. And people should go watch this clip. But if I may read it, Bowie says, “Never play to the gallery.
And for me personally, like I said, the co-op, the couch co-op experience has been really… like that aspect of it is really great, just all of it. It’s one of the greatest games in recent history. One of the things I wanted to mention, ’cause this is a powerful speech, is sort of instead of doing some kind of a corporate goodbye as you were leaving Blizzard, you allegedly shared with your team a video of David Bowie giving advice. And people should go watch this clip. But if I may read it, Bowie says, “Never play to the gallery.
Lex Fridman
Always remember that the reason that you initially started working was that there was something inside yourself that you felt that if you could manifest in some way, you would understand more about yourself and how you co-exist with the rest of society. I think it’s terribly dangerous for an artist to fulfill other peoples’ expectations. I think they generally produce their worst work when they do that. And the other thing I would say is that if you feel safe in the area that you’re working in, you’re not working in the right area. Always go a little further into the water than you feel you’re capable of being in. Go a little bit out of your depth.
Always remember that the reason that you initially started working was that there was something inside yourself that you felt that if you could manifest in some way, you would understand more about yourself and how you co-exist with the rest of society. I think it’s terribly dangerous for an artist to fulfill other peoples’ expectations. I think they generally produce their worst work when they do that. And the other thing I would say is that if you feel safe in the area that you’re working in, you’re not working in the right area. Always go a little further into the water than you feel you’re capable of being in. Go a little bit out of your depth.
Lex Fridman
And when you don’t feel that your feet are quite touching the bottom, you’re just about in the right place to do something exciting.” Speaking of which, you are just about in a place to do something exciting. After leaving Blizzard you told me that you tried to take some time off. How did that work out for you?
And when you don’t feel that your feet are quite touching the bottom, you’re just about in the right place to do something exciting.” Speaking of which, you are just about in a place to do something exciting. After leaving Blizzard you told me that you tried to take some time off. How did that work out for you?
Jeff Kaplan
Not so well. My wife, who is wonderful, told me I needed to take at least a year off and just, you know, I’d been going really hard. I’d gone 19 years barely taking vacation and I let Blizzard consume me. And, you know, I was crushed by leaving because I loved the place, and I didn’t know what to do with myself. I was pulling weeds in the backyard.
Not so well. My wife, who is wonderful, told me I needed to take at least a year off and just, you know, I’d been going really hard. I’d gone 19 years barely taking vacation and I let Blizzard consume me. And, you know, I was crushed by leaving because I loved the place, and I didn’t know what to do with myself. I was pulling weeds in the backyard.
Lex Fridman
Literally. Gardening.
Literally. Gardening.
Jeff Kaplan
Yeah. Well, she won’t let me garden in the garden ’cause that’s hers- … but I’m allowed to pull the weeds. So I got very good at that. I was very proficient. And then of all things, I cracked out on Call of Duty: Black Ops Cold War and I unlocked Dark Matter Ultra, which I’d… that’s like a crazy achievement to do in that game.
Yeah. Well, she won’t let me garden in the garden ’cause that’s hers- … but I’m allowed to pull the weeds. So I got very good at that. I was very proficient. And then of all things, I cracked out on Call of Duty: Black Ops Cold War and I unlocked Dark Matter Ultra, which I’d… that’s like a crazy achievement to do in that game.
Jeff Kaplan
So I did that, and then I just, I couldn’t help it, like it’s how I’m programmed. It was like, at this point, it’s late spring, early summer and I’m just sitting in the backyard and I just started writing with Notepad about, “Here’s a game I want to make.” And it was so terrifying because for 19 years I had worked with the greatest developers, I thought, in the industry. And, you know, there’d be moments where it’s like, “Okay, I wanna do like a game world map.” Like, “Hey, Erin, you’re amazing at making game world maps. Like, you do that.” And you know, I, like, “I need some story hooks. Hey, Chris, what do you think would be cool here?”
So I did that, and then I just, I couldn’t help it, like it’s how I’m programmed. It was like, at this point, it’s late spring, early summer and I’m just sitting in the backyard and I just started writing with Notepad about, “Here’s a game I want to make.” And it was so terrifying because for 19 years I had worked with the greatest developers, I thought, in the industry. And, you know, there’d be moments where it’s like, “Okay, I wanna do like a game world map.” Like, “Hey, Erin, you’re amazing at making game world maps. Like, you do that.” And you know, I, like, “I need some story hooks. Hey, Chris, what do you think would be cool here?”
Jeff Kaplan
Like, you know, it’s so collaborative and I was surrounded by the best of the best, and there I was by myself. And I was out there again, and I loved it. It brought all the joy of game making. I thought games were no longer fun to make because it was only about business, and somebody’s asking me for unreasonable amounts of money and unreasonable amounts of time. And I had forgotten the pure joy of the craft of making games, and I was designing, I was going on, I was watching YouTube videos to learn Unreal and Adobe Illustrator and all these things to like help me make games, whatever, Blender. Um, I had no right to be doing any of that, and it just felt so amazing to do it. And I sort of realized, I came to two realizations. One, I never wanna work for someone else again.
Like, you know, it’s so collaborative and I was surrounded by the best of the best, and there I was by myself. And I was out there again, and I loved it. It brought all the joy of game making. I thought games were no longer fun to make because it was only about business, and somebody’s asking me for unreasonable amounts of money and unreasonable amounts of time. And I had forgotten the pure joy of the craft of making games, and I was designing, I was going on, I was watching YouTube videos to learn Unreal and Adobe Illustrator and all these things to like help me make games, whatever, Blender. Um, I had no right to be doing any of that, and it just felt so amazing to do it. And I sort of realized, I came to two realizations. One, I never wanna work for someone else again.
Jeff Kaplan
I never wanna create something and then have somebody take my baby away from me, you know? That’s really hard when that happens, and it’s sort of happened a few times now, you know, where you have to just let something go that you created. And I wanted it all to be focused on the craft of making games, the art, programming, design, audio, you know? Like, just not about the bullshit of the games industry. I’m not interested in the games industry. I’m not interested in the business of games. I’m not interested in the entertainment industry. It’s just game jamming, making stuff that we’re gonna play together. And around that time, my I call him my development soulmate. There’s a programmer named Tim Ford.
I never wanna create something and then have somebody take my baby away from me, you know? That’s really hard when that happens, and it’s sort of happened a few times now, you know, where you have to just let something go that you created. And I wanted it all to be focused on the craft of making games, the art, programming, design, audio, you know? Like, just not about the bullshit of the games industry. I’m not interested in the games industry. I’m not interested in the business of games. I’m not interested in the entertainment industry. It’s just game jamming, making stuff that we’re gonna play together. And around that time, my I call him my development soulmate. There’s a programmer named Tim Ford.
Jeff Kaplan
He reached out and he’s like, “Hey, man…” He was like an associate tech director on Overwatch at the time. And he’s like, “Yeah, I don’t think I can do this anymore. It’s just not like it was, you know, I just handed in my notice.” And I’m like, “Whoa, you know, well, if you wanna do something together, like fuck it. Let’s take a stab and, you know, just see what happens.” And Tim came over to my house, and well, before that, he says, “My last day’s on Friday.”
He reached out and he’s like, “Hey, man…” He was like an associate tech director on Overwatch at the time. And he’s like, “Yeah, I don’t think I can do this anymore. It’s just not like it was, you know, I just handed in my notice.” And I’m like, “Whoa, you know, well, if you wanna do something together, like fuck it. Let’s take a stab and, you know, just see what happens.” And Tim came over to my house, and well, before that, he says, “My last day’s on Friday.”
Jeff Kaplan
“And my exit interview’s at like 1:00. I’m gonna be over to your house at like 2:00 that afternoon.” And I’m like, “Well, don’t you think you should take some time off, Tim, you know, before whatever’s next for you? Take a month off, you know? Meg, his wife, will appreciate it, you know? Just go pull weeds in the garden for a while.” And he’s like, “I’m a programmer. All I’m gonna do is program for a month if I take a month off. I might as well start programming our game.” Which-
“And my exit interview’s at like 1:00. I’m gonna be over to your house at like 2:00 that afternoon.” And I’m like, “Well, don’t you think you should take some time off, Tim, you know, before whatever’s next for you? Take a month off, you know? Meg, his wife, will appreciate it, you know? Just go pull weeds in the garden for a while.” And he’s like, “I’m a programmer. All I’m gonna do is program for a month if I take a month off. I might as well start programming our game.” Which-
Lex Fridman
Brilliant
Brilliant
Jeff Kaplan
… it was so awesome when he said that.
… it was so awesome when he said that.
Lex Fridman
Brilliant.
Brilliant.
Jeff Kaplan
He came over and I pitched him this idea for a game, and I pitched him, “Let’s start a company.” And that was it. Like, that was the birth of us making a studio.
He came over and I pitched him this idea for a game, and I pitched him, “Let’s start a company.” And that was it. Like, that was the birth of us making a studio.
Lex Fridman
Now, meanwhile, as far as the outside world is concerned, you’ve disappeared off the face of the Earth, but you were actually working on a game.
Now, meanwhile, as far as the outside world is concerned, you’ve disappeared off the face of the Earth, but you were actually working on a game.
Jeff Kaplan
Yeah, I needed to be away from the world. I needed to not have… I wanted to not get attention from anyone. I needed to not read my name on Reddit or… you know, any internet site. I wanted to not come up, let some other Jeff Kaplan bubble to the top- … of the Google, you know, search list.
Yeah, I needed to be away from the world. I needed to not have… I wanted to not get attention from anyone. I needed to not read my name on Reddit or… you know, any internet site. I wanted to not come up, let some other Jeff Kaplan bubble to the top- … of the Google, you know, search list.
Lex Fridman
You know our man Dinoflask is gonna be all over this conversation, right?
You know our man Dinoflask is gonna be all over this conversation, right?
Jeff Kaplan
Oh, God, well, there’s, yeah, this one’s gonna set him back some time. But, yeah, I needed-
Oh, God, well, there’s, yeah, this one’s gonna set him back some time. But, yeah, I needed-
Lex Fridman
You know what to do.
You know what to do.
Jeff Kaplan
I needed for none of that to happen. I just needed to be able to, like, mourn the loss of Blizzard-
I needed for none of that to happen. I just needed to be able to, like, mourn the loss of Blizzard-
Jeff Kaplan
… and create on my own so it was great. And at that time, like as soon as it was announced that I was leaving Blizzard, I had like 60 people reach out to me. It was, this was April of 2021 and investment money was nuts. Both like the VC money and the strategic money was crazy, like the, especially the Chinese companies, because apparently they weren’t getting publishing numbers in China or something. The whole economy was crazy, and so just everybody was trying to throw money at me, which was a very good position to sort of be at to start a company. So what Tim and I did was say, “We’re not doing this for money, but here’s the game we wanna make, and it’s gonna take this many developers, and we think it’s gonna take this length of time, and that means the budget is this.
… and create on my own so it was great. And at that time, like as soon as it was announced that I was leaving Blizzard, I had like 60 people reach out to me. It was, this was April of 2021 and investment money was nuts. Both like the VC money and the strategic money was crazy, like the, especially the Chinese companies, because apparently they weren’t getting publishing numbers in China or something. The whole economy was crazy, and so just everybody was trying to throw money at me, which was a very good position to sort of be at to start a company. So what Tim and I did was say, “We’re not doing this for money, but here’s the game we wanna make, and it’s gonna take this many developers, and we think it’s gonna take this length of time, and that means the budget is this.
Jeff Kaplan
And we need, for any of these people who wanna invest in us, we gotta hit that number, but after that, we’re not gonna go for more money. It’s not an auction to raise as high as we can go. We’re gonna optimize for control.”
And we need, for any of these people who wanna invest in us, we gotta hit that number, but after that, we’re not gonna go for more money. It’s not an auction to raise as high as we can go. We’re gonna optimize for control.”
The Legend of California
Lex Fridman
I don’t know if this is something that you can talk about, but I got a chance to see the game for a few hours, and I have to say it’s incredible, Jeff. Like, it’s incredible. But I almost immediately fell in love with the world and everything I saw. See, I’m tempted to say some of the things I saw but it’s just an incredible game. So how much can you talk about it? Do you know what it’s going to be called? Can you talk about that? Do you know about the company? Are you allowed to say any of that?
I don’t know if this is something that you can talk about, but I got a chance to see the game for a few hours, and I have to say it’s incredible, Jeff. Like, it’s incredible. But I almost immediately fell in love with the world and everything I saw. See, I’m tempted to say some of the things I saw but it’s just an incredible game. So how much can you talk about it? Do you know what it’s going to be called? Can you talk about that? Do you know about the company? Are you allowed to say any of that?
Jeff Kaplan
Sure. The most unconventional way to talk about this stuff for the first time. So, our company name is Kintsugiyama, which most people will struggle to pronounce.
Sure. The most unconventional way to talk about this stuff for the first time. So, our company name is Kintsugiyama, which most people will struggle to pronounce.
Lex Fridman
Nice.
Nice.
Jeff Kaplan
And the company name has a deep meaning to me, which I’m happy to explain later if you’re interested. And the game name that we’re working on, it’s called The Legend of California, and it’s an open world game. People are gonna call it a survival crafting game. People like to compartmentalize these. I think it’s an action game. It’s a game that takes place on a mythical island of California.
And the company name has a deep meaning to me, which I’m happy to explain later if you’re interested. And the game name that we’re working on, it’s called The Legend of California, and it’s an open world game. People are gonna call it a survival crafting game. People like to compartmentalize these. I think it’s an action game. It’s a game that takes place on a mythical island of California.
Lex Fridman
Mm-hmm. In the 1800s.
Mm-hmm. In the 1800s.
Jeff Kaplan
In the gold rush. If you’re trying to-
In the gold rush. If you’re trying to-
Lex Fridman
In the gold rush
In the gold rush
Jeff Kaplan
… if you’re trying to nail the most important time in California history, it’s gotta be that gold rush.
… if you’re trying to nail the most important time in California history, it’s gotta be that gold rush.
Lex Fridman
So, it’s this beautiful, almost ultra-realistic version of California, but it’s in an alternate history, alternate version of California-
So, it’s this beautiful, almost ultra-realistic version of California, but it’s in an alternate history, alternate version of California-
Jeff Kaplan
Yes
Yes
Lex Fridman
… where it’s an island, almost like an Atlantis type of ethereal island, but still very realistic to what the California terrain is- … and that time period. So it’s this weird, like, amalgamation of this ultra-realistic and the surreal.
… where it’s an island, almost like an Atlantis type of ethereal island, but still very realistic to what the California terrain is- … and that time period. So it’s this weird, like, amalgamation of this ultra-realistic and the surreal.
Jeff Kaplan
The theme of the game is very weird. We’re not trying to make a historical game. There’s no historical accuracy to this. In fact, the island when first discovered is uninhabited. That’s already not true. As we know, there were lots of people in California. It’s an island, which we know is not true. We want it to feel authentic to that time period because we think that time period is cool. Prospectors, you know, cowboys. Like, it’s a really fun thing for us to explore, all of those themes—people in mines. We wanna build mines and we just wanna create a world that you can live in. I love creating worlds. Everything that I’ve worked on before, from World of Warcraft to Overwatch, it’s always been, how do you create this place for players to escape to? So.
The theme of the game is very weird. We’re not trying to make a historical game. There’s no historical accuracy to this. In fact, the island when first discovered is uninhabited. That’s already not true. As we know, there were lots of people in California. It’s an island, which we know is not true. We want it to feel authentic to that time period because we think that time period is cool. Prospectors, you know, cowboys. Like, it’s a really fun thing for us to explore, all of those themes—people in mines. We wanna build mines and we just wanna create a world that you can live in. I love creating worlds. Everything that I’ve worked on before, from World of Warcraft to Overwatch, it’s always been, how do you create this place for players to escape to? So.
Lex Fridman
So, it’s an online, multiplayer game. I should say the experience of it is just gorgeous, and then the music is wonderful.
So, it’s an online, multiplayer game. I should say the experience of it is just gorgeous, and then the music is wonderful.
Jeff Kaplan
I’m glad you like it.
I’m glad you like it.
Lex Fridman
And one of my favorite things is just going down to the mine and digging. I mean, that’s done extremely well. And as you described, the whole world is voxels, so it’s generated. Can you explain how that works?
And one of my favorite things is just going down to the mine and digging. I mean, that’s done extremely well. And as you described, the whole world is voxels, so it’s generated. Can you explain how that works?
Jeff Kaplan
Yeah. As a world, we handcrafted the world, so like the shape of California is always the familiar shape of California, except it’s an island. So, you know, there’s no Nevada on the eastern side. We handcrafted all of that. It looks gorgeous and places like Yosemite are where you would expect Yosemite to be. And so all of those familiar landmarks are there, but then we have like dozens of points of interest, and those move around the map in, depending on the map seed. And the map is also tiered in terms of difficulty. We don’t really have levels in this game. We have tiers, and there’s only four tiers right now. Maybe, maybe that will change. But the way that the map tiers itself each time changes with every world seed. So not only…
Yeah. As a world, we handcrafted the world, so like the shape of California is always the familiar shape of California, except it’s an island. So, you know, there’s no Nevada on the eastern side. We handcrafted all of that. It looks gorgeous and places like Yosemite are where you would expect Yosemite to be. And so all of those familiar landmarks are there, but then we have like dozens of points of interest, and those move around the map in, depending on the map seed. And the map is also tiered in terms of difficulty. We don’t really have levels in this game. We have tiers, and there’s only four tiers right now. Maybe, maybe that will change. But the way that the map tiers itself each time changes with every world seed. So not only…
Jeff Kaplan
Any server that you join will have a different seed in terms of how the tiers play out. So, Mojave might be the easiest newbie area on your server, but on my server it’s an endgame, tier four area. But all of our notable points of interest also move around. So, we have a really amazing point of interest that we call Dread Rock that’s inspired by Alcatraz. And like, sure, sometimes it’s in San Francisco, but sometimes it can be sitting in the middle of the Mojave Desert also.
Any server that you join will have a different seed in terms of how the tiers play out. So, Mojave might be the easiest newbie area on your server, but on my server it’s an endgame, tier four area. But all of our notable points of interest also move around. So, we have a really amazing point of interest that we call Dread Rock that’s inspired by Alcatraz. And like, sure, sometimes it’s in San Francisco, but sometimes it can be sitting in the middle of the Mojave Desert also.
Lex Fridman
Mm-hmm. It integrates it into the environment, to where it makes sense- … to be in that environment. And like you said, so much of what makes a world is sound and lighting. And that, that’s definitely a thing that I’ve noticed. I mean, it’s probably the most beautiful sunset and sunrise I’ve seen in a game.
Mm-hmm. It integrates it into the environment, to where it makes sense- … to be in that environment. And like you said, so much of what makes a world is sound and lighting. And that, that’s definitely a thing that I’ve noticed. I mean, it’s probably the most beautiful sunset and sunrise I’ve seen in a game.
Jeff Kaplan
We have a great lighting artist who’s this amazing guy named Mike Marra, and some of the inspiration for the game like… There’s a lot of inspirations for this game, but there’s a painter named Albert Bierstadt, who I discovered while researching California, and he painted these just epic landscape pieces of, you know, Yosemite and a lot of other, the gorgeous parts of-
We have a great lighting artist who’s this amazing guy named Mike Marra, and some of the inspiration for the game like… There’s a lot of inspirations for this game, but there’s a painter named Albert Bierstadt, who I discovered while researching California, and he painted these just epic landscape pieces of, you know, Yosemite and a lot of other, the gorgeous parts of-
Lex Fridman
Yeah, we’re looking at one, one photo of his.
Yeah, we’re looking at one, one photo of his.
Jeff Kaplan
Yeah, it’s just amazing, and his paintings were huge, too. I’d love to see one in person.
Yeah, it’s just amazing, and his paintings were huge, too. I’d love to see one in person.
Lex Fridman
And so you see a painting like that and you’re saying, “We wanna create that world.”
And so you see a painting like that and you’re saying, “We wanna create that world.”
Jeff Kaplan
Yeah. I mean, when I see that painting, this is, this is what video games brings to the table. So, every art form that evolves after another gets to incorporate previous art forms.
Yeah. I mean, when I see that painting, this is, this is what video games brings to the table. So, every art form that evolves after another gets to incorporate previous art forms.
Jeff Kaplan
Movies got to take sound and, you know, fine art. We get to take everything, including movies. So, you know, it’s, it’s Katamari Damacy, the art form. But like… I see a Bierstadt painting, and I wanna walk around that world. I wanna see what’s around the corner. And our lighting artist, Mike, he, you know, he sees these pictures, and he’s like, “Okay. Yeah. Hold my beer.” Like, “I’ll make it look like that.” And he, and he… We are all blown away by the, like, how much impact just the lighting has. And I’m not an artist, so I don’t think about things like the color theory, the lights, the clouds, what all of that’s bringing to this. I just know I want to live in that world, and these are the types of worlds that we want to make.
Movies got to take sound and, you know, fine art. We get to take everything, including movies. So, you know, it’s, it’s Katamari Damacy, the art form. But like… I see a Bierstadt painting, and I wanna walk around that world. I wanna see what’s around the corner. And our lighting artist, Mike, he, you know, he sees these pictures, and he’s like, “Okay. Yeah. Hold my beer.” Like, “I’ll make it look like that.” And he, and he… We are all blown away by the, like, how much impact just the lighting has. And I’m not an artist, so I don’t think about things like the color theory, the lights, the clouds, what all of that’s bringing to this. I just know I want to live in that world, and these are the types of worlds that we want to make.
Lex Fridman
So, what do you want the tone of the game to be, the feeling of the game?
So, what do you want the tone of the game to be, the feeling of the game?
Jeff Kaplan
This is really different. It’s been hard for people. When people were talking to us about, you know, they know me and Tim, and they’re, “Oh, the Blizzard guys, the Overwatch guys. You’re making, like, a bright, aspirational future team-based hero shooter, right?” And I’m like, “Why would I want to do that?” I felt like, first of all, respects to Blizzard, and I don’t want to try to crib Blizzard and make a pseudo-Blizzard game, you know? This is… I want to make a Kintsugiyama game, you know? Me and Tim and this crack team, you know, we’re only 34 people. We want to define what a Kintsugiyama game is, and this world seemed so inspiring to us, you know? The setting is really interesting. You know, I think California can be a game world.
This is really different. It’s been hard for people. When people were talking to us about, you know, they know me and Tim, and they’re, “Oh, the Blizzard guys, the Overwatch guys. You’re making, like, a bright, aspirational future team-based hero shooter, right?” And I’m like, “Why would I want to do that?” I felt like, first of all, respects to Blizzard, and I don’t want to try to crib Blizzard and make a pseudo-Blizzard game, you know? This is… I want to make a Kintsugiyama game, you know? Me and Tim and this crack team, you know, we’re only 34 people. We want to define what a Kintsugiyama game is, and this world seemed so inspiring to us, you know? The setting is really interesting. You know, I think California can be a game world.
Jeff Kaplan
I think we can make it beautiful and interesting. We don’t have to follow history or geography. We can kind of do a spin where, you know, it feels authentic. We can have guns that feel like they’re kind of from that time period, but we’re not spaceships and aliens and steampunk. That’s what we would have done at Blizzard. We’re gonna be a little different here. So, the tone of this game, you know… Metzen would describe Blizzard as the hero factory. You know, we make… And what he means by that is not only are we making heroes, but we make the players into heroes.
I think we can make it beautiful and interesting. We don’t have to follow history or geography. We can kind of do a spin where, you know, it feels authentic. We can have guns that feel like they’re kind of from that time period, but we’re not spaceships and aliens and steampunk. That’s what we would have done at Blizzard. We’re gonna be a little different here. So, the tone of this game, you know… Metzen would describe Blizzard as the hero factory. You know, we make… And what he means by that is not only are we making heroes, but we make the players into heroes.
Jeff Kaplan
This game is gonna have an edgier tone. You’re gonna enter this world. It’s gonna feel lonelier. It’s gonna feel mysterious, larger than you. You’re gonna feel small until you earn the right to feel big. It’s gonna feel really dangerous. You’re gonna want to see what’s over that next hill, but if the sun is setting, like, get to shelter. Can’t wait to get back to my ranch and put my cozy fireplace on and wait till morning, you know? We want more of that vibe.
This game is gonna have an edgier tone. You’re gonna enter this world. It’s gonna feel lonelier. It’s gonna feel mysterious, larger than you. You’re gonna feel small until you earn the right to feel big. It’s gonna feel really dangerous. You’re gonna want to see what’s over that next hill, but if the sun is setting, like, get to shelter. Can’t wait to get back to my ranch and put my cozy fireplace on and wait till morning, you know? We want more of that vibe.
Lex Fridman
It’s more solitary, almost scary but beautiful. That mix, that tension. I hate to ask this question, but given our previous discussion about a timeline slide, what do you think a timeline looks like? When do you think it’s possible for somebody in the world to be able to play this game?
It’s more solitary, almost scary but beautiful. That mix, that tension. I hate to ask this question, but given our previous discussion about a timeline slide, what do you think a timeline looks like? When do you think it’s possible for somebody in the world to be able to play this game?
Jeff Kaplan
So, this is the beauty of me and Tim kind of getting to run the show and why we’re excited about it. We can kinda do whatever we want- … within reason. So we’re just gonna kinda quietly put it up on Steam and see what happens.
So, this is the beauty of me and Tim kind of getting to run the show and why we’re excited about it. We can kinda do whatever we want- … within reason. So we’re just gonna kinda quietly put it up on Steam and see what happens.
Lex Fridman
Nice.
Nice.
Jeff Kaplan
You know, no, like, big corporate marketing group would ever think to do that in a million years- … without, like, some, you know, $10 million announce or whatever. We’ll just kinda put it on Steam and be cool if people wishlisted it. There’s my plug. And then I think we are shooting to have some sort of public-ish alpha in March. And then our plan, and something I’m really excited about, ’cause I’ve never gotten to do this before, we wanna put the game in early access. Some people hate early access and won’t touch it, and I understand it, and then some people are like, “I wanna be in on the ground floor and see the thing from day one and watch it evolve.” So, we’ll put it into early access, and we’ll just run that until who knows, you know?
You know, no, like, big corporate marketing group would ever think to do that in a million years- … without, like, some, you know, $10 million announce or whatever. We’ll just kinda put it on Steam and be cool if people wishlisted it. There’s my plug. And then I think we are shooting to have some sort of public-ish alpha in March. And then our plan, and something I’m really excited about, ’cause I’ve never gotten to do this before, we wanna put the game in early access. Some people hate early access and won’t touch it, and I understand it, and then some people are like, “I wanna be in on the ground floor and see the thing from day one and watch it evolve.” So, we’ll put it into early access, and we’ll just run that until who knows, you know?
Lex Fridman
Is it scary to you to have a sort of game with some rough edges out there in the wild where people are interacting with it through the alpha- … through the beta?
Is it scary to you to have a sort of game with some rough edges out there in the wild where people are interacting with it through the alpha- … through the beta?
Jeff Kaplan
Yes, and this game has more rough edges, like, the most rough edges we would have at Blizzard is, like, showing it at BlizzCon, which was heavily polished and controlled. This is gonna be more, you know, in development than anything else I’ve ever worked on. But that’s-
Yes, and this game has more rough edges, like, the most rough edges we would have at Blizzard is, like, showing it at BlizzCon, which was heavily polished and controlled. This is gonna be more, you know, in development than anything else I’ve ever worked on. But that’s-
Lex Fridman
I love it.
I love it.
Jeff Kaplan
… part of the excitement too, you know? It’s kind of like this is how the sausage gets made. I mean, you’re gonna see it front row.
… part of the excitement too, you know? It’s kind of like this is how the sausage gets made. I mean, you’re gonna see it front row.
Lex Fridman
I’m gonna try to get myself into the alpha somehow. Anybody who is listening to this, I highly recommend this game. You will not be disappointed. The world itself is just beautiful. So, whoever’s behind it, you and Tim and the team, are just doing an incredible job. And thank you for putting out rough versions of it so we get to-
I’m gonna try to get myself into the alpha somehow. Anybody who is listening to this, I highly recommend this game. You will not be disappointed. The world itself is just beautiful. So, whoever’s behind it, you and Tim and the team, are just doing an incredible job. And thank you for putting out rough versions of it so we get to-
Jeff Kaplan
Yeah. Of course.
Yeah. Of course.
Lex Fridman
… not wait forever for the perfect thing. And because you feel in… You feel like you’re a part of it if you get the imperfect thing. I’m one of the people who like the imperfect. We get to see the rough versions develop, and get to be a part of it developing. I saw the logo. It’s a mountain. Can you explain the meaning behind the name?
… not wait forever for the perfect thing. And because you feel in… You feel like you’re a part of it if you get the imperfect thing. I’m one of the people who like the imperfect. We get to see the rough versions develop, and get to be a part of it developing. I saw the logo. It’s a mountain. Can you explain the meaning behind the name?
Jeff Kaplan
So, Kintsugi is a Japanese craft of repairing broken pottery. So there’s a lot of philosophy that goes into it as well. And you know, I wanna do a good job of explaining it, but basically, like, you take a broken piece of pottery, and then they would use golden joinery-
So, Kintsugi is a Japanese craft of repairing broken pottery. So there’s a lot of philosophy that goes into it as well. And you know, I wanna do a good job of explaining it, but basically, like, you take a broken piece of pottery, and then they would use golden joinery-
Jeff Kaplan
Like golden lacquer to put the piece back together. And the thought was rather than hiding the scars, you make them more beautiful. And the philosophical parts that sort of appealed to me with that is there’s a lot of me and Tim in that, of… We’re so appreciative for our time at Blizzard, but we didn’t come away unscarred. And there’s also a philosophy in Kintsugi that nothing’s ever perfect, and the pursuit of perfection is actually a mistake, and that there’s beauty in imperfection. And so I relate that to myself personally. That’s how I feel in an aspirational way. I’m not saying I’ve achieved it, but in an aspirational way, I want to be that way. And I think it’s also an analogy for the making of games. Like, it’s a…
Like golden lacquer to put the piece back together. And the thought was rather than hiding the scars, you make them more beautiful. And the philosophical parts that sort of appealed to me with that is there’s a lot of me and Tim in that, of… We’re so appreciative for our time at Blizzard, but we didn’t come away unscarred. And there’s also a philosophy in Kintsugi that nothing’s ever perfect, and the pursuit of perfection is actually a mistake, and that there’s beauty in imperfection. And so I relate that to myself personally. That’s how I feel in an aspirational way. I’m not saying I’ve achieved it, but in an aspirational way, I want to be that way. And I think it’s also an analogy for the making of games. Like, it’s a…
Jeff Kaplan
Making of games is a constant pursuit of imperfection. A game is never gonna be perfect. Just ask the players. They’re very vocal about it. And seeing the beauty and the imperfections and the strength in something that’s been broken that can be stronger.
Making of games is a constant pursuit of imperfection. A game is never gonna be perfect. Just ask the players. They’re very vocal about it. And seeing the beauty and the imperfections and the strength in something that’s been broken that can be stronger.
Greatest video game of all time
Lex Fridman
You had a heck of a difficult couple years here. And so in some sense, it represents that beauty in imperfection. So everybody listening to this I hope, I hope you do have it out on, on Steam. Go check out Legend of California. Truly a beautiful world. I’m so glad you are actually creating this, low-key, quietly creating this beautiful, incredible world. Ridiculous question, but can we talk about some of the greatest games of all time?
You had a heck of a difficult couple years here. And so in some sense, it represents that beauty in imperfection. So everybody listening to this I hope, I hope you do have it out on, on Steam. Go check out Legend of California. Truly a beautiful world. I’m so glad you are actually creating this, low-key, quietly creating this beautiful, incredible world. Ridiculous question, but can we talk about some of the greatest games of all time?
Jeff Kaplan
Yes.
Yes.
Lex Fridman
What… I mean, I know this is a bit of a nerding-out kind of thing, and outside of the games you’ve been part of creating, I think Blizzard has created some of the greatest games of all time. Outside of those, what do you think are in the list?
What… I mean, I know this is a bit of a nerding-out kind of thing, and outside of the games you’ve been part of creating, I think Blizzard has created some of the greatest games of all time. Outside of those, what do you think are in the list?
Jeff Kaplan
So there’s one that’s the best. It’s Legend of Zelda: Breath of the Wild. And then there’s this list of greatest games: Zork, Ultima,
So there’s one that’s the best. It’s Legend of Zelda: Breath of the Wild. And then there’s this list of greatest games: Zork, Ultima,
Lex Fridman
So Breath of the Wild is, is the best, yeah?
So Breath of the Wild is, is the best, yeah?
Jeff Kaplan
The greatest game ever made.
The greatest game ever made.
Lex Fridman
What makes it the greatest game ever made for you?
What makes it the greatest game ever made for you?
Jeff Kaplan
Every aspect is so thoughtful, so well designed. The art matches the design and the tech, and even integrating with the Switch in the way it does. How do you keep making Zelda better? How can Legend of Zelda: Ocarina of Time exist and somebody make an even better Zelda game? The way you can chop down a tree and float in a river, and, like, the world is a toy and everything works as you wished and hoped it would work. And there’s a narrative aspect to it, and there’s really fun combat and action and itemization. There’s so many things that that game gets right that other games are lucky if they get one of those things right, and are… become best in their genre just for getting that one thing right. And Breath of the Wild does them all right and the best.
Every aspect is so thoughtful, so well designed. The art matches the design and the tech, and even integrating with the Switch in the way it does. How do you keep making Zelda better? How can Legend of Zelda: Ocarina of Time exist and somebody make an even better Zelda game? The way you can chop down a tree and float in a river, and, like, the world is a toy and everything works as you wished and hoped it would work. And there’s a narrative aspect to it, and there’s really fun combat and action and itemization. There’s so many things that that game gets right that other games are lucky if they get one of those things right, and are… become best in their genre just for getting that one thing right. And Breath of the Wild does them all right and the best.
Lex Fridman
There’s a certain kinda lightness to the way the world feels, the openness of the world feels. That’s unlike any other game, right? That’s uniquely that company, uniquely that-
There’s a certain kinda lightness to the way the world feels, the openness of the world feels. That’s unlike any other game, right? That’s uniquely that company, uniquely that-
Jeff Kaplan
Yeah. No one else-
Yeah. No one else-
Lex Fridman
Because nobody else creates that. You’re right. Under the pressure of having created a bunch of Zeldas that are, like, really great games, to be able to deliver once again.
Because nobody else creates that. You’re right. Under the pressure of having created a bunch of Zeldas that are, like, really great games, to be able to deliver once again.
Jeff Kaplan
Nintendo is, like, the Mecca. Like, they’re the best, you know? That’s all there is to it.
Nintendo is, like, the Mecca. Like, they’re the best, you know? That’s all there is to it.
Lex Fridman
Do you understand how that company works?
Do you understand how that company works?
Jeff Kaplan
No.
No.
Lex Fridman
That they’re not…
That they’re not…
Jeff Kaplan
I don’t at all.
I don’t at all.
Lex Fridman
Like, because, I mean, they’ve been around for a long time and still to be able to deliver.
Like, because, I mean, they’ve been around for a long time and still to be able to deliver.
Jeff Kaplan
I kind of rationally or irrationally just worship. It’s just sort of: if it’s from Nintendo, it’s gonna be great.
I kind of rationally or irrationally just worship. It’s just sort of: if it’s from Nintendo, it’s gonna be great.
Jeff Kaplan
And even if my first impression is like, “Wow, they’re doing what weird thing with the controller this time,” and then you get your hands on it and you’re like, “God.” My son and I, we both played Legend of Zelda: Breath of the Wild, and he makes games also. And we had this moment where he’s like, “I’m so sad after I played it.” And he’s like, “I know I’ll never make anything like this.” And it’s that weird, like, you honor it so much and think it’s so great. Red Dead was like that for me. Red Dead Redemption 2 is… that’s a game I put on a shrine. Not just how brilliant the game itself is, but as a game maker, as a craftsperson who makes games, how the hell do you make that? Like, only Rockstar with all the years of making those types of games. No one else can come in entry level-
And even if my first impression is like, “Wow, they’re doing what weird thing with the controller this time,” and then you get your hands on it and you’re like, “God.” My son and I, we both played Legend of Zelda: Breath of the Wild, and he makes games also. And we had this moment where he’s like, “I’m so sad after I played it.” And he’s like, “I know I’ll never make anything like this.” And it’s that weird, like, you honor it so much and think it’s so great. Red Dead was like that for me. Red Dead Redemption 2 is… that’s a game I put on a shrine. Not just how brilliant the game itself is, but as a game maker, as a craftsperson who makes games, how the hell do you make that? Like, only Rockstar with all the years of making those types of games. No one else can come in entry level-
Jeff Kaplan
… and compete with that. So that’s-
… and compete with that. So that’s-
Lex Fridman
Purely single player, narrative driven. So you also respect that kind of, like, pure-
Purely single player, narrative driven. So you also respect that kind of, like, pure-
Jeff Kaplan
Yeah. I don’t give anyone a pass. I feel like a lot of gamers and game developers, like, if it has writing, they’re like, “The story’s so good.” I’m like, actually, very few games have great story. But Red Dead has a great story. It’s got great character development. It’s got a good plot. And the dialogue is like… It’s like Tarantino-level- … high-quality dialogue. So… Red Dead’s up there. I have my other games that make the list for me, and these are… Both these games are… I would never tell you to play them. EverQuest and Rust are two of the most defining games to me and my career and my life. And Rust, I would never recommend somebody go and play it. Rust will come calling to you if you are up to play it.
Yeah. I don’t give anyone a pass. I feel like a lot of gamers and game developers, like, if it has writing, they’re like, “The story’s so good.” I’m like, actually, very few games have great story. But Red Dead has a great story. It’s got great character development. It’s got a good plot. And the dialogue is like… It’s like Tarantino-level- … high-quality dialogue. So… Red Dead’s up there. I have my other games that make the list for me, and these are… Both these games are… I would never tell you to play them. EverQuest and Rust are two of the most defining games to me and my career and my life. And Rust, I would never recommend somebody go and play it. Rust will come calling to you if you are up to play it.
Lex Fridman
It is a cult. It’s 100% a cult.
It is a cult. It’s 100% a cult.
Jeff Kaplan
That’s-
That’s-
Lex Fridman
It… When you are ready, it will come down.
It… When you are ready, it will come down.
Jeff Kaplan
It will come down. It will let you know.
It will come down. It will let you know.
Lex Fridman
The, the sky will part. Okay.
The, the sky will part. Okay.
Jeff Kaplan
In Rust, you are considered a complete noob that doesn’t know what he’s doing- … if you don’t have a thousand hours. Even a thousand hours-
In Rust, you are considered a complete noob that doesn’t know what he’s doing- … if you don’t have a thousand hours. Even a thousand hours-
Lex Fridman
A thousand hours
A thousand hours
Jeff Kaplan
… people would be like, “Oh, you only have a thousand hours-” “… in that game.” Yeah. But Rust and a lot of inspiration for me in the game I’m working on now… My game is not like Rust in that it’s not a PvP-centric game, but it will have PvP.
… people would be like, “Oh, you only have a thousand hours-” “… in that game.” Yeah. But Rust and a lot of inspiration for me in the game I’m working on now… My game is not like Rust in that it’s not a PvP-centric game, but it will have PvP.
Lex Fridman
What aspect of Rust do you draw inspiration from? Just…
What aspect of Rust do you draw inspiration from? Just…
Jeff Kaplan
I love the resetting world. It’s a- … great game mechanic and it’s one that I want to evolve and work upon.
I love the resetting world. It’s a- … great game mechanic and it’s one that I want to evolve and work upon.
Lex Fridman
How often is the world reset do you think, in Legend of California?
How often is the world reset do you think, in Legend of California?
Jeff Kaplan
I don’t know yet. Probably every month. We want it to be fast enough that you’re not too attached, but we wanna make it rewarding. Like, the trick is coming up with not why am I upset that the world resets, but why am I excited that the world- … resets? And we know players can get very angry about resetting worlds, but anybody who’s played 5,000 hours of Rust, like some of us, the resetting world is the magic. It’s, “I can’t wait for the next reset because the adventure starts all over again.” And if you wanna play the first time with me—like, if we wanna play World of Warcraft, and I’m level 80 and you’re level one, there’s no meaningful experience we can have together—but in Rust, we just wait for a reset and we’re both naked on the beach, you know, from minute one.
I don’t know yet. Probably every month. We want it to be fast enough that you’re not too attached, but we wanna make it rewarding. Like, the trick is coming up with not why am I upset that the world resets, but why am I excited that the world- … resets? And we know players can get very angry about resetting worlds, but anybody who’s played 5,000 hours of Rust, like some of us, the resetting world is the magic. It’s, “I can’t wait for the next reset because the adventure starts all over again.” And if you wanna play the first time with me—like, if we wanna play World of Warcraft, and I’m level 80 and you’re level one, there’s no meaningful experience we can have together—but in Rust, we just wait for a reset and we’re both naked on the beach, you know, from minute one.
Lex Fridman
What about the experience of Rust where you can have everything taken away from you? So that part that you-
What about the experience of Rust where you can have everything taken away from you? So that part that you-
Jeff Kaplan
We’re not doing that.
We’re not doing that.
Lex Fridman
Great, great. Because that feels awfully stressful.
Great, great. Because that feels awfully stressful.
Jeff Kaplan
See… I just lost the entire Rust audience when I said we’re not doing that because- … if you’re a Rust player, you’re not thinking you’re gonna lose everything you have. You’re thinking, “I’m gonna take everything somebody else has.” But-
See… I just lost the entire Rust audience when I said we’re not doing that because- … if you’re a Rust player, you’re not thinking you’re gonna lose everything you have. You’re thinking, “I’m gonna take everything somebody else has.” But-
Lex Fridman
See, my perception of the Rust audience is there’s, like, three people, they’re in a castle somewhere. It’s a very exclusive group.
See, my perception of the Rust audience is there’s, like, three people, they’re in a castle somewhere. It’s a very exclusive group.
Jeff Kaplan
They are, they are highly skilled, highly passionate… highly knowledgeable, but yeah, it’s an inspiration for me. That and EverQuest were defining… And I’ve… The amount of hours I’ve logged in both those games are insane.
They are, they are highly skilled, highly passionate… highly knowledgeable, but yeah, it’s an inspiration for me. That and EverQuest were defining… And I’ve… The amount of hours I’ve logged in both those games are insane.
Lex Fridman
What do you think has more hours from Jeff Kaplan, EverQuest or Rust?
What do you think has more hours from Jeff Kaplan, EverQuest or Rust?
Jeff Kaplan
Well, you said I was 6K on EQ, so that puts me at… I’m at 5K in Rust.
Well, you said I was 6K on EQ, so that puts me at… I’m at 5K in Rust.
Lex Fridman
And, and also in that collection is Zork.
And, and also in that collection is Zork.
Jeff Kaplan
Zork was… I mean, Zork, it just brings me back to that old IBM PC with my mom and my brother, trying to figure out, you know, like, how to keep the lights on or else a grue’s gonna eat us, you know?
Zork was… I mean, Zork, it just brings me back to that old IBM PC with my mom and my brother, trying to figure out, you know, like, how to keep the lights on or else a grue’s gonna eat us, you know?
AI and future of video games
Lex Fridman
Yeah. So certain games just capture your heart and they stay with you forever. What do you think is the future of video games? So there’s a lot of conversations about AI helping expand maybe the storytelling aspects, the world creation aspects, becoming a tool that people can use more so. Maybe creating more believable NPCs, that kind of thing. But also there’s, as we’ve talked about, the video game industry is changing and evolving and trying to figure out, well, there’s the indie game makers that will have more power… Or these larger game makers will have more power, so what do you think the future of games looks like?
Yeah. So certain games just capture your heart and they stay with you forever. What do you think is the future of video games? So there’s a lot of conversations about AI helping expand maybe the storytelling aspects, the world creation aspects, becoming a tool that people can use more so. Maybe creating more believable NPCs, that kind of thing. But also there’s, as we’ve talked about, the video game industry is changing and evolving and trying to figure out, well, there’s the indie game makers that will have more power… Or these larger game makers will have more power, so what do you think the future of games looks like?
Jeff Kaplan
I think with AI in mind in particular, I think the current state of AI, trying to integrate it into development is mostly a hot mess.
I think with AI in mind in particular, I think the current state of AI, trying to integrate it into development is mostly a hot mess.
Jeff Kaplan
But I do think that, you know, games are a technology-driven art form. And somebody much smarter than me once described it—and I’m paraphrasing—making a game is like making a movie if you had to invent the camera every time, because you’re kind of inventing the technology of your specific game. And I think AI can play a role in that, and it would be silly not to look at it as an option. The problem with AI right now is it’s overconfident in what it tries to deliver. Like, I fooled around, obviously like everybody, you mess around with, you know, ChatGPT and Gemini and you fool around with some of the art generation, and it’s fun for non-artists to fool around on Midjourney. But it’s mostly weird and shitty.
But I do think that, you know, games are a technology-driven art form. And somebody much smarter than me once described it—and I’m paraphrasing—making a game is like making a movie if you had to invent the camera every time, because you’re kind of inventing the technology of your specific game. And I think AI can play a role in that, and it would be silly not to look at it as an option. The problem with AI right now is it’s overconfident in what it tries to deliver. Like, I fooled around, obviously like everybody, you mess around with, you know, ChatGPT and Gemini and you fool around with some of the art generation, and it’s fun for non-artists to fool around on Midjourney. But it’s mostly weird and shitty.
Jeff Kaplan
And even, like, when trying to have AI answer for me… Like, I don’t normally make UI in a game, and so I’m trying to figure out, like, UMG and Unreal Engine and I’m asking ChatGPT how to fix, like, a simple problem, like, how do I make the chat wrap, you know? And it, like, overconfidently gives me the wrong answer. And it’s, like, right one in 10 times. So its hit rate has to be a lot better. I think there’s a lot of moral concerns around AI when it comes to creative pursuits as well, like no one’s creative work should ever be used by AI without their permission.
And even, like, when trying to have AI answer for me… Like, I don’t normally make UI in a game, and so I’m trying to figure out, like, UMG and Unreal Engine and I’m asking ChatGPT how to fix, like, a simple problem, like, how do I make the chat wrap, you know? And it, like, overconfidently gives me the wrong answer. And it’s, like, right one in 10 times. So its hit rate has to be a lot better. I think there’s a lot of moral concerns around AI when it comes to creative pursuits as well, like no one’s creative work should ever be used by AI without their permission.
Jeff Kaplan
You know, voice actors and artists, it can’t be lifting from them without their permission. That’s just immoral. It’s no different than just sort of stealing. So that’s wrong. I think how I’m curious, like especially as somebody who runs a small studio with 34 people, it’s like, what are the points of tedium that maybe AI could help out with that I don’t wanna do, and I’m not gonna hire someone to do? So I have, like a really dumb example: I’m making a bunch of images, I size them all incorrectly ’cause I’m dumb and I’m not an artist, and I did it all in Photoshop, and I have like 2,000 images that are the wrong size. I can have ChatGPT resize those and zip it in a file for me, and it literally takes it like a minute to do that.
You know, voice actors and artists, it can’t be lifting from them without their permission. That’s just immoral. It’s no different than just sort of stealing. So that’s wrong. I think how I’m curious, like especially as somebody who runs a small studio with 34 people, it’s like, what are the points of tedium that maybe AI could help out with that I don’t wanna do, and I’m not gonna hire someone to do? So I have, like a really dumb example: I’m making a bunch of images, I size them all incorrectly ’cause I’m dumb and I’m not an artist, and I did it all in Photoshop, and I have like 2,000 images that are the wrong size. I can have ChatGPT resize those and zip it in a file for me, and it literally takes it like a minute to do that.
Jeff Kaplan
I wasn’t gonna hire an intern to do it. I was just gonna work an hour later or two hours later that night to do it. Like, it made my life easier. It didn’t take a job. That seems okay. As long as that ethical line stays in place, what I- what I don’t worry about is, no matter how good AI gets, never gonna draw a picture like Arnold Tsang. It’s never gonna tell a story like Chris Metzen. You know, that human spirit is irreplaceable.
I wasn’t gonna hire an intern to do it. I was just gonna work an hour later or two hours later that night to do it. Like, it made my life easier. It didn’t take a job. That seems okay. As long as that ethical line stays in place, what I- what I don’t worry about is, no matter how good AI gets, never gonna draw a picture like Arnold Tsang. It’s never gonna tell a story like Chris Metzen. You know, that human spirit is irreplaceable.
Lex Fridman
Yeah, it’s hard to put into words what is that magic that humans produce, but they do. Truly great creative minds, truly great creative teams, they- they create something special. It’s hard to really articulate exactly what’s missing with- with AI, you know, what people call AI slop. ‘Cause it creates really beautiful imagery and beautiful stories, and very believable text. But it’s not quite… It doesn’t have that, I don’t know what it is, the edge that’s human. Maybe it’s the imperfections.
Yeah, it’s hard to put into words what is that magic that humans produce, but they do. Truly great creative minds, truly great creative teams, they- they create something special. It’s hard to really articulate exactly what’s missing with- with AI, you know, what people call AI slop. ‘Cause it creates really beautiful imagery and beautiful stories, and very believable text. But it’s not quite… It doesn’t have that, I don’t know what it is, the edge that’s human. Maybe it’s the imperfections.
Jeff Kaplan
Yeah, I think so. Like AI to me right now currently, it’s like an interesting fever dream, you know?
Yeah, I think so. Like AI to me right now currently, it’s like an interesting fever dream, you know?
Lex Fridman
Yeah. Yeah. Yeah.
Yeah. Yeah. Yeah.
Jeff Kaplan
That’s the point I’m at with it.
That’s the point I’m at with it.
Lex Fridman
And a useful tool for the mundane tasks, like you said. But do you think the small studios have hope in the future of gaming?
And a useful tool for the mundane tasks, like you said. But do you think the small studios have hope in the future of gaming?
Jeff Kaplan
Small studios are the future of gaming. The big studios basically acquire the small studios for new IP and ideas, and the small studios grow in. The really compelling, new, innovative ideas are gonna come out of small studios.
Small studios are the future of gaming. The big studios basically acquire the small studios for new IP and ideas, and the small studios grow in. The really compelling, new, innovative ideas are gonna come out of small studios.
Lex Fridman
What advice would you give to video game creators, small teams, if they wanna create a truly special game?
What advice would you give to video game creators, small teams, if they wanna create a truly special game?
Jeff Kaplan
Well, they know how to do it. I mean, if they’re doing it, they know how to do it. It’s more to video game developers in general, own the craft. Own our art form. Stop giving it to these fucking corporate jackals. You are the golden goose. Keep your eggs.
Well, they know how to do it. I mean, if they’re doing it, they know how to do it. It’s more to video game developers in general, own the craft. Own our art form. Stop giving it to these fucking corporate jackals. You are the golden goose. Keep your eggs.
Lex Fridman
Jeff, formerly from the Overwatch team, I have to say from the bottom of my heart, and I think I speak for millions of people, thank you for everything you’ve created in this world. Now that I’ve gotten the chance to see the new game, I’m, I can’t tell you how excited I am to try it. Thank you for everything you’ve created. Thank you for everything you represent. Thank you for remaining and fighting for us as one of us. So thank you, and thank you for talking today.
Jeff, formerly from the Overwatch team, I have to say from the bottom of my heart, and I think I speak for millions of people, thank you for everything you’ve created in this world. Now that I’ve gotten the chance to see the new game, I’m, I can’t tell you how excited I am to try it. Thank you for everything you’ve created. Thank you for everything you represent. Thank you for remaining and fighting for us as one of us. So thank you, and thank you for talking today.
Jeff Kaplan
Thank you, Lex.
Thank you, Lex.
Lex Fridman
Thanks for listening to this conversation with Jeff Kaplan. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from Franz Kafka, “Don’t bend. Don’t water it down. Don’t try to make it logical. Don’t edit your own soul according to the fashion. Rather, follow your most intense obsessions mercilessly.” Thank you for listening, and hope to see you next time.
Thanks for listening to this conversation with Jeff Kaplan. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from Franz Kafka, “Don’t bend. Don’t water it down. Don’t try to make it logical. Don’t edit your own soul according to the fashion. Rather, follow your most intense obsessions mercilessly.” Thank you for listening, and hope to see you next time.
#492 – Rick Beato: Greatest Guitarists of All Time, History & Future of Music
Podcast: Play in new window | Download
Subscribe: Spotify | TuneIn | RSS
Rick Beato is a music educator, interviewer, producer, songwriter, and a true multi-instrument musician, playing guitar, bass, cello & piano. His incredible YouTube channel celebrates great musicians & musical ideas, and helps millions of people fall in love with great music all over again.
Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep492-sc
See below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.
Transcript:
https://lexfridman.com/rick-beato-transcript
CONTACT LEX:
Feedback – give feedback to Lex: https://lexfridman.com/survey
AMA – submit questions, videos or call-in: https://lexfridman.com/ama
Hiring – join our team: https://lexfridman.com/hiring
Other – other ways to get in touch: https://lexfridman.com/contact
EPISODE LINKS:
Rick’s YouTube: https://youtube.com/RickBeato
Rick’s X: https://x.com/rickbeato
Rick’s Instagram: https://instagram.com/rickbeato1
Rick’s Website: https://rickbeato.com
Rick’s Ear Training: https://beatoeartraining.com
The Beato Book: https://beatobook.com
SPONSORS:
To support this podcast, check out our sponsors & get discounts:
UPLIFT Desk: Standing desks and office ergonomics.
Go to https://upliftdesk.com/lex
BetterHelp: Online therapy and counseling.
Go to https://betterhelp.com/lex
LMNT: Zero-sugar electrolyte drink mix.
Go to https://drinkLMNT.com/lex
Fin: AI agent for customer service.
Go to https://fin.ai/lex
Shopify: Sell stuff online.
Go to https://shopify.com/lex
Perplexity: AI-powered answer engine.
Go to https://perplexity.ai/
OUTLINE:
(00:00) – Introduction
(00:28) – Sponsors, Comments, and Reflections
(09:17) – Guitar solos
(13:16) – Gypsy jazz and Django Reinhardt
(14:48) – Bebop jazz
(19:00) – Perfect pitch vs relative pitch
(23:37) – Learning to play guitar
(47:08) – Miles Davis
(52:34) – Bass guitar
(53:41) – Greatest guitar solos of all time
(1:22:56) – 27 Club
(1:27:37) – Elton John
(1:30:51) – Metallica
(1:35:21) – Tom Waits
(1:41:12) – Greatest rock stars
(1:44:35) – Beethoven
(1:51:10) – Bach
(1:54:01) – AI in music
(2:07:52) – Sabrina Carpenter
(2:11:23) – YouTube copyright strikes
(2:16:59) – Spotify
(2:27:51) – Guitars
(2:32:13) – Advice
PODCAST LINKS:
– Podcast Website: https://lexfridman.com/podcast
– Apple Podcasts: https://apple.co/2lwqZIr
– Spotify: https://spoti.fi/2nEwCF8
– RSS: https://lexfridman.com/feed/podcast/
– Podcast Playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
– Clips Channel: https://www.youtube.com/lexclips
Transcript for Rick Beato: Greatest Guitarists of All Time, History & Future of Music | Lex Fridman Podcast #492
This is a transcript of Lex Fridman Podcast #492 with Rick Beato.
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
- Go back to this episode’s main page
- Watch the full YouTube version of the podcast
Table of Contents
Here are the loose “chapters” in the conversation.
Click link to jump approximately to that part in the transcript:
Click link to jump approximately to that part in the transcript:
- 0:00 – Introduction
- 0:44 – Guitar solos
- 4:43 – Gypsy jazz and Django Reinhardt
- 6:14 – Bebop jazz
- 10:27 – Perfect pitch vs relative pitch
- 15:04 – Learning to play guitar
- 38:34 – Miles Davis
- 44:01 – Bass guitar
- 45:08 – Greatest guitar solos of all time
- 1:14:23 – 27 Club
- 1:19:04 – Elton John
- 1:22:18 – Metallica
- 1:26:48 – Tom Waits
- 1:32:39 – Greatest rock stars
- 1:36:02 – Beethoven
- 1:42:37 – Bach
- 1:45:27 – AI in music
- 1:59:18 – Sabrina Carpenter
- 2:02:49 – YouTube copyright strikes
- 2:08:26 – Spotify
- 2:19:18 – Guitars
- 2:23:40 – Advice
Introduction
Lex Fridman
The following is a conversation with Rick Beato, legendary music educator, interviewer, producer, songwriter, and a true multi-instrument musician, playing guitar, bass, cello, and piano. Rick, with his incredible YouTube channel, celebrates great musicians and musical ideas, and helps millions of people, including me, fall in love with great music all over again. This is Lex Fridman Podcast. To support it, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Rick Beato. You had, I think, an incredibly fun and diverse beginning to your music journey.
The following is a conversation with Rick Beato, legendary music educator, interviewer, producer, songwriter, and a true multi-instrument musician, playing guitar, bass, cello, and piano. Rick, with his incredible YouTube channel, celebrates great musicians and musical ideas, and helps millions of people, including me, fall in love with great music all over again. This is Lex Fridman Podcast. To support it, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Rick Beato. You had, I think, an incredibly fun and diverse beginning to your music journey.
Guitar solos
Lex Fridman
I heard somewhere that one of the things that made you fall in love with music was listening to guitar solos, some epic guitar solos. What’s an early guitar solo that you remember you connected to spiritually , musically, where you’re like, “Wow, there’s magic in this”?
I heard somewhere that one of the things that made you fall in love with music was listening to guitar solos, some epic guitar solos. What’s an early guitar solo that you remember you connected to spiritually , musically, where you’re like, “Wow, there’s magic in this”?
Rick Beato
Well, the first solo that I learned was Hey Joe. It was actually a good beginner song, you know, when I first started playing the guitar, because it has pretty simple chords, right? So it’s like E, C, G, D, A. And I learned the solo, and I figured out this, like, I’ll say it’s this pentatonic scale, E minor pentatonic scale though. I didn’t know that’s what it was called, but learned this thing, and it’s like, “Whoa, he’s just in this one shape here.” Now, there was no… You couldn’t go look anything up. You just, if you could figure out the notes, you noticed that there was a little pattern to it.
Well, the first solo that I learned was Hey Joe. It was actually a good beginner song, you know, when I first started playing the guitar, because it has pretty simple chords, right? So it’s like E, C, G, D, A. And I learned the solo, and I figured out this, like, I’ll say it’s this pentatonic scale, E minor pentatonic scale though. I didn’t know that’s what it was called, but learned this thing, and it’s like, “Whoa, he’s just in this one shape here.” Now, there was no… You couldn’t go look anything up. You just, if you could figure out the notes, you noticed that there was a little pattern to it.
Rick Beato
And then I got so obsessed with it, and I showed my younger brother John, who started playing guitar right at the same time I did. So I was 14, he was 11. And I would play rhythm for him for five minutes while he would solo over Hey Joe. And then as soon as I’d start soloing, he’d throw the guitar down, then we’d get in a fight. And so my mom eventually was like, “What is going on here?” And I was like, “John won’t play rhythm.” “John won’t play rhythm for me.” She’s like, “Okay, I’ll play rhythm for you. What, what are the chords?” And-
And then I got so obsessed with it, and I showed my younger brother John, who started playing guitar right at the same time I did. So I was 14, he was 11. And I would play rhythm for him for five minutes while he would solo over Hey Joe. And then as soon as I’d start soloing, he’d throw the guitar down, then we’d get in a fight. And so my mom eventually was like, “What is going on here?” And I was like, “John won’t play rhythm.” “John won’t play rhythm for me.” She’s like, “Okay, I’ll play rhythm for you. What, what are the chords?” And-
Lex Fridman
That’s awesome.
That’s awesome.
Rick Beato
… I was like, “Okay, it’s like E, C, G, D, A.” And so my mom would literally play rhythm for 20 minutes while I’d play.
… I was like, “Okay, it’s like E, C, G, D, A.” And so my mom would literally play rhythm for 20 minutes while I’d play.
Lex Fridman
Hashtag parenting.
Hashtag parenting.
Rick Beato
That’s amazing. When I look back on it now, my mom’s been gone for 10 years now. When I look back on it, it’s like, “My God, my parents were so cool.”
That’s amazing. When I look back on it now, my mom’s been gone for 10 years now. When I look back on it, it’s like, “My God, my parents were so cool.”
Lex Fridman
We should mention that Hey Joe, and Hendrix in general, is kind of known for the rhythm not being simple rhythm, just the chords that you mentioned. It’s what you do with those chords. It’s almost improvisation, the rhythm side.
We should mention that Hey Joe, and Hendrix in general, is kind of known for the rhythm not being simple rhythm, just the chords that you mentioned. It’s what you do with those chords. It’s almost improvisation, the rhythm side.
Rick Beato
He did all those really cool chord fragment riffs and things like that, that’s just part of his… That’s the Hendrix style.
He did all those really cool chord fragment riffs and things like that, that’s just part of his… That’s the Hendrix style.
Lex Fridman
What do you think? I mean, many people put Hendrix as the greatest guitarist of all time. What do you think is part of that?
What do you think? I mean, many people put Hendrix as the greatest guitarist of all time. What do you think is part of that?
Rick Beato
You know, I make lists.
You know, I make lists.
Lex Fridman
You do. If you somehow don’t know who Rick Beato is, go on YouTube right now and watch your excellent interviews with musicians, watch your breakdown analysis of different songs, and watch your top 20 lists, where you’re very opinionated, sometimes very openly critical about certain kinds of songs. It’s fun. Opinions are fun.
You do. If you somehow don’t know who Rick Beato is, go on YouTube right now and watch your excellent interviews with musicians, watch your breakdown analysis of different songs, and watch your top 20 lists, where you’re very opinionated, sometimes very openly critical about certain kinds of songs. It’s fun. Opinions are fun.
Rick Beato
But they do change, Lex, from day to day.
But they do change, Lex, from day to day.
Lex Fridman
Yeah, exactly.
Yeah, exactly.
Rick Beato
You know, like I… But when, anytime I do a list, if I do 20, I like to do 20 because that gives me some leeway to throw in. I have to throw in something that is so weird that people, you know… Something that a lot of people won’t know, just to have it on there, so I can at least introduce a person. You know, I’ll put somebody like Allan Holdsworth, who’s a famous fusion guitar player. I’ll throw in one of his solos or something—just some, some oddball solo in there, just so that people, as they’re listening down the list, will get exposed to something they would not necessarily get exposed to.
You know, like I… But when, anytime I do a list, if I do 20, I like to do 20 because that gives me some leeway to throw in. I have to throw in something that is so weird that people, you know… Something that a lot of people won’t know, just to have it on there, so I can at least introduce a person. You know, I’ll put somebody like Allan Holdsworth, who’s a famous fusion guitar player. I’ll throw in one of his solos or something—just some, some oddball solo in there, just so that people, as they’re listening down the list, will get exposed to something they would not necessarily get exposed to.
Lex Fridman
Yeah, a lot of variety. But Hendrix… Did you show up here today, Rick, try to tell me that Hendrix is not up there? I just am getting that vibe right now.
Yeah, a lot of variety. But Hendrix… Did you show up here today, Rick, try to tell me that Hendrix is not up there? I just am getting that vibe right now.
Rick Beato
No, I’m not. But I don’t want to say greatest, you know… You can say, well, there are people that inspired Jimi Hendrix. Charlie Christian, older guitar players. Charlie Christian and Django Reinhardt were the first two really big, and probably Andrés Segovia—those were three of the giants of the 20th century, as far as guitar influences for most of the players that were to follow.
No, I’m not. But I don’t want to say greatest, you know… You can say, well, there are people that inspired Jimi Hendrix. Charlie Christian, older guitar players. Charlie Christian and Django Reinhardt were the first two really big, and probably Andrés Segovia—those were three of the giants of the 20th century, as far as guitar influences for most of the players that were to follow.
Gypsy jazz and Django Reinhardt
Lex Fridman
So here, going to Perplexity, Django Reinhardt was, of course, a jazz guitarist and composer, active mainly in France, and is widely regarded as one of the greatest guitarists in jazz history.
So here, going to Perplexity, Django Reinhardt was, of course, a jazz guitarist and composer, active mainly in France, and is widely regarded as one of the greatest guitarists in jazz history.
Rick Beato
So, Django was… Well, there’s a huge movement right now, Gypsy Jazz Movement, as they call it- … that is kind of built around this style of music that he played back in the early 20th century. One of the things about Django is that he was in a fire, and he had two of his third and fourth finger, so his ring finger and pinky were essentially melted together. He had no use of them. Although he could use them while he was chording, but a lot of these incredibly fast lines, he’s just playing with two fingers. And it’s amazing.
So, Django was… Well, there’s a huge movement right now, Gypsy Jazz Movement, as they call it- … that is kind of built around this style of music that he played back in the early 20th century. One of the things about Django is that he was in a fire, and he had two of his third and fourth finger, so his ring finger and pinky were essentially melted together. He had no use of them. Although he could use them while he was chording, but a lot of these incredibly fast lines, he’s just playing with two fingers. And it’s amazing.
Lex Fridman
That… What is that? So that’s Gypsy Jazz.
That… What is that? So that’s Gypsy Jazz.
Rick Beato
That’s Gypsy Jazz, yeah. Him; Stéphane Grappelli was a violinist that played with him a lot.
That’s Gypsy Jazz, yeah. Him; Stéphane Grappelli was a violinist that played with him a lot.
Lex Fridman
How much of this is improvisation?
How much of this is improvisation?
Rick Beato
Everything he’s doing there is improvised.
Everything he’s doing there is improvised.
Bebop jazz
Lex Fridman
It feels so free. And fun like swing, and then at least you said pre-bebop. So bebop was a kind of jazz that was also influential on you in your own life journey. And it’s this complicated, legendary kind of jazz that was very influential on the music that followed. So what, what was bebop?
It feels so free. And fun like swing, and then at least you said pre-bebop. So bebop was a kind of jazz that was also influential on you in your own life journey. And it’s this complicated, legendary kind of jazz that was very influential on the music that followed. So what, what was bebop?
Rick Beato
Well, after the big bands were happening in the, you know, from the ’20s through the ’40s, people would go out and play in small groups that they would tour with. And Charlie Parker, who’s really kind of the, one of the main figures of early bebop, really developed the language of it. Usually, the music that they’re playing over are standard chord progressions- … that they would use as vehicles to improvise over. A lot of them were AABA form. And Charlie Parker created this language of improvisation that was far more sophisticated than the swing players of the big band era. You know, think of people like Benny Goodman of that era. They would have really fast tempo songs, angular lines, chromaticism, things like that, chromatic notes.
Well, after the big bands were happening in the, you know, from the ’20s through the ’40s, people would go out and play in small groups that they would tour with. And Charlie Parker, who’s really kind of the, one of the main figures of early bebop, really developed the language of it. Usually, the music that they’re playing over are standard chord progressions- … that they would use as vehicles to improvise over. A lot of them were AABA form. And Charlie Parker created this language of improvisation that was far more sophisticated than the swing players of the big band era. You know, think of people like Benny Goodman of that era. They would have really fast tempo songs, angular lines, chromaticism, things like that, chromatic notes.
Lex Fridman
Chromatic notes are just notes next to each other on-
Chromatic notes are just notes next to each other on-
Rick Beato
Next to each other, yeah.
Next to each other, yeah.
Lex Fridman
… on the keyboard.
… on the keyboard.
Rick Beato
I like to think of it as connecting notes.
I like to think of it as connecting notes.
Lex Fridman
Connecting. You’re putting in more notes than are supposed to be there and so doing, creating some interesting texture.
Connecting. You’re putting in more notes than are supposed to be there and so doing, creating some interesting texture.
Rick Beato
Yeah, so that is one of the most difficult styles to master, because all these things are a language. Blues playing, they’re all just languages, right? It’s like, just like you’d learn any type of language. My dad loved bebop. Now, when I was a little kid and he’s listening to these bebop records, whether it’s Charlie Parker or Dizzy Gillespie or Oscar Peterson, Joe Pass, great jazz guitar player, I’m just hearing this stuff. I don’t know any different. My dad was not a musician, but for some reason, he liked incredibly sophisticated-
Yeah, so that is one of the most difficult styles to master, because all these things are a language. Blues playing, they’re all just languages, right? It’s like, just like you’d learn any type of language. My dad loved bebop. Now, when I was a little kid and he’s listening to these bebop records, whether it’s Charlie Parker or Dizzy Gillespie or Oscar Peterson, Joe Pass, great jazz guitar player, I’m just hearing this stuff. I don’t know any different. My dad was not a musician, but for some reason, he liked incredibly sophisticated-
Rick Beato
… music that was very technical. And I just heard it and just was like, “Oh, yeah, okay, cool.” And not realizing that it was developing my ear, because I really, bebop is one of the hardest to improvise in that style, in that language of bebop. It’s very difficult to do. And hearing it as a kid is one of the things that I think enables you, just like languages, enables you to learn it as opposed to somebody that’s never been exposed to it and tries to learn it as a teenager. So I think it’s very similar to learning languages, which kinda is like my theory on perfect pitch, that every child is born with perfect pitch. And they start to lose the ability around nine months-
… music that was very technical. And I just heard it and just was like, “Oh, yeah, okay, cool.” And not realizing that it was developing my ear, because I really, bebop is one of the hardest to improvise in that style, in that language of bebop. It’s very difficult to do. And hearing it as a kid is one of the things that I think enables you, just like languages, enables you to learn it as opposed to somebody that’s never been exposed to it and tries to learn it as a teenager. So I think it’s very similar to learning languages, which kinda is like my theory on perfect pitch, that every child is born with perfect pitch. And they start to lose the ability around nine months-
Rick Beato
… when people become culturally bound listeners, when babies do. They start out as citizens of the world, you know? They have the neural pathways to hear the sounds, the phonemes of all 6,500 languages spoken on Earth. But then around nine months, they begin to lose that ability and they, when they become these culturally bound listeners, there’s a great YouTube video with this woman, Patricia Kuhl. She’s a language researcher. And I watched this, “The Linguistic Genius of Babies.”
… when people become culturally bound listeners, when babies do. They start out as citizens of the world, you know? They have the neural pathways to hear the sounds, the phonemes of all 6,500 languages spoken on Earth. But then around nine months, they begin to lose that ability and they, when they become these culturally bound listeners, there’s a great YouTube video with this woman, Patricia Kuhl. She’s a language researcher. And I watched this, “The Linguistic Genius of Babies.”
Rick Beato
I saw this in 2010, this lecture that she did, like a TED Talk, and she talks about this, that kids, they did an experiment. They exposed kids to Mandarin three times a week for 25-minute sessions, just a person speaking Mandarin to these babies. And they were able to recognize the sounds, the phonemes of that language even later on. And when I realized that my son Dylan had perfect pitch, I thought, “Why does Dylan have perfect pitch but no one in my family had ever had perfect pitch?” And I thought, “Well, it must be because of the things I exposed to him prenatally and then in the first nine months of his life.” ‘Cause that’s the only way I could explain it.
I saw this in 2010, this lecture that she did, like a TED Talk, and she talks about this, that kids, they did an experiment. They exposed kids to Mandarin three times a week for 25-minute sessions, just a person speaking Mandarin to these babies. And they were able to recognize the sounds, the phonemes of that language even later on. And when I realized that my son Dylan had perfect pitch, I thought, “Why does Dylan have perfect pitch but no one in my family had ever had perfect pitch?” And I thought, “Well, it must be because of the things I exposed to him prenatally and then in the first nine months of his life.” ‘Cause that’s the only way I could explain it.
Perfect pitch vs relative pitch
Lex Fridman
We’re gonna return to Joe Pass. We gotta go to Dylan. You mentioned Dylan. I guess that’s in part one of the origin stories of you putting out videos into the world, is the early videos you did with Dylan, a set of videos on his perfect pitch. And for people who don’t know, maybe you can speak to what perfect pitch means.
We’re gonna return to Joe Pass. We gotta go to Dylan. You mentioned Dylan. I guess that’s in part one of the origin stories of you putting out videos into the world, is the early videos you did with Dylan, a set of videos on his perfect pitch. And for people who don’t know, maybe you can speak to what perfect pitch means.
Rick Beato
It’s the ability to identify any note without a reference tone. So you can play, it doesn’t matter how quickly they are, a person with perfect pitch can hear a note and immediately identify it. Or a collection of notes.
It’s the ability to identify any note without a reference tone. So you can play, it doesn’t matter how quickly they are, a person with perfect pitch can hear a note and immediately identify it. Or a collection of notes.
Lex Fridman
And taking a tangent upon a tangent, you also have a course on ear training.
And taking a tangent upon a tangent, you also have a course on ear training.
Rick Beato
Yes, but my course is for relative pitch-
Yes, but my course is for relative pitch-
Lex Fridman
Relative pitch
Relative pitch
Rick Beato
… not to be confused with perfect pitch.
… not to be confused with perfect pitch.
Lex Fridman
Is it fair to say that relative pitch, as far as the thing you would learn, is more useful-
Is it fair to say that relative pitch, as far as the thing you would learn, is more useful-
Rick Beato
Yes
Yes
Lex Fridman
… for musicians?
… for musicians?
Rick Beato
Yes.
Yes.
Lex Fridman
Can you explain the difference between the two?
Can you explain the difference between the two?
Rick Beato
Relative pitch is basically learning how to identify pitches relative to a stated tonic or something that you’ve heard, or just relative to each other. If you hear a note and then you hear another note after it, you can recognize, let’s say, it’s a minor third interval. So if you’re on the note A, the next note would be C. So once you’re given a reference note, you can use relative pitch to identify the relative nature from one pitch to another.
Relative pitch is basically learning how to identify pitches relative to a stated tonic or something that you’ve heard, or just relative to each other. If you hear a note and then you hear another note after it, you can recognize, let’s say, it’s a minor third interval. So if you’re on the note A, the next note would be C. So once you’re given a reference note, you can use relative pitch to identify the relative nature from one pitch to another.
Lex Fridman
And of course, intervals make up scales, and intervals make up chords-
And of course, intervals make up scales, and intervals make up chords-
Rick Beato
Chords, yup.
Chords, yup.
Lex Fridman
… and so that if you develop it to any degree, relative pitch, you can understand, you can hear the music better. What does it take, since we’re taking a tangent on a tangent, what does it take to train your ear? What’s a TL;DR on the course before people go out and sign up?
… and so that if you develop it to any degree, relative pitch, you can understand, you can hear the music better. What does it take, since we’re taking a tangent on a tangent, what does it take to train your ear? What’s a TL;DR on the course before people go out and sign up?
Rick Beato
It’s just practice, basically. You start with intervals. Typically with small intervals like minor second, major second. So minor second would be a half-step, major second would be a whole-step.
It’s just practice, basically. You start with intervals. Typically with small intervals like minor second, major second. So minor second would be a half-step, major second would be a whole-step.
Lex Fridman
Are you listening to the tone one after the other or two of them together?
Are you listening to the tone one after the other or two of them together?
Rick Beato
Both. So played separately, it’s called melodic intervals, right, like a melody? And harmonic intervals are played like a harmony, together. So you have to be able to identify them both, both ways.
Both. So played separately, it’s called melodic intervals, right, like a melody? And harmonic intervals are played like a harmony, together. So you have to be able to identify them both, both ways.
Lex Fridman
What’s an early journey? Like, we’ll give people a preview of what they should… Like, what does that look like? What does practice look like?
What’s an early journey? Like, we’ll give people a preview of what they should… Like, what does that look like? What does practice look like?
Rick Beato
Well, my course, it will play you an interval, and then you identify it by clicking on whether it’s, you know, a major third, or minor third, or major sixth, or minor sixth, or perfect fifth, or tritone, whatever it is. And it will teach you gradually, over time, how to recognize all the intervals.
Well, my course, it will play you an interval, and then you identify it by clicking on whether it’s, you know, a major third, or minor third, or major sixth, or minor sixth, or perfect fifth, or tritone, whatever it is. And it will teach you gradually, over time, how to recognize all the intervals.
Lex Fridman
So you listen to a melodic interval or a harmonic interval. How quickly does the ear in the various age groups that we humans are in, how quickly does the ear learn the different intervals? Is it a week? Two weeks? A month? Two months? Five years?
So you listen to a melodic interval or a harmonic interval. How quickly does the ear in the various age groups that we humans are in, how quickly does the ear learn the different intervals? Is it a week? Two weeks? A month? Two months? Five years?
Rick Beato
I think you’d do it pretty quickly. Within, you know, if you practice, within a couple of months, you can really make a lot of progress on it, if you practice daily.
I think you’d do it pretty quickly. Within, you know, if you practice, within a couple of months, you can really make a lot of progress on it, if you practice daily.
Lex Fridman
What benefit does it have to you as a musician in general?
What benefit does it have to you as a musician in general?
Rick Beato
Well, it’s great if you wanna hear a chord progression if you’re trying to figure out a song. And you can say, “Oh, that’s going from the six minor chord, or the four major, to the five major, to the one major.” And you can just identify it immediately, and then you figure out what the first chord is, then you know what the rest of the chords are ’cause they’re in relation to whatever that first chord is. And for learning solos, for example, or learning melodies, being able to sound something out.
Well, it’s great if you wanna hear a chord progression if you’re trying to figure out a song. And you can say, “Oh, that’s going from the six minor chord, or the four major, to the five major, to the one major.” And you can just identify it immediately, and then you figure out what the first chord is, then you know what the rest of the chords are ’cause they’re in relation to whatever that first chord is. And for learning solos, for example, or learning melodies, being able to sound something out.
Lex Fridman
Now, do you recommend people couple that with music theory in terms of education, the education journey?
Now, do you recommend people couple that with music theory in terms of education, the education journey?
Rick Beato
They have to be taught together because these terms are really music theory, right? Those intervals: major second, minor second, major third, minor third, perfect fourth. So as you’re doing that, and then you… Once you learn the intervals, the 12 intervals in an octave, then you learn them both melodically and harmonically, so played together and separate. Then you learn chords, and so then you learn to identify major, minor, diminished, augmented, suspended chords, things like that. Well, you’re basically learning music theory at the same time with that. Because learning… Music theory is just the name of things in music.
They have to be taught together because these terms are really music theory, right? Those intervals: major second, minor second, major third, minor third, perfect fourth. So as you’re doing that, and then you… Once you learn the intervals, the 12 intervals in an octave, then you learn them both melodically and harmonically, so played together and separate. Then you learn chords, and so then you learn to identify major, minor, diminished, augmented, suspended chords, things like that. Well, you’re basically learning music theory at the same time with that. Because learning… Music theory is just the name of things in music.
Lex Fridman
So there’s the sound of things. There’s the name of things, and then there’s the haptic, like playing the thing- … probably. So playing chords, playing scales, you have, I believe, a course on scales and on chords? Okay. Since we’re doing the tangent, let’s go. How do you recommend people… There’s a bunch of people listening to this that are curious about how they can start in playing guitar, maybe even playing piano and maybe playing other instruments. Although guitar, of course, is the greatest instrument of all time.
So there’s the sound of things. There’s the name of things, and then there’s the haptic, like playing the thing- … probably. So playing chords, playing scales, you have, I believe, a course on scales and on chords? Okay. Since we’re doing the tangent, let’s go. How do you recommend people… There’s a bunch of people listening to this that are curious about how they can start in playing guitar, maybe even playing piano and maybe playing other instruments. Although guitar, of course, is the greatest instrument of all time.
Learning to play guitar
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
What are the early steps of that journey? What do you recommend people do in general?
What are the early steps of that journey? What do you recommend people do in general?
Rick Beato
Well, if you’re a beginner getting a good beginner guitar course and learning, first of all, the open chords in first position. A lot of songs can be played that way. A lot of old songs can be played that way, maybe not new modern songs necessarily.
Well, if you’re a beginner getting a good beginner guitar course and learning, first of all, the open chords in first position. A lot of songs can be played that way. A lot of old songs can be played that way, maybe not new modern songs necessarily.
Lex Fridman
So learning a few chords and with an eye towards maybe playing a song?
So learning a few chords and with an eye towards maybe playing a song?
Rick Beato
Yeah. With an eye towards… You learn the chord shapes and you learn how to strum basic patterns to begin with. I think the first thing for learning guitar is actually how to position your fingers so that you don’t mute strings that you don’t want to mute. That’s the hardest thing for people to do, basically, is to get their fingers arched to where they… If you’re playing a C major chord, your index finger’s on the first fret of the B string, and you have to have that open E string ringing there. And it’s hard for people to make those micro-adjustments. You take it for granted, like, you’ve been playing guitar- … for, I don’t know, how many years? Forever, right?
Yeah. With an eye towards… You learn the chord shapes and you learn how to strum basic patterns to begin with. I think the first thing for learning guitar is actually how to position your fingers so that you don’t mute strings that you don’t want to mute. That’s the hardest thing for people to do, basically, is to get their fingers arched to where they… If you’re playing a C major chord, your index finger’s on the first fret of the B string, and you have to have that open E string ringing there. And it’s hard for people to make those micro-adjustments. You take it for granted, like, you’ve been playing guitar- … for, I don’t know, how many years? Forever, right?
Lex Fridman
Forever, yeah.
Forever, yeah.
Rick Beato
And you don’t even think about stuff like that when you’re playing a guitar solo. Every little thing that you do if you’re playing your Comfortably Numb guitar solo- … you have to, out of mid-air, strike the string that your finger’s on to play the note. And these are all fine adjustments that you’re doing.
And you don’t even think about stuff like that when you’re playing a guitar solo. Every little thing that you do if you’re playing your Comfortably Numb guitar solo- … you have to, out of mid-air, strike the string that your finger’s on to play the note. And these are all fine adjustments that you’re doing.
Lex Fridman
I’m just a hobbyist recreational player, but it… Wow, you’re taking me all the way back. You’re right, it’s the haptic, the physical aspect of it is really tricky. Comfortably Numb is a good example, but if you do lead, you have to get a super clean sound. Now, that’s both when you’re playing fast, you want it to be super precise, but when you play slow, when you have one note, and you’re holding it, and you’re bending it- … it better be really clean.
I’m just a hobbyist recreational player, but it… Wow, you’re taking me all the way back. You’re right, it’s the haptic, the physical aspect of it is really tricky. Comfortably Numb is a good example, but if you do lead, you have to get a super clean sound. Now, that’s both when you’re playing fast, you want it to be super precise, but when you play slow, when you have one note, and you’re holding it, and you’re bending it- … it better be really clean.
Rick Beato
Yes.
Yes.
Lex Fridman
And for that, it’s… I guess you have to really place the finger in the right place. Plus, there’s the… Well, there’s the calluses, so it doesn’t hurt. And then the positioning of the string on the curvature of- … of the finger. Where does it fall? Like, how much do you bend the finger?
And for that, it’s… I guess you have to really place the finger in the right place. Plus, there’s the… Well, there’s the calluses, so it doesn’t hurt. And then the positioning of the string on the curvature of- … of the finger. Where does it fall? Like, how much do you bend the finger?
Rick Beato
You have to have enough flesh on it to actually raise the string in pitch.
You have to have enough flesh on it to actually raise the string in pitch.
Lex Fridman
Yep. Yep.
Yep. Yep.
Rick Beato
otherwise it-
otherwise it-
Lex Fridman
Yeah, ’cause you’re lifting it with part of a flesh. And of course, you have to decide, depends on how OCD you are, do you wanna be like the perfect, the proper musician? Or do you wanna do a Hendrix? So the thumb over the top.
Yeah, ’cause you’re lifting it with part of a flesh. And of course, you have to decide, depends on how OCD you are, do you wanna be like the perfect, the proper musician? Or do you wanna do a Hendrix? So the thumb over the top.
Rick Beato
Way over the top, yes.
Way over the top, yes.
Lex Fridman
And so, like, you… I if you have a fretboard here, I think the more, like, classical guitarists, the very proper, perfect perpendicular alignment of the fingertips to the fretboard, versus, like, Hendrix’s, like, “Fuck it. You nerds. I’m gonna do it.” With the messiness is part of the magic. Of course, like B.B. King is also kind of messy looking in terms of his positioning of the fingers, but his tone is incredibly clean.
And so, like, you… I if you have a fretboard here, I think the more, like, classical guitarists, the very proper, perfect perpendicular alignment of the fingertips to the fretboard, versus, like, Hendrix’s, like, “Fuck it. You nerds. I’m gonna do it.” With the messiness is part of the magic. Of course, like B.B. King is also kind of messy looking in terms of his positioning of the fingers, but his tone is incredibly clean.
Rick Beato
Yes, super clean.
Yes, super clean.
Lex Fridman
So, like, that teaches you that maybe any position can converge towards a super clean tone. You just have to figure it out.
So, like, that teaches you that maybe any position can converge towards a super clean tone. You just have to figure it out.
Rick Beato
I think a lot of it has to do with how they wear their guitars. If you wear your guitar low, if you’re Hendrix and you’re wearing your guitar-
I think a lot of it has to do with how they wear their guitars. If you wear your guitar low, if you’re Hendrix and you’re wearing your guitar-
Lex Fridman
That’s true.
That’s true.
Rick Beato
… if you’re wearing it lower, lower, then you can’t get your fingers on top of it like that. And, the thumb acts as a way to mute the lower strings from ringing if you’re playing through a loud amplifier. So there’s so many other micro-adjustments when you’re playing leads, ’cause you have to kind of mute the other strings that are… so they don’t ring out— … if you’re pl- playing the first note in Comfortably Numb and the solo at the end, and you’re at the ninth fret of the G-string, and you bend that- … if you bend that G-string and you accidentally hit the B-string under it- … you don’t want that ringing. So you have to kind of angle your index finger so it-
… if you’re wearing it lower, lower, then you can’t get your fingers on top of it like that. And, the thumb acts as a way to mute the lower strings from ringing if you’re playing through a loud amplifier. So there’s so many other micro-adjustments when you’re playing leads, ’cause you have to kind of mute the other strings that are… so they don’t ring out— … if you’re pl- playing the first note in Comfortably Numb and the solo at the end, and you’re at the ninth fret of the G-string, and you bend that- … if you bend that G-string and you accidentally hit the B-string under it- … you don’t want that ringing. So you have to kind of angle your index finger so it-
Lex Fridman
To mute
To mute
Rick Beato
… to mute that. So all these micro-adjustments that you don’t even think about… I mean, you’re not thinking about that, Lex, when you’re playing it. You’ve done it so many times that these things are just part of your brain. That’s why this is such a great brain developer for kids to learn instruments.
… to mute that. So all these micro-adjustments that you don’t even think about… I mean, you’re not thinking about that, Lex, when you’re playing it. You’ve done it so many times that these things are just part of your brain. That’s why this is such a great brain developer for kids to learn instruments.
Lex Fridman
Yeah, of course, you have to solve that puzzle. It must be really frustrating in the beginning, like holding a chord. Like all of ’em, and it hurts too, right?
Yeah, of course, you have to solve that puzzle. It must be really frustrating in the beginning, like holding a chord. Like all of ’em, and it hurts too, right?
Rick Beato
It does hurt.
It does hurt.
Lex Fridman
If you’re doing acoustic guitar.
If you’re doing acoustic guitar.
Rick Beato
Not for that long, though. For like a week.
Not for that long, though. For like a week.
Lex Fridman
Couple, couple, yeah.
Couple, couple, yeah.
Rick Beato
Couple weeks.
Couple weeks.
Lex Fridman
Couple.
Couple.
Rick Beato
I don’t want to discourage anyone, you know. It’s actually pretty easy to learn basic stuff.
I don’t want to discourage anyone, you know. It’s actually pretty easy to learn basic stuff.
Lex Fridman
Right, but the pain is temporary, I guess is the point I’m trying to make. So, what else? So the physical component, play a few chords, where does the journey continue if you’re learning guitar?
Right, but the pain is temporary, I guess is the point I’m trying to make. So, what else? So the physical component, play a few chords, where does the journey continue if you’re learning guitar?
Rick Beato
Well then, it’s like if you play electric guitar, then you get into single note playing and stuff like that. That’s where it gets, to me, where it gets really fun. You know, you have single note playing with riffs, if you think of Back In Black, right, that has a riff embedded in the actual melody. Or many songs that have riffs, the Hendrix stuff that has chordal riffs, and you’re moving up the neck and involving all the fingers and things like that. So there’s… it really depends on what you wanna, what styles you wanna play.
Well then, it’s like if you play electric guitar, then you get into single note playing and stuff like that. That’s where it gets, to me, where it gets really fun. You know, you have single note playing with riffs, if you think of Back In Black, right, that has a riff embedded in the actual melody. Or many songs that have riffs, the Hendrix stuff that has chordal riffs, and you’re moving up the neck and involving all the fingers and things like that. So there’s… it really depends on what you wanna, what styles you wanna play.
Lex Fridman
So you’re thinking about song learning. So different components of song learning: so riffs in songs, lead-in songs.
So you’re thinking about song learning. So different components of song learning: so riffs in songs, lead-in songs.
Rick Beato
And then you have finger picking, if you have Stairway to Heaven, songs like that. How ’bout wanting to learn that? That involves finger picking, because you have to isolate certain notes of the chord and play two together, you know, and multiple times.
And then you have finger picking, if you have Stairway to Heaven, songs like that. How ’bout wanting to learn that? That involves finger picking, because you have to isolate certain notes of the chord and play two together, you know, and multiple times.
Lex Fridman
There’s a few crossroads where you get to select things. So I guess you’re speaking to the fact there’s a… if you’re righty, there’s a right hand that you can use your fingers or you can use a pick. And that’s a choice you make.
There’s a few crossroads where you get to select things. So I guess you’re speaking to the fact there’s a… if you’re righty, there’s a right hand that you can use your fingers or you can use a pick. And that’s a choice you make.
Rick Beato
And sometimes you use both, ’cause in Stairway to Heaven, you’re using the fingers at the beginning, or fingers and pick, hybrid, they call it hybrid picking, and then later on, you’re using the pick to flat pick the picking patterns.
And sometimes you use both, ’cause in Stairway to Heaven, you’re using the fingers at the beginning, or fingers and pick, hybrid, they call it hybrid picking, and then later on, you’re using the pick to flat pick the picking patterns.
Lex Fridman
On the music theory front, do you recommend people learn scales and chords and like the theory of it?
On the music theory front, do you recommend people learn scales and chords and like the theory of it?
Rick Beato
Later on, I would say. I wouldn’t say necessarily right off the bat. I think learning songs is the first thing that you should do ’cause you want to keep people motivated.
Later on, I would say. I wouldn’t say necessarily right off the bat. I think learning songs is the first thing that you should do ’cause you want to keep people motivated.
Lex Fridman
So you get them to like fall in love with music and playing? All right. And that takes a couple months, three months?
So you get them to like fall in love with music and playing? All right. And that takes a couple months, three months?
Rick Beato
Depends on how motivated they are.
Depends on how motivated they are.
Lex Fridman
So you recommend practicing, what, every day?
So you recommend practicing, what, every day?
Rick Beato
Every day. My son, Dylan, when he started learning the guitar a couple years ago, I said, “It’s better to practice 10 minutes a day, seven days a week than to practice one day for an hour, which is roughly the same amount of time.”
Every day. My son, Dylan, when he started learning the guitar a couple years ago, I said, “It’s better to practice 10 minutes a day, seven days a week than to practice one day for an hour, which is roughly the same amount of time.”
Lex Fridman
Yeah, but it usually turns into something longer. But otherwise, like, if you’re a busy life, you know, taking a day off… that day turns into a week, and then a week turns into a month, and all of a sudden you haven’t touched the instrument for months.
Yeah, but it usually turns into something longer. But otherwise, like, if you’re a busy life, you know, taking a day off… that day turns into a week, and then a week turns into a month, and all of a sudden you haven’t touched the instrument for months.
Rick Beato
Which is why I leave my guitar on a stand all the time, so that if I walk by it, I’m like, “Oh, okay, I’ll just pick it up for a second.” Then that second turns into 10 minutes, and an hour, two hours.
Which is why I leave my guitar on a stand all the time, so that if I walk by it, I’m like, “Oh, okay, I’ll just pick it up for a second.” Then that second turns into 10 minutes, and an hour, two hours.
Lex Fridman
All right, we gotta talk about this Dylan video. So this might be one of the earliest-
All right, we gotta talk about this Dylan video. So this might be one of the earliest-
Rick Beato
That’s the first one.
That’s the first one.
Lex Fridman
That’s the first video on the channel.
That’s the first video on the channel.
Rick Beato
It was actually before the channel, ’cause this actually blew up on Facebook-
It was actually before the channel, ’cause this actually blew up on Facebook-
Lex Fridman
Facebook
Rick Beato
… and then I put it on YouTube after.
… and then I put it on YouTube after.
Lex Fridman
So if it’s okay?
So if it’s okay?
Rick Beato
Yeah. Okay, Dylan, we’re gonna do the hardest ear training test of all time. Are you ready?
Yeah. Okay, Dylan, we’re gonna do the hardest ear training test of all time. Are you ready?
Lex Fridman
Ready. Oh.
Ready. Oh.
Rick Beato
Now, I… just a quick backstory on this. I made this for my friend Shane’s wife who wanted to see… ’cause Shane was a friend that I was producing, and he was there, and Dylan had come down the day, in the day, and I said, “Oh, check this out,” and I played this stuff. He’s like, “That’s amazing. Can you make a video so I can show my wife?” And I was on the way to a school board meeting, ’cause I was on the school board at Dylan’s school- … and I said, “Hey, Dylan, come downstairs. I want to make this video. It’ll take one minute, just need to do this thing for my friend, Shane.” And he’s like, “I don’t want to.” And I said, “Come on, this’ll take one minute.” “I don’t want to.” So I said to my wife, I’m like, “Nia, would you tell Dylan to come downstairs? I want to do this video.
Now, I… just a quick backstory on this. I made this for my friend Shane’s wife who wanted to see… ’cause Shane was a friend that I was producing, and he was there, and Dylan had come down the day, in the day, and I said, “Oh, check this out,” and I played this stuff. He’s like, “That’s amazing. Can you make a video so I can show my wife?” And I was on the way to a school board meeting, ’cause I was on the school board at Dylan’s school- … and I said, “Hey, Dylan, come downstairs. I want to make this video. It’ll take one minute, just need to do this thing for my friend, Shane.” And he’s like, “I don’t want to.” And I said, “Come on, this’ll take one minute.” “I don’t want to.” So I said to my wife, I’m like, “Nia, would you tell Dylan to come downstairs? I want to do this video.
Rick Beato
It’ll take one minute.” She’s like, “Dylan, go downstairs.” And he had, he has a mouthful of candy there- … ’cause he was eating candy. So if you look at him, he literally has a mouthful of candy while he’s doing this.
It’ll take one minute.” She’s like, “Dylan, go downstairs.” And he had, he has a mouthful of candy there- … ’cause he was eating candy. So if you look at him, he literally has a mouthful of candy while he’s doing this.
Lex Fridman
And we should say, on Facebook it went quite viral.
And we should say, on Facebook it went quite viral.
Rick Beato
Yeah, like got-… I don’t know, 80 million views. Something like… it had like 250,000 comments. Something like that. Insane.
Yeah, like got-… I don’t know, 80 million views. Something like… it had like 250,000 comments. Something like that. Insane.
Lex Fridman
How old is Dylan here?
How old is Dylan here?
Rick Beato
He’s eight.
He’s eight.
Lex Fridman
Eight years old? Can you actually give some more backstory about, like, how you discovered that Dylan has perfect pitch?
Eight years old? Can you actually give some more backstory about, like, how you discovered that Dylan has perfect pitch?
Rick Beato
So when Dylan was about two, he… I was doing a FaceTime with my brother Jon, and I was like, “Check this out, Jon.” And I played the Stone in Love, Neal Schon’s solo from Journey, and I was like, “Check this out.” And Dylan would sing along and my brother Jon was like, “Wow, Dylan can sing all the notes.” And I was like, “Yeah.” Then I played Black Dog, Zeppelin-
So when Dylan was about two, he… I was doing a FaceTime with my brother Jon, and I was like, “Check this out, Jon.” And I played the Stone in Love, Neal Schon’s solo from Journey, and I was like, “Check this out.” And Dylan would sing along and my brother Jon was like, “Wow, Dylan can sing all the notes.” And I was like, “Yeah.” Then I played Black Dog, Zeppelin-
Rick Beato
… and Dylan would sing that. And it’s like, “Dylan’s got a good ear.” Then Jon and I were like, “Well, we have good ears, too.” So it was probably… Maybe we could have done that when we were that age. So a couple years, more years goes by. Well, he was about three and a half, and I’m in the car. I was like, “Dylan, sing the Star Wars theme.” And he sings it, and I’m like, “That’s in the right key.” And I checked. I play it on my phone, and I was like, “Oh my gosh.” Then I ask him, “Play… Sing the Superman theme.” Because we’d been listening to John Williams soundtracks the week before, and he sings that. And that was in the right key. And I ask him another song. So I turn the car around, I go back to the studio.
… and Dylan would sing that. And it’s like, “Dylan’s got a good ear.” Then Jon and I were like, “Well, we have good ears, too.” So it was probably… Maybe we could have done that when we were that age. So a couple years, more years goes by. Well, he was about three and a half, and I’m in the car. I was like, “Dylan, sing the Star Wars theme.” And he sings it, and I’m like, “That’s in the right key.” And I checked. I play it on my phone, and I was like, “Oh my gosh.” Then I ask him, “Play… Sing the Superman theme.” Because we’d been listening to John Williams soundtracks the week before, and he sings that. And that was in the right key. And I ask him another song. So I turn the car around, I go back to the studio.
Rick Beato
I go to the piano, I hit the note B-flat, and Dylan says, “Star Wars.” Star Wars starts on a big B-flat major chord, but it’s the note B-flat is the main one that you hear. And then I play the note G, and he goes, “Superman.” And that’s the first note in the trumpet part of the- … of the Superman theme. And then I realized that he had perfect pitch, and then in five minutes, I taught him the name of the 12 notes. Which he already knew, but he just didn’t know the names.
I go to the piano, I hit the note B-flat, and Dylan says, “Star Wars.” Star Wars starts on a big B-flat major chord, but it’s the note B-flat is the main one that you hear. And then I play the note G, and he goes, “Superman.” And that’s the first note in the trumpet part of the- … of the Superman theme. And then I realized that he had perfect pitch, and then in five minutes, I taught him the name of the 12 notes. Which he already knew, but he just didn’t know the names.
Lex Fridman
Oh, so you just associate the names- … of the thing he knows. What do you think is this in his mind? ‘Cause it’s not just individual notes. He can, like, hear everything. What is that?
Oh, so you just associate the names- … of the thing he knows. What do you think is this in his mind? ‘Cause it’s not just individual notes. He can, like, hear everything. What is that?
Rick Beato
He doesn’t see colors. He just says every note sounds completely different.
He doesn’t see colors. He just says every note sounds completely different.
Lex Fridman
Wow. Like you said, maybe it’s a language thing. Because it really is a… He just learned the language.
Wow. Like you said, maybe it’s a language thing. Because it really is a… He just learned the language.
Rick Beato
Yeah, the language.
Yeah, the language.
Lex Fridman
There’s-
There’s-
Rick Beato
It’s like native music fluency, if you think of it like that.
It’s like native music fluency, if you think of it like that.
Lex Fridman
So let’s listen to some of this.
So let’s listen to some of this.
Rick Beato
Turn around. Here we go. As fast as you can, we’re going to start with single notes, then we’re going to do some intervals, then chords. Okay, here we go. A. C-sharp. B-flat. C. D. A-flat.
Turn around. Here we go. As fast as you can, we’re going to start with single notes, then we’re going to do some intervals, then chords. Okay, here we go. A. C-sharp. B-flat. C. D. A-flat.
Lex Fridman
Okay, good. Two notes at once. Here we go.
Okay, good. Two notes at once. Here we go.
Rick Beato
C-flat. Great. How about this? B-flat, A. Great. What about this? B-flat, A-flat.
C-flat. Great. How about this? B-flat, A. Great. What about this? B-flat, A-flat.
Lex Fridman
This is incredible.
This is incredible.
Rick Beato
Great. How about this? C, B-flat.
Great. How about this? C, B-flat.
Lex Fridman
And then how about this?
And then how about this?
Rick Beato
E-flat. What is it? E, E-flat. Correct. Okay. He’s annoyed. He’s annoyed. The part of this, when I play these next chords, that’s really I think why the video went so viral, the next part of this. Where I play these super complex polychords. Okay, I’m going to do some polychords for you. These are really going to be hard. You ready? What’s this? C augmented over D-flat augmented. Okay, sing a B-flat. Very good. What’s this chord? A-flat major over A major. Great, sing an F-sharp. Excellent. What’s this chord? A minor over D-flat major. Great. What’s this chord? E add9 over F major. Excellent. E add9 over F major. So I had to look at my hand to make sure that that’s what it was- … ’cause they’re all in inversions.
E-flat. What is it? E, E-flat. Correct. Okay. He’s annoyed. He’s annoyed. The part of this, when I play these next chords, that’s really I think why the video went so viral, the next part of this. Where I play these super complex polychords. Okay, I’m going to do some polychords for you. These are really going to be hard. You ready? What’s this? C augmented over D-flat augmented. Okay, sing a B-flat. Very good. What’s this chord? A-flat major over A major. Great, sing an F-sharp. Excellent. What’s this chord? A minor over D-flat major. Great. What’s this chord? E add9 over F major. Excellent. E add9 over F major. So I had to look at my hand to make sure that that’s what it was- … ’cause they’re all in inversions.
Rick Beato
So I think the reason that this went so viral is that the more that someone knew about music, the more that they shared the video. Because these polychords… So the people that were the best musicians looked at it and were like, “Oh my God.” You know, it’s C augmented over D-flat augmented. And the second chord was A-flat major over A major, but they were both in inversion, right? So it was like a first inversion A-flat major chord, first inversion A major chord. And then an A minor over D-flat major, and then E add9 over F major. And for an eight-year-old… I mean, for anyone- … plus they’re all close-voiced. They’re all just right next to each other.
So I think the reason that this went so viral is that the more that someone knew about music, the more that they shared the video. Because these polychords… So the people that were the best musicians looked at it and were like, “Oh my God.” You know, it’s C augmented over D-flat augmented. And the second chord was A-flat major over A major, but they were both in inversion, right? So it was like a first inversion A-flat major chord, first inversion A major chord. And then an A minor over D-flat major, and then E add9 over F major. And for an eight-year-old… I mean, for anyone- … plus they’re all close-voiced. They’re all just right next to each other.
Lex Fridman
Yeah, yeah.
Yeah, yeah.
Rick Beato
It’s not like, you know, where you can hear them clear. It’s all in the mid-range of the piano. So you have to really listen and you have to… He has to dissect each one. Like, what are the notes being played there, and what is… Like, what’s the theory? ‘Cause he’s actually using music theory- … to dissect them.
It’s not like, you know, where you can hear them clear. It’s all in the mid-range of the piano. So you have to really listen and you have to… He has to dissect each one. Like, what are the notes being played there, and what is… Like, what’s the theory? ‘Cause he’s actually using music theory- … to dissect them.
Lex Fridman
It must be in his brain, those components of the chords all sound different. Like—
It must be in his brain, those components of the chords all sound different. Like—
Rick Beato
Yes.
Yes.
Lex Fridman
… very clearly different. It’s truly incredible. The human mind’s incredible. So you’re saying, like, some part of that is the things you hear in the first few months of life?
… very clearly different. It’s truly incredible. The human mind’s incredible. So you’re saying, like, some part of that is the things you hear in the first few months of life?
Rick Beato
I did a thing where I played what I call high information music. High information music would be Bach, Well-Tempered Clavier, fugues, yeah, anything Bach. And I would play the Well-Tempered Clavier, and I would play… I have a friend who… Turkish pianist who’s one of the greatest improvisers I’ve ever heard. His name’s Aydan Esen. And I would play Aydan’s improvisations for Dylan. It had very sophisticated harmony and linear things in it. And Keith Jarrett, and mainly jazz, classical, and modern classical music. And then, then we would play, listen to rock music once he was born. I’m talking on my wife’s stomach before Dylan was born- … starting at 15 weeks, for 30 minutes a night. And then when Dylan was born, I would sit with him for an hour every morning and listen to music-
I did a thing where I played what I call high information music. High information music would be Bach, Well-Tempered Clavier, fugues, yeah, anything Bach. And I would play the Well-Tempered Clavier, and I would play… I have a friend who… Turkish pianist who’s one of the greatest improvisers I’ve ever heard. His name’s Aydan Esen. And I would play Aydan’s improvisations for Dylan. It had very sophisticated harmony and linear things in it. And Keith Jarrett, and mainly jazz, classical, and modern classical music. And then, then we would play, listen to rock music once he was born. I’m talking on my wife’s stomach before Dylan was born- … starting at 15 weeks, for 30 minutes a night. And then when Dylan was born, I would sit with him for an hour every morning and listen to music-
Rick Beato
… and I would look at him. In order for them to hear these phonemes apparently and develop this language, or get the … The language acquisition has to involve the social brain. So, when kids look at you, when a baby’s looking at you, they’re looking at your mouth and they’re getting social cues from that. And this is also another component of saying, “This is where this word stops or starts and stops. These are how the phonemes are separated from one another. These are how they’re connected.” So I believe that all kids are born with perfect pitch and then around nine months they begin to lose it. If you don’t engage their social brain, making these pitches know … I never played pitches for Dylan and said, “This is a C, this is a B-flat-” “… this is a G.” I just played complex-
… and I would look at him. In order for them to hear these phonemes apparently and develop this language, or get the … The language acquisition has to involve the social brain. So, when kids look at you, when a baby’s looking at you, they’re looking at your mouth and they’re getting social cues from that. And this is also another component of saying, “This is where this word stops or starts and stops. These are how the phonemes are separated from one another. These are how they’re connected.” So I believe that all kids are born with perfect pitch and then around nine months they begin to lose it. If you don’t engage their social brain, making these pitches know … I never played pitches for Dylan and said, “This is a C, this is a B-flat-” “… this is a G.” I just played complex-
Rick Beato
… high-information music for him. And played with him.
… high-information music for him. And played with him.
Lex Fridman
And that applies maybe even more generally to high-information language. And it starts before they’re born. I think I saw some of these incredible scientists that work on the neuroscience, the neurobiology, psychology of language in early life. I think a big part is in the mother’s stomach you’re listening to the mother speak.
And that applies maybe even more generally to high-information language. And it starts before they’re born. I think I saw some of these incredible scientists that work on the neuroscience, the neurobiology, psychology of language in early life. I think a big part is in the mother’s stomach you’re listening to the mother speak.
Rick Beato
Yes. That’s right.
Yes. That’s right.
Lex Fridman
So, like, that’s how on the language side you’re picking up the language already.
So, like, that’s how on the language side you’re picking up the language already.
Rick Beato
That’s right. And you’re picking up the music, musical language. So, native music fluency, you could call it.
That’s right. And you’re picking up the music, musical language. So, native music fluency, you could call it.
Lex Fridman
So if the mother’s sitting back and listening to Bach and some bebop jazz, you have a pretty good chance.
So if the mother’s sitting back and listening to Bach and some bebop jazz, you have a pretty good chance.
Rick Beato
Much better chance.
Much better chance.
Lex Fridman
Okay. All right. So that, as we unwind our way back Joe Pass and bebop. You were s- You were funny enough talking about what is bebop jazz, and that would be people like Joe Pass. And in your own life, your dad was somehow listening to that kind of incredibly complex and sophisticated music-
Okay. All right. So that, as we unwind our way back Joe Pass and bebop. You were s- You were funny enough talking about what is bebop jazz, and that would be people like Joe Pass. And in your own life, your dad was somehow listening to that kind of incredibly complex and sophisticated music-
Rick Beato
But wasn’t a musician.
But wasn’t a musician.
Lex Fridman
Wasn’t a musician.
Wasn’t a musician.
Rick Beato
Which was very weird. I… we never… My… I have six siblings and we could never figure out why Dad liked really sophisticated jazz.
Which was very weird. I… we never… My… I have six siblings and we could never figure out why Dad liked really sophisticated jazz.
Lex Fridman
We just took it for granted at that time.
We just took it for granted at that time.
Rick Beato
Yeah, just took it for granted. And my dad passed away in 2004 and we never really talked about that, but he and I used to listen to music together all the time. He’d… we’d put on a record, I’d sit on one side of the room, he’d sit on the other and not say a word. Listen through the whole side A. I’d go flip it over, listen to side B, never say a word. And then get up and go do stuff. And we did that all the time.
Yeah, just took it for granted. And my dad passed away in 2004 and we never really talked about that, but he and I used to listen to music together all the time. He’d… we’d put on a record, I’d sit on one side of the room, he’d sit on the other and not say a word. Listen through the whole side A. I’d go flip it over, listen to side B, never say a word. And then get up and go do stuff. And we did that all the time.
Lex Fridman
And so the first time you impressed your dad was with the Joe Pass song, right? And by the way, we’ll have to go to this song ’cause people must have forgot, right? People just think you’re like a good communicator or something. They don’t realize how good you are at guitar, how good you are at actually a lot of instruments, but guitar especially. And there’s this video, “The greatest guitar solo, period.” Can you give me some context for this particular intricate, complicated solo? Who’s Joe Pass?
And so the first time you impressed your dad was with the Joe Pass song, right? And by the way, we’ll have to go to this song ’cause people must have forgot, right? People just think you’re like a good communicator or something. They don’t realize how good you are at guitar, how good you are at actually a lot of instruments, but guitar especially. And there’s this video, “The greatest guitar solo, period.” Can you give me some context for this particular intricate, complicated solo? Who’s Joe Pass?
Rick Beato
Joe Pass was a guitarist. He lived from 1929 to 1994. And he was one of the greatest bebop players and solo guitar players. So he made a record that this is off of called Virtuoso in 1973 that my dad gave me for Christmas when I was in 10th grade. And he said… And this is not like my dad. My dad worked for the railroad. He was very, you know, few words spoken. Born in 1919. He said, “If you ever learn to play guitar like this, you’ve accomplished something with your life.” And I was like, “What?” So this record stayed… was unopened until about March after Christmas. And one day I was like, “Okay, I’ll open it up.” And I put it on, I start listening to it. And I was like, “Whoa, this is kinda cool.”
Joe Pass was a guitarist. He lived from 1929 to 1994. And he was one of the greatest bebop players and solo guitar players. So he made a record that this is off of called Virtuoso in 1973 that my dad gave me for Christmas when I was in 10th grade. And he said… And this is not like my dad. My dad worked for the railroad. He was very, you know, few words spoken. Born in 1919. He said, “If you ever learn to play guitar like this, you’ve accomplished something with your life.” And I was like, “What?” So this record stayed… was unopened until about March after Christmas. And one day I was like, “Okay, I’ll open it up.” And I put it on, I start listening to it. And I was like, “Whoa, this is kinda cool.”
Rick Beato
And so I said, “I think I can figure out some of this stuff.” So I figured out this thing.
And so I said, “I think I can figure out some of this stuff.” So I figured out this thing.
Lex Fridman
Is it by ear mostly?
Is it by ear mostly?
Rick Beato
Yeah, just by ear. I didn’t know any of the chords or anything.
Yeah, just by ear. I didn’t know any of the chords or anything.
Lex Fridman
If you can listen to a little bit here.
If you can listen to a little bit here.
Rick Beato
If you go back to that Brother to Brother, Gino Vannelli thing with Carlos Rios playing, that stuff is incredibly hard. This, I’m starting, I don’t know any of these chords. So I start out … I don’t even know what that chord is, but I figured it out. I just, and it’s weird. I mean, look at that weird bar.
If you go back to that Brother to Brother, Gino Vannelli thing with Carlos Rios playing, that stuff is incredibly hard. This, I’m starting, I don’t know any of these chords. So I start out … I don’t even know what that chord is, but I figured it out. I just, and it’s weird. I mean, look at that weird bar.
Lex Fridman
So you’re just finding like, playing around with the, putting your fingers- … on the various positions.
So you’re just finding like, playing around with the, putting your fingers- … on the various positions.
Rick Beato
Right, but trying every combination of fingers. I had never played that chord. That’s a weird-looking chord. And, but I kept … I moved my fingers around till I heard where it sounded like, “Oh, that’s it, definitely.” And I just looked at my hands like, “What is that?” Had no idea what it was.
Right, but trying every combination of fingers. I had never played that chord. That’s a weird-looking chord. And, but I kept … I moved my fingers around till I heard where it sounded like, “Oh, that’s it, definitely.” And I just looked at my hands like, “What is that?” Had no idea what it was.
Lex Fridman
So you were connected to the—you were really connected to the music. The … And so that’s why you can hear … It’s not necessarily … Did you even—you didn’t have perfect pitch.
So you were connected to the—you were really connected to the music. The … And so that’s why you can hear … It’s not necessarily … Did you even—you didn’t have perfect pitch.
Rick Beato
No.
No.
Lex Fridman
You, and not even relative pitch?
You, and not even relative pitch?
Rick Beato
No, I did not. No, I didn’t know anything about intervals. I didn’t know anything about music theory, anything. This is all just-
No, I did not. No, I didn’t know anything about intervals. I didn’t know anything about music theory, anything. This is all just-
Lex Fridman
Yeah. You’re just like playing-
Yeah. You’re just like playing-
Rick Beato
Ear
Ear
Lex Fridman
… around with different shapes. That’s amazing.
… around with different shapes. That’s amazing.
Rick Beato
That’s right. I mean, look at that weird bar there. But then you get into these things. So that stuff there, I could figure out … And then this. That stuff I could figure out. And then these things here. Those are just inversions of an—but I didn’t know that. I had heard Joe play that on the record. This is the last song on there. I’d listened to it a bunch of times and I started-
That’s right. I mean, look at that weird bar there. But then you get into these things. So that stuff there, I could figure out … And then this. That stuff I could figure out. And then these things here. Those are just inversions of an—but I didn’t know that. I had heard Joe play that on the record. This is the last song on there. I’d listened to it a bunch of times and I started-
Lex Fridman
So you just replay over and over and over and over, and you’re, like, trying to replicate it.
So you just replay over and over and over and over, and you’re, like, trying to replicate it.
Rick Beato
Yes. And I’m memorizing every different chord shape. All the chord shapes that I had never played before.
Yes. And I’m memorizing every different chord shape. All the chord shapes that I had never played before.
Lex Fridman
Would you recommend people do something like that on a really complicated song?
Would you recommend people do something like that on a really complicated song?
Rick Beato
Yeah, but there are so many YouTube videos that you can go and just learn it without having to—Yes. Yeah, I would recommend.
Yeah, but there are so many YouTube videos that you can go and just learn it without having to—Yes. Yeah, I would recommend.
Lex Fridman
I feel like the struggle-
I feel like the struggle-
Rick Beato
The struggle is where it’s at.
The struggle is where it’s at.
Lex Fridman
… this is true for education in general. People… Like, there’s all these educators that try to make learning easier and more fun, and all that kind of stuff. Great, wonderful, but part of the thing is the struggle.
… this is true for education in general. People… Like, there’s all these educators that try to make learning easier and more fun, and all that kind of stuff. Great, wonderful, but part of the thing is the struggle.
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
But yeah, let’s—
But yeah, let’s—
Rick Beato
I’m sorry, hearing there’s .
I’m sorry, hearing there’s .
Lex Fridman
Let’s… You’re nuts.
Let’s… You’re nuts.
Rick Beato
I heard licks like that all over this, so I knew that that was… and then these licks here, he plays a lot of ideas like that. That’s basically a C9 chord in the top notes of it. So all these are just inversions of the same chord. So if I could play that, then it’s just figuring out the single notes, okay? So… Okay, so if you just take this first part here when he goes… So this intro part is…
I heard licks like that all over this, so I knew that that was… and then these licks here, he plays a lot of ideas like that. That’s basically a C9 chord in the top notes of it. So all these are just inversions of the same chord. So if I could play that, then it’s just figuring out the single notes, okay? So… Okay, so if you just take this first part here when he goes… So this intro part is…
Lex Fridman
You make it sound so simple when you break it down. And, and by the way, Joe Pass, incredible guitar player. Like, this is obvious. .
You make it sound so simple when you break it down. And, and by the way, Joe Pass, incredible guitar player. Like, this is obvious. .
Rick Beato
And he improvised all this. He could have played it like this.
And he improvised all this. He could have played it like this.
Lex Fridman
But, you know, the first was the individual notes. Look at that.
But, you know, the first was the individual notes. Look at that.
Rick Beato
Ooh, that’s hard. Maybe just play it like that. That sounds more realistic.
Ooh, that’s hard. Maybe just play it like that. That sounds more realistic.
Lex Fridman
The amount of different genres that you’re able to replicate is just incredible.
The amount of different genres that you’re able to replicate is just incredible.
Rick Beato
This is just taking the needle, moving it there, then going back a little, oh, there. And then by the end, the record was so scratched. It was—but it was worth it. When I played it for my dad— … he couldn’t believe. I mean, he didn’t say, “That’s amazing.” He was just like, “Hmm, pretty good.”
This is just taking the needle, moving it there, then going back a little, oh, there. And then by the end, the record was so scratched. It was—but it was worth it. When I played it for my dad— … he couldn’t believe. I mean, he didn’t say, “That’s amazing.” He was just like, “Hmm, pretty good.”
Miles Davis
Lex Fridman
So what was the role of bebop jazz in the history of music? It seems like it was influential in your life. Another guy you had an incredible interview with: Flea. People should go listen to that. Was a great conversation. One of the things that surprised me is just how many musical genres influenced Flea. And the guy showed up in a Miles Davis T-shirt.
So what was the role of bebop jazz in the history of music? It seems like it was influential in your life. Another guy you had an incredible interview with: Flea. People should go listen to that. Was a great conversation. One of the things that surprised me is just how many musical genres influenced Flea. And the guy showed up in a Miles Davis T-shirt.
Rick Beato
That’s right.
That’s right.
Lex Fridman
And-
And-
Rick Beato
Bebop.
Bebop.
Lex Fridman
And –
And –
Rick Beato
Miles Davis played with Charlie Parker- … when he was 18 years old. And that’s… He was… Charlie Parker was really his mentor.
Miles Davis played with Charlie Parker- … when he was 18 years old. And that’s… He was… Charlie Parker was really his mentor.
Lex Fridman
Can you explain to me why, with many of the folks you’ve interviewed and in general out there, in the world of jazz, all roads lead to Miles Davis? Why he’s such an influential figure?
Can you explain to me why, with many of the folks you’ve interviewed and in general out there, in the world of jazz, all roads lead to Miles Davis? Why he’s such an influential figure?
Rick Beato
Because he was the greatest innovator in the history of jazz. He was at the forefront of all these different styles of jazz. I mean, he started as a bebop player, and then he had records like the Birth of the Cool, and modal jazz, and hard bop, and records like Bitches Brew, where he started to, I guess you would call it fusion. You start to get these records. You had two main groups of Miles Davis. You had the Miles Davis ’50s quintet and the Miles Davis ’60s quintet.
Because he was the greatest innovator in the history of jazz. He was at the forefront of all these different styles of jazz. I mean, he started as a bebop player, and then he had records like the Birth of the Cool, and modal jazz, and hard bop, and records like Bitches Brew, where he started to, I guess you would call it fusion. You start to get these records. You had two main groups of Miles Davis. You had the Miles Davis ’50s quintet and the Miles Davis ’60s quintet.
Rick Beato
Now, Miles made records with many people, but the ’50s quintet had John Coltrane in it. Had, I mean, had different piano players—Wynton Kelly—but Paul Chambers on the bass, Philly Joe Jones on the drums. And that particular group made just incredibly important records. And then he had his ’60s group, which was Herbie Hancock on the piano, Ron Carter on the bass, Tony Williams on the drums, and Wayne Shorter on the saxophone. And they made all these incredibly important records.
Now, Miles made records with many people, but the ’50s quintet had John Coltrane in it. Had, I mean, had different piano players—Wynton Kelly—but Paul Chambers on the bass, Philly Joe Jones on the drums. And that particular group made just incredibly important records. And then he had his ’60s group, which was Herbie Hancock on the piano, Ron Carter on the bass, Tony Williams on the drums, and Wayne Shorter on the saxophone. And they made all these incredibly important records.
Lex Fridman
I forget who said it in an interview with you, but they talked about like Miles Davis, his music feeling like I think toes hanging over the cliff or something like this. Meaning, like, there’s always a risk, there’s a danger that you’re willing to make, to fuck it all up live. And that feeling is what creates the aliveness of the music. Like, can you speak to that? Just the creating in the music, the feeling like you’re on the edge. Like, you’re challenging the possibilities of what can happen, and it all can go to shit, and because of that, it feels alive.
I forget who said it in an interview with you, but they talked about like Miles Davis, his music feeling like I think toes hanging over the cliff or something like this. Meaning, like, there’s always a risk, there’s a danger that you’re willing to make, to fuck it all up live. And that feeling is what creates the aliveness of the music. Like, can you speak to that? Just the creating in the music, the feeling like you’re on the edge. Like, you’re challenging the possibilities of what can happen, and it all can go to shit, and because of that, it feels alive.
Rick Beato
Well, when I interviewed Ron Carter that played in Miles’s ’60s quintet, I asked Ron, ’cause Ron played bass on 2,200 recordings, famous records. And I said, “Did you guys ever rehearse with Miles?” “No, never.” I said, “So, what would you do?” He goes, “We’d just show up at the studio, and he’d have the charts, put them on the stand and we would just roll.”
Well, when I interviewed Ron Carter that played in Miles’s ’60s quintet, I asked Ron, ’cause Ron played bass on 2,200 recordings, famous records. And I said, “Did you guys ever rehearse with Miles?” “No, never.” I said, “So, what would you do?” He goes, “We’d just show up at the studio, and he’d have the charts, put them on the stand and we would just roll.”
Rick Beato
And I said, “Would you listen to it after?” “No.” And I said, “Well, what about the live records that you did, when you’d record at clubs and things like that?” He goes, “We never knew that we were recording.” He goes, “Maybe I’d see a microphone, a different kind of microphone in my bass amp.” He goes, “Then months later, a record would come out and I’d see it, and I was on it, and I would take it down to the union and say, ‘I played on this record,’ so you get paid for it.” But he said, “We didn’t even know we were recording.” So Miles was always about, you know, don’t think about it, just play.
And I said, “Would you listen to it after?” “No.” And I said, “Well, what about the live records that you did, when you’d record at clubs and things like that?” He goes, “We never knew that we were recording.” He goes, “Maybe I’d see a microphone, a different kind of microphone in my bass amp.” He goes, “Then months later, a record would come out and I’d see it, and I was on it, and I would take it down to the union and say, ‘I played on this record,’ so you get paid for it.” But he said, “We didn’t even know we were recording.” So Miles was always about, you know, don’t think about it, just play.
Lex Fridman
That’s crazy. That was on purpose. That was done on purpose. Not to do the rehearsals. None of that.
That’s crazy. That was on purpose. That was done on purpose. Not to do the rehearsals. None of that.
Rick Beato
Yeah, he wanted people to just feel it, play it. Thought is the enemy of flow, as Vinnie Colaiuta told me.
Yeah, he wanted people to just feel it, play it. Thought is the enemy of flow, as Vinnie Colaiuta told me.
Lex Fridman
Thought is the enemy of flow. How do you make sense that Flea, the bassist for the Red Hot Chili Peppers, is influenced by bebop jazz?
Thought is the enemy of flow. How do you make sense that Flea, the bassist for the Red Hot Chili Peppers, is influenced by bebop jazz?
Rick Beato
So his stepfather was a jazz bass player. And his… When his parents got divorced, he was born in Australia, and then they moved to New York. Then his parents got divorced, and his mom married his stepfather, who was a jazz musician. And they used to have jam sessions at their place, and Flea loved it. It was kind of like my upbringing with my dad, playing jazz all the time. Once it gets inside you, it’s just there. And so he is heavily influenced by jazz musicians.
So his stepfather was a jazz bass player. And his… When his parents got divorced, he was born in Australia, and then they moved to New York. Then his parents got divorced, and his mom married his stepfather, who was a jazz musician. And they used to have jam sessions at their place, and Flea loved it. It was kind of like my upbringing with my dad, playing jazz all the time. Once it gets inside you, it’s just there. And so he is heavily influenced by jazz musicians.
Lex Fridman
Yeah, his impression was just hilarious. I mean, he’s a character. His whole physical way of being is a character. And his impression of just upright bass is just fun to watch. His whole-
Yeah, his impression was just hilarious. I mean, he’s a character. His whole physical way of being is a character. And his impression of just upright bass is just fun to watch. His whole-
Rick Beato
His intensity when he picked up his bass during the interview… He’s an intense guy and funny, and you know, really emotional. And he picks up his bass, and there’s a fierceness that you immediately feel. And he talks about how he practices. And then when he starts doing the slapping stuff, he gets so into it. And I’m just sitting there going, “Whoa.” Like, “Wow.”
His intensity when he picked up his bass during the interview… He’s an intense guy and funny, and you know, really emotional. And he picks up his bass, and there’s a fierceness that you immediately feel. And he talks about how he practices. And then when he starts doing the slapping stuff, he gets so into it. And I’m just sitting there going, “Whoa.” Like, “Wow.”
Bass guitar
Lex Fridman
Yeah, he talked about his practicing routine with you. And one of the things, he’s like, “I have to practice the slap.” And- … you know, there’s differences in the structure of the different bands. But usually, like, the bassist has a vibe to them. I don’t know if we can put words to exactly what that is. There’s a kind of energy that drives the band.
Yeah, he talked about his practicing routine with you. And one of the things, he’s like, “I have to practice the slap.” And- … you know, there’s differences in the structure of the different bands. But usually, like, the bassist has a vibe to them. I don’t know if we can put words to exactly what that is. There’s a kind of energy that drives the band.
Rick Beato
To me, the bass is one of the only instruments that, when you play a bad note, everybody notices. I started on the bass- … as a kid.
To me, the bass is one of the only instruments that, when you play a bad note, everybody notices. I started on the bass- … as a kid.
Lex Fridman
Oh, interesting. But you also played drums. You also played-
Oh, interesting. But you also played drums. You also played-
Rick Beato
Yeah, but my first instrument was the cello in third grade. And then I switched to the bass in sixth grade. And my, I majored, my undergrad degree is in classical bass. So I always think of myself as a bass player first. And I always think the bass is the most important instrument because-
Yeah, but my first instrument was the cello in third grade. And then I switched to the bass in sixth grade. And my, I majored, my undergrad degree is in classical bass. So I always think of myself as a bass player first. And I always think the bass is the most important instrument because-
Lex Fridman
Strong words.
Strong words.
Rick Beato
… because as much as I love to play the guitar, and I love to play the guitar more than anything, I think, but the bass really defines what the quality of the chord is. ‘Cause you can put the root in there. You can put the third of the chord in the bass. You can put the fifth in there. You can play a lot of notes. And whatever you play in the bass kinda defines what kind of chord it is. So, the bass player has a lot of power.
… because as much as I love to play the guitar, and I love to play the guitar more than anything, I think, but the bass really defines what the quality of the chord is. ‘Cause you can put the root in there. You can put the third of the chord in the bass. You can put the fifth in there. You can play a lot of notes. And whatever you play in the bass kinda defines what kind of chord it is. So, the bass player has a lot of power.
Greatest guitar solos of all time
Lex Fridman
I have to go back to our, the beginning of our conversation. What do you think are some of the great solos of all time? Can we put a few into consideration? You have a great list of the top 20 rock guitar solos of all time.
I have to go back to our, the beginning of our conversation. What do you think are some of the great solos of all time? Can we put a few into consideration? You have a great list of the top 20 rock guitar solos of all time.
Rick Beato
Yeah, so I put Comfortably Numb as my favorite, as my top one.
Yeah, so I put Comfortably Numb as my favorite, as my top one.
Lex Fridman
Yeah, on that day, right?
Yeah, on that day, right?
Rick Beato
On that day. Right. Now the day later, I would have said, “It’s the second solo.” But I did the first solo because nobody talks about that solo. And that solo is equally great. And when David Gilmour… When I played it for him, and we talked about it in my interview with him, it was… Just to watch his face when he listened to it was incredible. I mean, I’m thinking to myself, it’s like, I’m sitting with David Gilmour, and he’s listening to Comfortably Numb. And he’s hearing it. He’s played it a million times live, but how many times has he gone back and listened to it on the record? Probably not for a long time. And then he’s hearing it, and he’s like, “Ooh.”
On that day. Right. Now the day later, I would have said, “It’s the second solo.” But I did the first solo because nobody talks about that solo. And that solo is equally great. And when David Gilmour… When I played it for him, and we talked about it in my interview with him, it was… Just to watch his face when he listened to it was incredible. I mean, I’m thinking to myself, it’s like, I’m sitting with David Gilmour, and he’s listening to Comfortably Numb. And he’s hearing it. He’s played it a million times live, but how many times has he gone back and listened to it on the record? Probably not for a long time. And then he’s hearing it, and he’s like, “Ooh.”
Lex Fridman
Maybe you just don’t look back. When you do great things, you don’t look back.
Maybe you just don’t look back. When you do great things, you don’t look back.
Rick Beato
Miles never looked back. He never wanted to hear the old stuff. He always moved on.
Miles never looked back. He never wanted to hear the old stuff. He always moved on.
Lex Fridman
There was this funny moment where you made a video why David Gilmour will never be on the channel. And then you ended up, of course, interviewing him twice. He’s one of the greatest guitar players of all time. What do you think is at the core of his genius?
There was this funny moment where you made a video why David Gilmour will never be on the channel. And then you ended up, of course, interviewing him twice. He’s one of the greatest guitar players of all time. What do you think is at the core of his genius?
Rick Beato
He has just an incredible melodic sense. He knows how phrases should be put together. There’s a flow to his ideas that I think is just incredible. It’s the same with Hendrix. This flow, how one idea leads to the next, how there’s space between them. It’s just like speaking.
He has just an incredible melodic sense. He knows how phrases should be put together. There’s a flow to his ideas that I think is just incredible. It’s the same with Hendrix. This flow, how one idea leads to the next, how there’s space between them. It’s just like speaking.
Lex Fridman
That’s what I read about Miles Davis is, he’s very good at understanding tempo and the value of silence. And I think David Gilmour doesn’t always play fast. But he does a lot with less. And then some of that is also on the more technical side, probably the tone of the… I mean, he has one of the most uniquely recognizable tones in all of music. What do you understand about what it takes to shape the tone that is David Gilmour?
That’s what I read about Miles Davis is, he’s very good at understanding tempo and the value of silence. And I think David Gilmour doesn’t always play fast. But he does a lot with less. And then some of that is also on the more technical side, probably the tone of the… I mean, he has one of the most uniquely recognizable tones in all of music. What do you understand about what it takes to shape the tone that is David Gilmour?
Rick Beato
He has a very sophisticated setup- … for his tone, and that was one of the things when I went to his studio. And I said to him, “So David, is there anything I’m not supposed to see here?” I mean, he never sits down and shows- … people his gear, and he laughed about it.
He has a very sophisticated setup- … for his tone, and that was one of the things when I went to his studio. And I said to him, “So David, is there anything I’m not supposed to see here?” I mean, he never sits down and shows- … people his gear, and he laughed about it.
Rick Beato
But there I am, sitting there right next to all these pedals that… And I asked his tech, Phil, I said, “These are the same ones you used on the records?” He’s like, “Yeah.” His tech has been with him for, like, 50 years. And I mean, the exact ones? Yes. It’s just, it’s hard to… It’s hard to imagine that those things still… Of course, though. He’s just kept it. Yeah, this is his Binson Echorec that he played through, and this is this. You know, these are all the same effects pedals. And the… Wait, is this the same Hiwatt amp? Yeah. Is this the same… Yes. Yeah, you get some new stuff. But they keep all their own gear, and that’s… I mean, he does sell his guitars for charity.
But there I am, sitting there right next to all these pedals that… And I asked his tech, Phil, I said, “These are the same ones you used on the records?” He’s like, “Yeah.” His tech has been with him for, like, 50 years. And I mean, the exact ones? Yes. It’s just, it’s hard to… It’s hard to imagine that those things still… Of course, though. He’s just kept it. Yeah, this is his Binson Echorec that he played through, and this is this. You know, these are all the same effects pedals. And the… Wait, is this the same Hiwatt amp? Yeah. Is this the same… Yes. Yeah, you get some new stuff. But they keep all their own gear, and that’s… I mean, he does sell his guitars for charity.
Rick Beato
But, like, he has a black Strat that is a, it’s a signature version. It’s like an exact copy of his old one. So to him, it sounds exactly the same, plays the same.
But, like, he has a black Strat that is a, it’s a signature version. It’s like an exact copy of his old one. So to him, it sounds exactly the same, plays the same.
Lex Fridman
Well, of course, they converge towards that kind of hardware. But there’s so many tiny details over the years. You see the final result of it, but there’s a, there’s a journey there, of exploring. And of course, he’s not… I guess he’s not doing any soft… Like, no emulation, no amp?
Well, of course, they converge towards that kind of hardware. But there’s so many tiny details over the years. You see the final result of it, but there’s a, there’s a journey there, of exploring. And of course, he’s not… I guess he’s not doing any soft… Like, no emulation, no amp?
Rick Beato
He does do emulation, actually. He does. He has this thing, this is… I asked him in the first interview about this. There’s a little rack thing that I had heard that he used, but I asked him for sure. It’s called the Zoom 9030. I put out a short where he talks about it. I said, “So, that Zoom 9030, is that a real thing?” ‘Cause I’ve read about it. He’s like, “Yeah.” And he talks about how, when he’s sitting there recording on his own… And he runs Pro Tools himself, and so he’ll be sitting there. There’s no one there to help him. He’s like, “I’ll just plug into this thing, and then I’ll play a solo with this model.” It’s like a kind of ’90s modeling, early modeling thing.
He does do emulation, actually. He does. He has this thing, this is… I asked him in the first interview about this. There’s a little rack thing that I had heard that he used, but I asked him for sure. It’s called the Zoom 9030. I put out a short where he talks about it. I said, “So, that Zoom 9030, is that a real thing?” ‘Cause I’ve read about it. He’s like, “Yeah.” And he talks about how, when he’s sitting there recording on his own… And he runs Pro Tools himself, and so he’ll be sitting there. There’s no one there to help him. He’s like, “I’ll just plug into this thing, and then I’ll play a solo with this model.” It’s like a kind of ’90s modeling, early modeling thing.
Rick Beato
And he’ll play a solo, and then after a while, you hear the solo, and it’s like, “Well, I’m not gonna replay that. That sounds great.” You get used to the sound of it, and that’s what it is. So people always talked about, “Oh, well, he couldn’t have used that. He’s recording through an amp,” and… ‘Cause it sounds great. And then he’s like, “Yeah, yeah, so that’s what I use.” And then I have the video of it right there, and it says his presets, DG1 and DG2 and, you know, whatever.
And he’ll play a solo, and then after a while, you hear the solo, and it’s like, “Well, I’m not gonna replay that. That sounds great.” You get used to the sound of it, and that’s what it is. So people always talked about, “Oh, well, he couldn’t have used that. He’s recording through an amp,” and… ‘Cause it sounds great. And then he’s like, “Yeah, yeah, so that’s what I use.” And then I have the video of it right there, and it says his presets, DG1 and DG2 and, you know, whatever.
Lex Fridman
What’s your process for preparing for interviews like that? You’ve done a few legendary people.
What’s your process for preparing for interviews like that? You’ve done a few legendary people.
Rick Beato
I never prepare for interviews, because I ask people things that I’m interested in knowing.
I never prepare for interviews, because I ask people things that I’m interested in knowing.
Lex Fridman
So, just letting your curiosity just pull a-
So, just letting your curiosity just pull a-
Rick Beato
Yes
Yes
Lex Fridman
… pull you forward?
… pull you forward?
Rick Beato
And I can think of 100 questions to ask David Gilmour… but I always ask my questions based on what they say to me.
And I can think of 100 questions to ask David Gilmour… but I always ask my questions based on what they say to me.
Lex Fridman
Yeah.
Yeah.
Rick Beato
So, but I do make a playlist of songs that I wanna talk about. So, that kind of guides me… ‘Cause I wanna make sure that I… There’s specific things that I need to play to, so that you can jog his memory. ‘Cause anytime you play something that somebody recorded, even 50 years ago, they’ll remember. If they don’t remember the exact specifics, that brings it to life to them again. And they can kind of piece together some aspects about it, and they can really talk. He can talk about the phrasing and the, you know, the kind of melodic direction of things like that.
So, but I do make a playlist of songs that I wanna talk about. So, that kind of guides me… ‘Cause I wanna make sure that I… There’s specific things that I need to play to, so that you can jog his memory. ‘Cause anytime you play something that somebody recorded, even 50 years ago, they’ll remember. If they don’t remember the exact specifics, that brings it to life to them again. And they can kind of piece together some aspects about it, and they can really talk. He can talk about the phrasing and the, you know, the kind of melodic direction of things like that.
Lex Fridman
So, there’s a lot of tiny details that go into a particular song, whether it’s in the production or how it’s played or how it was composed, all that kind of stuff. And you don’t know what those are ahead of time.
So, there’s a lot of tiny details that go into a particular song, whether it’s in the production or how it’s played or how it was composed, all that kind of stuff. And you don’t know what those are ahead of time.
Rick Beato
No.
No.
Lex Fridman
You just know the song, and you just are looking to jog their memory, and maybe your own curiosity of like, “How did you do this?” Or, “How do, what, this sound or that?” You make it look easy, but you have to have a depth of knowledge. You’re saying you don’t prepare.
You just know the song, and you just are looking to jog their memory, and maybe your own curiosity of like, “How did you do this?” Or, “How do, what, this sound or that?” You make it look easy, but you have to have a depth of knowledge. You’re saying you don’t prepare.
Rick Beato
I have an incredibly good memory.
I have an incredibly good memory.
Lex Fridman
Exactly.
Exactly.
Rick Beato
That’s what it is. It’s that I can remember when records came out, who produced them, where they recorded them, who was the engineer, what songs are on it. And not only that, but the people I’m interviewing know that I can play all the parts- … of all the instruments, ’cause I’ve done breakdowns of their songs, which is why I get the interviews with them in the first place, really.
That’s what it is. It’s that I can remember when records came out, who produced them, where they recorded them, who was the engineer, what songs are on it. And not only that, but the people I’m interviewing know that I can play all the parts- … of all the instruments, ’cause I’ve done breakdowns of their songs, which is why I get the interviews with them in the first place, really.
Lex Fridman
But the actual, like, the skill of the interview, the thing you’re not saying, the preparation, is you listening to bebop.
But the actual, like, the skill of the interview, the thing you’re not saying, the preparation, is you listening to bebop.
Rick Beato
That’s right.
That’s right.
Lex Fridman
It’s the background knowledge, it’s the soul carrying with you, being able to radiate the love of the soul of music.
It’s the background knowledge, it’s the soul carrying with you, being able to radiate the love of the soul of music.
Rick Beato
I will say this, Lex, is that the other thing is that most of these people have a really good sense of humor. When I was, when… The first time I interviewed David in New York, my brother John came along, and he is a massive David Gilmour fan. That’s his biggest influence as a guitar player. And so he said, “You’re interviewing David Gilmour? Oh, I’m coming.” I was like, “All right. Come on. Come on down.” So my brother John’s standing about five feet away. And John is a sales guy, but he… Great guitar player. So John’s like… I was like, “This is David, this is my brother, John.” “David, great to meet you, buddy.” And you know, it sounds like it’s so… He’s a sales guy. And so during the interview, I said, I was like, “Hey, John, what was I gonna ask David?”
I will say this, Lex, is that the other thing is that most of these people have a really good sense of humor. When I was, when… The first time I interviewed David in New York, my brother John came along, and he is a massive David Gilmour fan. That’s his biggest influence as a guitar player. And so he said, “You’re interviewing David Gilmour? Oh, I’m coming.” I was like, “All right. Come on. Come on down.” So my brother John’s standing about five feet away. And John is a sales guy, but he… Great guitar player. So John’s like… I was like, “This is David, this is my brother, John.” “David, great to meet you, buddy.” And you know, it sounds like it’s so… He’s a sales guy. And so during the interview, I said, I was like, “Hey, John, what was I gonna ask David?”
Rick Beato
Oh, ask him about the Gilmour effect.” “Oh, yeah, that’s right.” And the Gilmour effect is my thing that I say in the comments section when people say… Anytime anybody plays anything technical, “Oh, yeah, that’s great, but I much prefer David Gilmour.” And so I always call it the Gilmour effect. Anytime I have, like, Yngwie Malmsteen- Anybody that played, that has chops that I- … interview, the, the, the negative comments are always, “Well, I prefer David Gilmour.”
Oh, ask him about the Gilmour effect.” “Oh, yeah, that’s right.” And the Gilmour effect is my thing that I say in the comments section when people say… Anytime anybody plays anything technical, “Oh, yeah, that’s great, but I much prefer David Gilmour.” And so I always call it the Gilmour effect. Anytime I have, like, Yngwie Malmsteen- Anybody that played, that has chops that I- … interview, the, the, the negative comments are always, “Well, I prefer David Gilmour.”
Lex Fridman
Yeah, yeah.
Yeah, yeah.
Rick Beato
And I said that, I told David that. He’s like, “Well, maybe they should keep their opinions to themselves.”
And I said that, I told David that. He’s like, “Well, maybe they should keep their opinions to themselves.”
Lex Fridman
Yeah, a lot of these folks have really wonderful personalities, with a trusted person to be able to reveal that personality. So, Comfortably Numb at the top on that day. What else is up there?
Yeah, a lot of these folks have really wonderful personalities, with a trusted person to be able to reveal that personality. So, Comfortably Numb at the top on that day. What else is up there?
Rick Beato
Stairway to Heaven. Hey Joe.
Stairway to Heaven. Hey Joe.
Lex Fridman
But in that list, your top Hendrix solo is Hey Joe?
But in that list, your top Hendrix solo is Hey Joe?
Rick Beato
It’s the first guitar solo I ever learned, so I had to put it on there. So, I don’t necessarily do these by… I do those in kind of how important they are to me and my development. So, there’s always a biographical component to these lists. Number three was Kid Charlemagne, a Steely Dan solo, Larry Carlton. Amazing solo, extremely difficult to figure out.
It’s the first guitar solo I ever learned, so I had to put it on there. So, I don’t necessarily do these by… I do those in kind of how important they are to me and my development. So, there’s always a biographical component to these lists. Number three was Kid Charlemagne, a Steely Dan solo, Larry Carlton. Amazing solo, extremely difficult to figure out.
Rick Beato
Probably, there’s two solos on the list that are just about, are very… That one I can play. But there’s a few solos that are very hard to play. Stone in Love by Journey, by Neal Schon, is very hard to play some licks. There’s a song… There’s a solo by a guitarist, Carlos Rios, that people don’t know. It’s Brother to Brother, a Gino Vannelli song, but it’s very hard to play and figure out. And that people don’t know the solos. I put it on my list ’cause I knew that a lot of people were gonna watch it and they’re gonna know what this solo is.
Probably, there’s two solos on the list that are just about, are very… That one I can play. But there’s a few solos that are very hard to play. Stone in Love by Journey, by Neal Schon, is very hard to play some licks. There’s a song… There’s a solo by a guitarist, Carlos Rios, that people don’t know. It’s Brother to Brother, a Gino Vannelli song, but it’s very hard to play and figure out. And that people don’t know the solos. I put it on my list ’cause I knew that a lot of people were gonna watch it and they’re gonna know what this solo is.
Lex Fridman
For me, the sentimental one, my first solo is Mr. Crowley, Randy Rhoads. I like the musicality of Mr. Crowley, that there is a melodic component to it. You’re playing really fast, but there’s a melody to it. And also, there’s like a legendary nature to the brief time we had Randy Rhoads.
For me, the sentimental one, my first solo is Mr. Crowley, Randy Rhoads. I like the musicality of Mr. Crowley, that there is a melodic component to it. You’re playing really fast, but there’s a melody to it. And also, there’s like a legendary nature to the brief time we had Randy Rhoads.
Rick Beato
Yes.
Yes.
Lex Fridman
It’s probably one of the greatest guitarists ever.
It’s probably one of the greatest guitarists ever.
Rick Beato
’56 to ’82, I think. Terrible. He was an absolute brilliant guitarist, had his own style.
’56 to ’82, I think. Terrible. He was an absolute brilliant guitarist, had his own style.
Lex Fridman
We should say he’s the guitarist for Ozzy Osbourne, the band.
We should say he’s the guitarist for Ozzy Osbourne, the band.
Rick Beato
Yeah. And that Mr. Crowley solo is a, is a great solo, great solo. And he’s incredibly influential as a guitar player too, for metal guitar players and I love Randy Rhoads.
Yeah. And that Mr. Crowley solo is a, is a great solo, great solo. And he’s incredibly influential as a guitar player too, for metal guitar players and I love Randy Rhoads.
Lex Fridman
Another guy, so one of my favorites is Mark Knopfler.
Another guy, so one of my favorites is Mark Knopfler.
Rick Beato
Yes. And I did have Mark Knopfler on my list, Sultans of Swing.
Yes. And I did have Mark Knopfler on my list, Sultans of Swing.
Lex Fridman
That’s right, you did have-
That’s right, you did have-
Rick Beato
Now, I had it high on the list, and I’ll tell you why. I would’ve had it lower ’cause it’s one of the early ones, ’cause I wanted people to be like, “Okay, oh, this is a serious list.” So Rick’s gonna talk about serious stuff. So- And Rick’s gonna play along with all these things. So I wanted to kind of state that at the beginning of the video. I mean, I made the video in one day to do 20 solos. I think I played 19 of them, but the Heart solo that I had on there- … Nancy Wilson, I played the video of. And I tried to get a couple of my friends to play the Ice Cream Man, Van Halen solo.
Now, I had it high on the list, and I’ll tell you why. I would’ve had it lower ’cause it’s one of the early ones, ’cause I wanted people to be like, “Okay, oh, this is a serious list.” So Rick’s gonna talk about serious stuff. So- And Rick’s gonna play along with all these things. So I wanted to kind of state that at the beginning of the video. I mean, I made the video in one day to do 20 solos. I think I played 19 of them, but the Heart solo that I had on there- … Nancy Wilson, I played the video of. And I tried to get a couple of my friends to play the Ice Cream Man, Van Halen solo.
Lex Fridman
Yeah, it was just-
Yeah, it was just-
Rick Beato
So I called Dweezil Zappa, and I was like, “Dweezil, can you play the Ice Cream Man solo? I’m making a video about it.” He’s like, “Oh, I’d have to practice that.” Then I called my friend Phil X who’s an amazing guitar player, and he’s like, “No, I’d have to practice that.” I was like, “Come on, man, can’t let me play Ice Cream Man?” The opening lick of Ice Cream Man that he plays is very hard to play ’cause it’s an incredibly long stretch. And it hurt my fingers to do, and Eddie would turn his guitar up like this to play. And plus, it’s a tricky… It just… It’s a tricky rhythm, and it’s such a big stretch. It’s like, “Man, I can’t… That hurts my hand.”
So I called Dweezil Zappa, and I was like, “Dweezil, can you play the Ice Cream Man solo? I’m making a video about it.” He’s like, “Oh, I’d have to practice that.” Then I called my friend Phil X who’s an amazing guitar player, and he’s like, “No, I’d have to practice that.” I was like, “Come on, man, can’t let me play Ice Cream Man?” The opening lick of Ice Cream Man that he plays is very hard to play ’cause it’s an incredibly long stretch. And it hurt my fingers to do, and Eddie would turn his guitar up like this to play. And plus, it’s a tricky… It just… It’s a tricky rhythm, and it’s such a big stretch. It’s like, “Man, I can’t… That hurts my hand.”
Lex Fridman
I just love that that’s the Van Halen solo you have. The top 20.
I just love that that’s the Van Halen solo you have. The top 20.
Rick Beato
See, I have to do some- … There’s so many Van Halen. My God, it could be… There… I could pick 25 different Van Halen solos.
See, I have to do some- … There’s so many Van Halen. My God, it could be… There… I could pick 25 different Van Halen solos.
Lex Fridman
But to me, I mean, there really is nobody like Mark Knopfler. I mean, his is unique guitars. There’s something about his tone. Speaking of Gilmour, there’s just the tone, the care, the timing of the notes. His improvisation, like the live performances of Sultans of Swing that’s been actually going somewhat viral recently, his pretty old live performance of Sultans of Swing. For me, Brothers in Arms, these kind of-
But to me, I mean, there really is nobody like Mark Knopfler. I mean, his is unique guitars. There’s something about his tone. Speaking of Gilmour, there’s just the tone, the care, the timing of the notes. His improvisation, like the live performances of Sultans of Swing that’s been actually going somewhat viral recently, his pretty old live performance of Sultans of Swing. For me, Brothers in Arms, these kind of-
Rick Beato
Great.
Great.
Lex Fridman
… soulful, mournful type of solos, he does really, really well. Also, the interesting instrumentation of Romeo and Juliet. Just so, so many… Just… Truly one of the greats.
… soulful, mournful type of solos, he does really, really well. Also, the interesting instrumentation of Romeo and Juliet. Just so, so many… Just… Truly one of the greats.
Rick Beato
Now, obviously the intro to Money for Nothing is one of the greatest. Almost impossible to recreate that because the sound is so unique and his… It’s just improvised. It’s so cool.
Now, obviously the intro to Money for Nothing is one of the greatest. Almost impossible to recreate that because the sound is so unique and his… It’s just improvised. It’s so cool.
Lex Fridman
Yeah. There’s certain songs like Europa by Santana, Santana can have that tone too. That Mark Knopfler makes me real- just how clean it is. I think he beats B.B. King in my book in terms of the cleanness of just pure beauty of a single note. It’s like the power of a single note. I don’t know anybody who beats Mark Knopfler.
Yeah. There’s certain songs like Europa by Santana, Santana can have that tone too. That Mark Knopfler makes me real- just how clean it is. I think he beats B.B. King in my book in terms of the cleanness of just pure beauty of a single note. It’s like the power of a single note. I don’t know anybody who beats Mark Knopfler.
Rick Beato
Well, that thing about being able to recognize somebody from a note. You know?
Well, that thing about being able to recognize somebody from a note. You know?
Lex Fridman
Yeah, that’s-
Yeah, that’s-
Rick Beato
When I hear Brian May, I can immediately recognize it’s Brian May. Incredibly melodic, the tone that he has. Gilmour, Hendrix, everyone that we’re talking about, Van Halen. It’s just, they have that one note. It’s like, “Oh, I know who that is.” And that’s why we’re talking about him.
When I hear Brian May, I can immediately recognize it’s Brian May. Incredibly melodic, the tone that he has. Gilmour, Hendrix, everyone that we’re talking about, Van Halen. It’s just, they have that one note. It’s like, “Oh, I know who that is.” And that’s why we’re talking about him.
Lex Fridman
That’d be funny. That’d be a good video-
That’d be funny. That’d be a good video-
Rick Beato
B.B. King, you hear one note.
B.B. King, you hear one note.
Lex Fridman
… as a test of like how quickly can you recognize just a solo starts playing-
… as a test of like how quickly can you recognize just a solo starts playing-
Rick Beato
That’s a great… I’m gonna make that video-
That’s a great… I’m gonna make that video-
Lex Fridman
… one note
… one note
Rick Beato
… tomorrow. Lex, you’ll-
… tomorrow. Lex, you’ll-
Lex Fridman
I don’t know.
I don’t know.
Rick Beato
The day after tomorrow, you’ll see it.
The day after tomorrow, you’ll see it.
Lex Fridman
I would love to see that.
I would love to see that.
Rick Beato
Can you say, can you recognize these players by one note?
Can you say, can you recognize these players by one note?
Lex Fridman
By one note. I think it’s… I think we’re being a little too aggressive with that. I think you need like two or three or four-
By one note. I think it’s… I think we’re being a little too aggressive with that. I think you need like two or three or four-
Rick Beato
No, no, no, no.
No, no, no, no.
Lex Fridman
… or five notes.
… or five notes.
Rick Beato
I guarantee you. So I was gonna do a video last week where I was gonna play songs in reverse, okay? See if you can recognize these songs in reverse. And I had my two assistants come in. It’s like, “Do you know what song that is?” They’re like, “Oh, that’s Adele.” Like, “What?” Then they’re like, “Oh, that’s, that’s Nirvana.” Instantly, they could recognize. Like, “Well, that’s not worth me.” It’s like, yeah, it’s so obvious. You hear the tone of the voice backwards, forwards, it doesn’t matter. You know who it is.
I guarantee you. So I was gonna do a video last week where I was gonna play songs in reverse, okay? See if you can recognize these songs in reverse. And I had my two assistants come in. It’s like, “Do you know what song that is?” They’re like, “Oh, that’s Adele.” Like, “What?” Then they’re like, “Oh, that’s, that’s Nirvana.” Instantly, they could recognize. Like, “Well, that’s not worth me.” It’s like, yeah, it’s so obvious. You hear the tone of the voice backwards, forwards, it doesn’t matter. You know who it is.
Lex Fridman
Oh, interesting. Okay. So it’s about the tone. How could you possibly know from a single note? I guess Van Halen, you can.
Oh, interesting. Okay. So it’s about the tone. How could you possibly know from a single note? I guess Van Halen, you can.
Rick Beato
One note of B.B. King’s vibrato, you could know. What I’ll do is I would separate the guitars. I can actually separate the tracks, and I’ll just play one note.
One note of B.B. King’s vibrato, you could know. What I’ll do is I would separate the guitars. I can actually separate the tracks, and I’ll just play one note.
Lex Fridman
You think you could, from a single vibrato, you can know it’s B.B. King?
You think you could, from a single vibrato, you can know it’s B.B. King?
Rick Beato
Yes. Well, we’ll see.
Yes. Well, we’ll see.
Lex Fridman
Put it on record, I’m skeptical.
Put it on record, I’m skeptical.
Rick Beato
I’m gonna do twenty of them. Can you recognize these guitarists from a single note?
I’m gonna do twenty of them. Can you recognize these guitarists from a single note?
Lex Fridman
Could you recognize Stevie Ray Vaughan-
Could you recognize Stevie Ray Vaughan-
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
… versus Eric Clapton? All right. You might be right. You might be right. Quick 30-second thank-you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We’ve got UPLIFT Desk, for my favorite office desks, BetterHelp, for mental health, LMNT, for electrolytes, Fin, for customer service AI agents, Shopify, for selling stuff online, and Perplexity, for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Rick Beato. What do you think is the best Eric Clapton song? One of the things we haven’t mentioned so far is the importance of lyrics and maybe meaning of the song- … and what it represents, so in that sense, Tears in Heaven.
… versus Eric Clapton? All right. You might be right. You might be right. Quick 30-second thank-you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We’ve got UPLIFT Desk, for my favorite office desks, BetterHelp, for mental health, LMNT, for electrolytes, Fin, for customer service AI agents, Shopify, for selling stuff online, and Perplexity, for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Rick Beato. What do you think is the best Eric Clapton song? One of the things we haven’t mentioned so far is the importance of lyrics and maybe meaning of the song- … and what it represents, so in that sense, Tears in Heaven.
Rick Beato
Well, the story behind that is heartbreaking.
Well, the story behind that is heartbreaking.
Lex Fridman
And then, I personally really love the sound of Wonderful Tonight.
And then, I personally really love the sound of Wonderful Tonight.
Rick Beato
That’s a great song. That’s one of my favorite Clapton songs.
That’s a great song. That’s one of my favorite Clapton songs.
Lex Fridman
And I, as I was listening to it, just doing a whole personal journey introspection, knowing that I’m gonna talk to Rick Beato, listening to just a bunch of songs, and I learned—it’s embarrassing that I didn’t know the stories behind the music—but I learned that Eric Clapton was married for a decade to the same woman that George Harrison was married to. And that this woman was the muse, the inspiration for like so many of the legendary songs of rock- … including Wonderful Tonight, including Layla- … and including George Harrison’s Something. Legendary song also. The same woman. Is she the greatest muse in rock history?
And I, as I was listening to it, just doing a whole personal journey introspection, knowing that I’m gonna talk to Rick Beato, listening to just a bunch of songs, and I learned—it’s embarrassing that I didn’t know the stories behind the music—but I learned that Eric Clapton was married for a decade to the same woman that George Harrison was married to. And that this woman was the muse, the inspiration for like so many of the legendary songs of rock- … including Wonderful Tonight, including Layla- … and including George Harrison’s Something. Legendary song also. The same woman. Is she the greatest muse in rock history?
Rick Beato
Probably, yes.
Probably, yes.
Lex Fridman
This is great. So in your interviews of musicians and producers, I think the thing you’re ultimately fascinated by is the process, the recording, the production, the songwriting, the different elements of the process. So are there examples of different things that stand out to you from all the interviews you’ve done? And by the way, all the recording and production you’ve done yourself. So on the recording front, on the production front, on the songwriting process front, just things that pop into memory.
This is great. So in your interviews of musicians and producers, I think the thing you’re ultimately fascinated by is the process, the recording, the production, the songwriting, the different elements of the process. So are there examples of different things that stand out to you from all the interviews you’ve done? And by the way, all the recording and production you’ve done yourself. So on the recording front, on the production front, on the songwriting process front, just things that pop into memory.
Rick Beato
When I’ve interviewed the guys that are the producers, like Rick Rubin, Daniel Lanois, Brendan O’Brien, Butch Vig, the thing about producers, as opposed to people that are musicians—if you’re a musician, even if you’re David Gilmour, you do a record, and then you tour, and then you do another record, maybe years go by, but producers are working on multiple records, sometimes at a time. Rick Rubin could be working on multiple records, and the variety of things that they do, you can talk to. I mean, I can talk to Rick about the Chili Peppers. I can talk to him about Johnny Cash. I can talk to him about Tom Petty, and all these records that I love, and there’s just so many interesting stories that …
When I’ve interviewed the guys that are the producers, like Rick Rubin, Daniel Lanois, Brendan O’Brien, Butch Vig, the thing about producers, as opposed to people that are musicians—if you’re a musician, even if you’re David Gilmour, you do a record, and then you tour, and then you do another record, maybe years go by, but producers are working on multiple records, sometimes at a time. Rick Rubin could be working on multiple records, and the variety of things that they do, you can talk to. I mean, I can talk to Rick about the Chili Peppers. I can talk to him about Johnny Cash. I can talk to him about Tom Petty, and all these records that I love, and there’s just so many interesting stories that …
Rick Beato
I mean, these interviews could go on for days with Rick, and the variety of records that he worked on. And there’s so much knowledge to be gained, for me at least, and I think that the craft of production and recording engineering is something that is not well-documented. Especially since there are so few studios nowadays, where there used to be a mentorship thing, where you’d go and you’d work as an assistant engineer.
I mean, these interviews could go on for days with Rick, and the variety of records that he worked on. And there’s so much knowledge to be gained, for me at least, and I think that the craft of production and recording engineering is something that is not well-documented. Especially since there are so few studios nowadays, where there used to be a mentorship thing, where you’d go and you’d work as an assistant engineer.
Rick Beato
And you’d work your way up. I interviewed a guy named Ken Scott that worked with the Beatles. I interviewed him at Abbey Road Studios, it’s just two months ago, and he started as a tape op when he was 16. He started on the A Hard Day’s Night record with the Beatles, and he worked his way up, and he said the first time he ever recorded an orchestra was he recorded I Am the Walrus, the orchestra part.
And you’d work your way up. I interviewed a guy named Ken Scott that worked with the Beatles. I interviewed him at Abbey Road Studios, it’s just two months ago, and he started as a tape op when he was 16. He started on the A Hard Day’s Night record with the Beatles, and he worked his way up, and he said the first time he ever recorded an orchestra was he recorded I Am the Walrus, the orchestra part.
Rick Beato
He set up the mics, and I asked him, I said, “So where was the band?” “Standing right behind me.” The Beatles, right behind him. The guy I’m interviewing at Abbey Road recorded I Am the Walrus there. I mean, he recorded many Beatles songs, and he was 18 years old, and the … I mean, I just can’t, I can’t even fathom that. We … They have a little cafe in the basement of Abbey Road, and I said, “Did the Beatles come in here?” He goes, “Oh, yeah, they come in here and get coffee,” and I remember when they got two microwaves that were like the first microwaves in 1965, and they were amazed by them, and it’s hard to imagine that I’m talking to people that worked on these historic records.
He set up the mics, and I asked him, I said, “So where was the band?” “Standing right behind me.” The Beatles, right behind him. The guy I’m interviewing at Abbey Road recorded I Am the Walrus there. I mean, he recorded many Beatles songs, and he was 18 years old, and the … I mean, I just can’t, I can’t even fathom that. We … They have a little cafe in the basement of Abbey Road, and I said, “Did the Beatles come in here?” He goes, “Oh, yeah, they come in here and get coffee,” and I remember when they got two microwaves that were like the first microwaves in 1965, and they were amazed by them, and it’s hard to imagine that I’m talking to people that worked on these historic records.
Rick Beato
But, you know, they all start with a blank tape or an empty hard drive, and then, you’ve eventually filled them up with this music that you can’t, you can never imagine it not existing, like Stairway to Heaven, or whatever it is.
But, you know, they all start with a blank tape or an empty hard drive, and then, you’ve eventually filled them up with this music that you can’t, you can never imagine it not existing, like Stairway to Heaven, or whatever it is.
Lex Fridman
Yeah. It’s funny, like, looking back, even probably for them, just to realize they’ve created that magic is hard to believe. ‘Cause you’re looking at a blank thing and then magic comes out, and you don’t even understand. You don’t understand, probably a lot of these artists don’t understand where that came from. They’re channeling some deeper thing.
Yeah. It’s funny, like, looking back, even probably for them, just to realize they’ve created that magic is hard to believe. ‘Cause you’re looking at a blank thing and then magic comes out, and you don’t even understand. You don’t understand, probably a lot of these artists don’t understand where that came from. They’re channeling some deeper thing.
Rick Beato
When I interviewed Brian May, he told me, I can’t even remember if this was, if we talked about it on camera or not, but we talked about Bohemian Rhapsody, and at the very end… There was a thing where he was depressing his whammy bar a little bit, and it sounds like the piano is out of tune. I never noticed it before. He mentioned this to me. And he said it always bothered him. And there’s always something about these songs that bothers people. Even these songs that he-
When I interviewed Brian May, he told me, I can’t even remember if this was, if we talked about it on camera or not, but we talked about Bohemian Rhapsody, and at the very end… There was a thing where he was depressing his whammy bar a little bit, and it sounds like the piano is out of tune. I never noticed it before. He mentioned this to me. And he said it always bothered him. And there’s always something about these songs that bothers people. Even these songs that he-
Lex Fridman
These old things, yeah.
These old things, yeah.
Rick Beato
Right. There’s always little things- … and they sit and they hear it, and they’re like, “Oh, man. I wish I’d been up a little higher on that,” or whatever.
Right. There’s always little things- … and they sit and they hear it, and they’re like, “Oh, man. I wish I’d been up a little higher on that,” or whatever.
Lex Fridman
I mean, that, that … there’s certain moments in songs that are just unlike anything else. In Bohemian Rhapsody when Freddie Mercury is, “Sometimes wish I’d never been born at all…” And then the guitar comes in. I mean, there’s just nothing like that. That was … That … I don’t even know. I mean, that, that whole thing, you’ve done videos on it. It’s an incredibly complicated composition. It’s, it’s crazy that a popular rock song could be this operatic, so complicated. The other thing akin to that moment is Phil Collins with In the Air Tonight, the drum bridge. Do do do do do do do do.
I mean, that, that … there’s certain moments in songs that are just unlike anything else. In Bohemian Rhapsody when Freddie Mercury is, “Sometimes wish I’d never been born at all…” And then the guitar comes in. I mean, there’s just nothing like that. That was … That … I don’t even know. I mean, that, that whole thing, you’ve done videos on it. It’s an incredibly complicated composition. It’s, it’s crazy that a popular rock song could be this operatic, so complicated. The other thing akin to that moment is Phil Collins with In the Air Tonight, the drum bridge. Do do do do do do do do.
Rick Beato
Yeah. Yeah.
Yeah. Yeah.
Lex Fridman
Yeah. What is that? I don’t understand how you can create that. What is that? Why is that so magical? Why is that so singular inside a particular song and in rock history period? Like, these moments, I don’t know, musically, I don’t understand how you create them ’cause it might be bigger than musical. It might be cultural, a bunch of different elements, and plus, it’s him filled with … Like, I’ve seen live performances. He has, like, a headset. He does something. He’s like a telemarketer or something. Like, his whole vibe and look to him, he doesn’t look like a rockstar, but he is.
Yeah. What is that? I don’t understand how you can create that. What is that? Why is that so magical? Why is that so singular inside a particular song and in rock history period? Like, these moments, I don’t know, musically, I don’t understand how you create them ’cause it might be bigger than musical. It might be cultural, a bunch of different elements, and plus, it’s him filled with … Like, I’ve seen live performances. He has, like, a headset. He does something. He’s like a telemarketer or something. Like, his whole vibe and look to him, he doesn’t look like a rockstar, but he is.
Rick Beato
Those are hooks when you think about it, right? It’s like, it’s as much of a hook as any, as the chorus of the song or any song. That drum thing is something that people wait for, and they air drum to it. Everybody air drums to it, and it is a hook, and those are hard to create. Those are … Those moments are really hard to create, and usually they’re done by accident.
Those are hooks when you think about it, right? It’s like, it’s as much of a hook as any, as the chorus of the song or any song. That drum thing is something that people wait for, and they air drum to it. Everybody air drums to it, and it is a hook, and those are hard to create. Those are … Those moments are really hard to create, and usually they’re done by accident.
Lex Fridman
Yes, it’s hard. If you chase it, you’re not gonna get it. In your conversation with Sting, he said something about how modern music is simpler more minimalistic, and, ” The bridge is gone,” I think- … he said. And he said he thought that, “The bridge is therapy.” It’s, like, a chance for you to reflect, I guess, on the verse- … before the chorus comes in.
Yes, it’s hard. If you chase it, you’re not gonna get it. In your conversation with Sting, he said something about how modern music is simpler more minimalistic, and, ” The bridge is gone,” I think- … he said. And he said he thought that, “The bridge is therapy.” It’s, like, a chance for you to reflect, I guess, on the verse- … before the chorus comes in.
Rick Beato
That’s right.
That’s right.
Lex Fridman
It changed my view of the bridge, I suppose, is the therapeutic nature of it, at least lyrically. You think he’s onto something? The value of the bridge?
It changed my view of the bridge, I suppose, is the therapeutic nature of it, at least lyrically. You think he’s onto something? The value of the bridge?
Rick Beato
The bridge is a place, I think, where you can kind of change the frame of reference of a song.
The bridge is a place, I think, where you can kind of change the frame of reference of a song.
Lex Fridman
You could probably do anything, I guess.
You could probably do anything, I guess.
Rick Beato
Lennon used to… He would have some kind of biting lyrics, like “We Can Work It Out.” So McCartney writes the, you know, “Try to see it my way. Do I have to keep on going until I can’t go on?” And then, but the bridge is very Lennon. “Life is very short, and there’s no time. For fussing and fighting, my friend. I have always thought that it’s a crime, so I’ll ask you once again.” I mean, it’s very, you know, very Lennon-esque. This is … That was really a … kind of a real collaboration between the two of those.
Lennon used to… He would have some kind of biting lyrics, like “We Can Work It Out.” So McCartney writes the, you know, “Try to see it my way. Do I have to keep on going until I can’t go on?” And then, but the bridge is very Lennon. “Life is very short, and there’s no time. For fussing and fighting, my friend. I have always thought that it’s a crime, so I’ll ask you once again.” I mean, it’s very, you know, very Lennon-esque. This is … That was really a … kind of a real collaboration between the two of those.
Lex Fridman
This is where different parts of the band can clash- … in interesting ways. I mean, the Beatles are the epitome of that. Such … Like, each individual Beatle is a great talent in their own right. How were the Beatles able to create some of the greatest songs of all time all before they turned 30 years old?
This is where different parts of the band can clash- … in interesting ways. I mean, the Beatles are the epitome of that. Such … Like, each individual Beatle is a great talent in their own right. How were the Beatles able to create some of the greatest songs of all time all before they turned 30 years old?
Rick Beato
I have never been able to figure that out, but I have a theory that- … because PA-
I have never been able to figure that out, but I have a theory that- … because PA-
Lex Fridman
I have a theory.
I have a theory.
Rick Beato
Because PA systems were so bad back then- … and the Beatles … People screamed so loudly that the Beatles thought, “Okay. We don’t, we don’t need … We can’t tour anymore ’cause we can’t even hear ourselves, so we’re just gonna be a studio band.” And maybe because of … We have all these great late Beatles records, they’re from 1966 on, just because they had bad PA systems. And they had no monitors. You know, they’re in Shea Stadium.
Because PA systems were so bad back then- … and the Beatles … People screamed so loudly that the Beatles thought, “Okay. We don’t, we don’t need … We can’t tour anymore ’cause we can’t even hear ourselves, so we’re just gonna be a studio band.” And maybe because of … We have all these great late Beatles records, they’re from 1966 on, just because they had bad PA systems. And they had no monitors. You know, they’re in Shea Stadium.
Rick Beato
People are screaming so loudly they can’t hear themselves. They’re like, “Okay, forget this. We can’t tour. We’ll just make studio records,” so that’s what they did, and in that one year, like, from August 6th, 1965, they put out Help!. Then in December 3rd, they put out Rubber Soul of ’65. Then, then August 5th, they put out Revolver. So within 365 days, they put out three 14, I think, 14-song records. So they wrote and recorded three incredibly important records. They were in the studio. It’s like working out.
People are screaming so loudly they can’t hear themselves. They’re like, “Okay, forget this. We can’t tour. We’ll just make studio records,” so that’s what they did, and in that one year, like, from August 6th, 1965, they put out Help!. Then in December 3rd, they put out Rubber Soul of ’65. Then, then August 5th, they put out Revolver. So within 365 days, they put out three 14, I think, 14-song records. So they wrote and recorded three incredibly important records. They were in the studio. It’s like working out.
Rick Beato
They’re practicing their craft every day, writing songs, trying to outdo the other ones, and so you had the, the perfect thing of, of four supremely talented musicians, songwriters, singers, and then the best producer you could possibly have, George Martin, and, and it was just a perfect storm. I think that when I would talk to friends that would just play in local clubs, and they’d play four-hour sets five nights a week, and they never lost their voices because they’re always working those muscles. And same with the Beatles. They were always in the studio singing every single day, doing takes, and I think that that was part of it, at least.
They’re practicing their craft every day, writing songs, trying to outdo the other ones, and so you had the, the perfect thing of, of four supremely talented musicians, songwriters, singers, and then the best producer you could possibly have, George Martin, and, and it was just a perfect storm. I think that when I would talk to friends that would just play in local clubs, and they’d play four-hour sets five nights a week, and they never lost their voices because they’re always working those muscles. And same with the Beatles. They were always in the studio singing every single day, doing takes, and I think that that was part of it, at least.
Lex Fridman
But you also have this theory- that you know, that the greatest productivity that musicians have is before they turn 30. The greatest, sort of, creative genius that can come out of the human mind musically is before the age of 30.
But you also have this theory- that you know, that the greatest productivity that musicians have is before they turn 30. The greatest, sort of, creative genius that can come out of the human mind musically is before the age of 30.
Rick Beato
Well, I think it’s the same in mathematics, as well, the- … you have this fluid intelligence versus crystallized intelligence. Fluid intelligence up until you’re about, you know, in your late 20s-
Well, I think it’s the same in mathematics, as well, the- … you have this fluid intelligence versus crystallized intelligence. Fluid intelligence up until you’re about, you know, in your late 20s-
Lex Fridman
Yeah
Yeah
Rick Beato
… 30 years old, and then crystallized, so you’re using… The crystallized is you’re using your life experience to write things, so you’ll find that composers like Bach, Beethoven, Mozart wrote their most important works at the end of their lives. Beethoven, the late string quartets, the Ninth Symphony, things like that. So, they have a whole lifetime of experience that lead up to this, and there’s not… They’re not improvising, but things for improvising, writing pop songs, and that… I think when your mind is really most active and your brain processing speed is at its pinnacle, that… This is just my theory-
… 30 years old, and then crystallized, so you’re using… The crystallized is you’re using your life experience to write things, so you’ll find that composers like Bach, Beethoven, Mozart wrote their most important works at the end of their lives. Beethoven, the late string quartets, the Ninth Symphony, things like that. So, they have a whole lifetime of experience that lead up to this, and there’s not… They’re not improvising, but things for improvising, writing pop songs, and that… I think when your mind is really most active and your brain processing speed is at its pinnacle, that… This is just my theory-
Rick Beato
… that people can come up with those kind of ideas. Same with improvising. I think that most jazz improvisers, not all, but most, do their best improvising before the age of 30.
… that people can come up with those kind of ideas. Same with improvising. I think that most jazz improvisers, not all, but most, do their best improvising before the age of 30.
Lex Fridman
Creating something new.
Creating something new.
27 Club
Rick Beato
Yes.
Yes.
Lex Fridman
Truly novel, that requires youth. It’s just a theory though, but it seems to apply. What do you think about the 27 Club? A bunch of the music greats died at 27. Hendrix, Brian Jones, Jim Morrison, Janis Joplin, Amy Winehouse.
Truly novel, that requires youth. It’s just a theory though, but it seems to apply. What do you think about the 27 Club? A bunch of the music greats died at 27. Hendrix, Brian Jones, Jim Morrison, Janis Joplin, Amy Winehouse.
Rick Beato
Kurt Cobain.
Kurt Cobain.
Lex Fridman
Kurt Cobain, of course. A big part of music history is linked to drug history. LSD, coke, heroin, weed.
Kurt Cobain, of course. A big part of music history is linked to drug history. LSD, coke, heroin, weed.
Rick Beato
Smoking.
Smoking.
Lex Fridman
Smoking.
Smoking.
Rick Beato
I think about this a lot. If you go back and you watch videos, The Beatles, any of their movies, they’re smoking all the time. The Get Back documentary, they’re smoking constantly. Go watch any of the MTV Unpluggeds, Nirvana, Kurt Cobain is smoking every second that he’s not playing, he’s smoking. Every singer smoked. Every musician smoked. Nowadays, I asked my son, Dylan, “Dylan, does anybody smoke?” at his high school. He’s like- …”Smoke? Nobody smokes.” The- it was an absurd question. And that was part of culture.
I think about this a lot. If you go back and you watch videos, The Beatles, any of their movies, they’re smoking all the time. The Get Back documentary, they’re smoking constantly. Go watch any of the MTV Unpluggeds, Nirvana, Kurt Cobain is smoking every second that he’s not playing, he’s smoking. Every singer smoked. Every musician smoked. Nowadays, I asked my son, Dylan, “Dylan, does anybody smoke?” at his high school. He’s like- …”Smoke? Nobody smokes.” The- it was an absurd question. And that was part of culture.
Lex Fridman
It was for everybody. I mean, that was, that was a big transformation over the past 20 years and just everybody stopped smoking. But I don’t think smoking has the kind of hard negative effect that we’re talking about. I mean, I almost would rather have them smoke than some of the other hard drugs. Maybe smoking distracts them from the hard… I mean, heroin and coke, I mean, those, those things really, and alcohol, unfortunately-
It was for everybody. I mean, that was, that was a big transformation over the past 20 years and just everybody stopped smoking. But I don’t think smoking has the kind of hard negative effect that we’re talking about. I mean, I almost would rather have them smoke than some of the other hard drugs. Maybe smoking distracts them from the hard… I mean, heroin and coke, I mean, those, those things really, and alcohol, unfortunately-
Lex Fridman
… can be easily abused, I think. It seems like it’s a… The life of a musician, this dopamine thing of getting on stage and being adored by tens of thousands, hundreds of thousands of people, the high of that, and then the comedown after is really hard life, for just even neurobiologically, of like, how do you deal with that? You have to be able to control the rollercoaster of your mind, and of course drugs will be a part of that. And you think everything is allowed and everything is possible. And then there’s also culture, depending on who you hang out with, that certain kinds of categories of drugs are good for your creativity.
… can be easily abused, I think. It seems like it’s a… The life of a musician, this dopamine thing of getting on stage and being adored by tens of thousands, hundreds of thousands of people, the high of that, and then the comedown after is really hard life, for just even neurobiologically, of like, how do you deal with that? You have to be able to control the rollercoaster of your mind, and of course drugs will be a part of that. And you think everything is allowed and everything is possible. And then there’s also culture, depending on who you hang out with, that certain kinds of categories of drugs are good for your creativity.
Lex Fridman
And so, naturally, you start to abuse those drugs. I don’t know. I think it’s really interesting the role that drugs have played in the history of music. They have certainly been extremely destructive, but they have also certainly been productive muses, inspirations for some of these folks.
And so, naturally, you start to abuse those drugs. I don’t know. I think it’s really interesting the role that drugs have played in the history of music. They have certainly been extremely destructive, but they have also certainly been productive muses, inspirations for some of these folks.
Rick Beato
Oh, absolutely. Now, would we want to, you know, advocate people doing things like that to boost their creativity?
Oh, absolutely. Now, would we want to, you know, advocate people doing things like that to boost their creativity?
Lex Fridman
No.
No.
Rick Beato
Well, I wouldn’t, but just like smoking, which I think improved people’s voices- I mean really, the raspiness of it- … this is the reason that so many of these, virtually every famous singer- … no matter what genre of music, jazz, soul, rock, they all smoked.
Well, I wouldn’t, but just like smoking, which I think improved people’s voices- I mean really, the raspiness of it- … this is the reason that so many of these, virtually every famous singer- … no matter what genre of music, jazz, soul, rock, they all smoked.
Lex Fridman
Yeah.
Yeah.
Rick Beato
Nat King Cole.
Nat King Cole.
Lex Fridman
Miles Davis too?
Miles Davis too?
Rick Beato
Miles smoked- everybody smoked. Miles did… Well, Miles was a heroin addict too. I mean-
Miles smoked- everybody smoked. Miles did… Well, Miles was a heroin addict too. I mean-
Lex Fridman
Yeah, yeah.
Yeah, yeah.
Rick Beato
… so many jazz musicians.
… so many jazz musicians.
Lex Fridman
Well, Miles had a sound to him. You’re right. I mean, smoking must play a gigantic role in that, adding some complexity to the voice.
Well, Miles had a sound to him. You’re right. I mean, smoking must play a gigantic role in that, adding some complexity to the voice.
Rick Beato
Yes.
Yes.
Lex Fridman
Yeah, some richness to the voice.
Yeah, some richness to the voice.
Rick Beato
Nat King Cole, he smoked, I think, four packs a day. He died of lung cancer. Lotta heavy smokers though, in singers. Frank Sinatra, heavy smoker. McCartney was a heavy smoker. Lennon, all those guys smoked.
Nat King Cole, he smoked, I think, four packs a day. He died of lung cancer. Lotta heavy smokers though, in singers. Frank Sinatra, heavy smoker. McCartney was a heavy smoker. Lennon, all those guys smoked.
Lex Fridman
Yeah, it’s hard to know, chicken or the egg. But I certainly wouldn’t recommend doing drugs as a way to get better at music.
Yeah, it’s hard to know, chicken or the egg. But I certainly wouldn’t recommend doing drugs as a way to get better at music.
Rick Beato
No, no.
No, no.
Lex Fridman
But, you know, it does seem to go hand-in-hand. And some of it has to do with the period, with the time period, with the place, ’cause sometimes it’s part of the culture. The drug is like you’re saying, smoking. If you were smoking now, that’s gonna be a very different experience than smoking 10 years ago, 20 years ago, 50 years ago. There’s a different vibe. So, sometimes the drug is a deep integrated part of the culture versus an actual chemical substance. The ’60s, right? I don’t know. They were on everything in the ’60s.
But, you know, it does seem to go hand-in-hand. And some of it has to do with the period, with the time period, with the place, ’cause sometimes it’s part of the culture. The drug is like you’re saying, smoking. If you were smoking now, that’s gonna be a very different experience than smoking 10 years ago, 20 years ago, 50 years ago. There’s a different vibe. So, sometimes the drug is a deep integrated part of the culture versus an actual chemical substance. The ’60s, right? I don’t know. They were on everything in the ’60s.
Rick Beato
Yeah…. I mean, it has to account for something, Lex, you know?
Yeah…. I mean, it has to account for something, Lex, you know?
Elton John
Lex Fridman
On the songwriting front, you mentioned a story about Elton John recording. So he’s one of the legendary songwriters. But yeah. You’ve met him, and you know something about the process of his, um-
On the songwriting front, you mentioned a story about Elton John recording. So he’s one of the legendary songwriters. But yeah. You’ve met him, and you know something about the process of his, um-
Rick Beato
Yeah, ’cause he was recording in a studio in Atlanta that I was working with a band that I was producing. And he was in—I was in Studio B, he was in Studio A. And this band that I was working with, they were called Jump, Little Children. And so, he had his assistant come in and ask, “Hey, is this… Are you guys Jump, Little Children?” “Yeah, yeah, yeah.” And then all of a sudden, I couldn’t see out into the live room. Elton walked into the thing, and we were getting ready to track, and I’m, I’m pressing the button. “Yo, where are, where are you guys? What’s up? I thought we were gonna start this.” And no one’s responding. I can hear talking, it’s like, “What, what is going on? Where are they?” Then all of a sudden they come back in the studio and they were stunned.
Yeah, ’cause he was recording in a studio in Atlanta that I was working with a band that I was producing. And he was in—I was in Studio B, he was in Studio A. And this band that I was working with, they were called Jump, Little Children. And so, he had his assistant come in and ask, “Hey, is this… Are you guys Jump, Little Children?” “Yeah, yeah, yeah.” And then all of a sudden, I couldn’t see out into the live room. Elton walked into the thing, and we were getting ready to track, and I’m, I’m pressing the button. “Yo, where are, where are you guys? What’s up? I thought we were gonna start this.” And no one’s responding. I can hear talking, it’s like, “What, what is going on? Where are they?” Then all of a sudden they come back in the studio and they were stunned.
Rick Beato
I said, “Where were you guys?” “Elton John just walked into our session. And he said he’s a big fan. He said to come over when we’re done and hang out in Studio A.” So we did, and he was there with Bernie Taupin, and they were working on a song. And we talked there for an hour, and he was talking about recording two records a year, and then they’d go on tour, and they’d write and record the whole record in two weeks. So Bernie would give him lyrics. Elton would go out and spend 15 minutes writing all the melody. He’d look at his lyrics, and he was doing that that day. Bernie was there, and they had a lyric sheet up on the piano. And Elton would go on, and they’d just re-… “Okay, just record this.” And Elton would sit there and play and come up with the song- … in 15 minutes or so.
I said, “Where were you guys?” “Elton John just walked into our session. And he said he’s a big fan. He said to come over when we’re done and hang out in Studio A.” So we did, and he was there with Bernie Taupin, and they were working on a song. And we talked there for an hour, and he was talking about recording two records a year, and then they’d go on tour, and they’d write and record the whole record in two weeks. So Bernie would give him lyrics. Elton would go out and spend 15 minutes writing all the melody. He’d look at his lyrics, and he was doing that that day. Bernie was there, and they had a lyric sheet up on the piano. And Elton would go on, and they’d just re-… “Okay, just record this.” And Elton would sit there and play and come up with the song- … in 15 minutes or so.
Lex Fridman
Yeah, that’s crazy.
Yeah, that’s crazy.
Rick Beato
There’s a great version of, I think, Tiny Dancer, where Elton is coming up with it on, it’s on YouTube. And he’s just coming up with the music right there. And then the band, “Okay, here’s how it goes.” And they record it right then. Then move onto the next song. I see this. I mean, it’s really incredible. That’s it. Yeah. True. There’s one there that I’ve sort of done the other day with Tiny Dancer, which is about Bernie’s girlfriend. So I just sort of ran it through and then put two verses together, then a mid-like, then a chorus, and then back to the sort of verse sort of thing. It’s, it happens very quickly. It sounds long, but it sort of starts off- << Blue jean baby, LA lady, seamstress for the band. Pirates man, pretty eye, you marry >>
There’s a great version of, I think, Tiny Dancer, where Elton is coming up with it on, it’s on YouTube. And he’s just coming up with the music right there. And then the band, “Okay, here’s how it goes.” And they record it right then. Then move onto the next song. I see this. I mean, it’s really incredible. That’s it. Yeah. True. There’s one there that I’ve sort of done the other day with Tiny Dancer, which is about Bernie’s girlfriend. So I just sort of ran it through and then put two verses together, then a mid-like, then a chorus, and then back to the sort of verse sort of thing. It’s, it happens very quickly. It sounds long, but it sort of starts off- << Blue jean baby, LA lady, seamstress for the band. Pirates man, pretty eye, you marry >>
Lex Fridman
Okay.
Okay.
Rick Beato
I mean, it’s really amazing that he just-
I mean, it’s really amazing that he just-
Lex Fridman
Yeah. He’s looking at just the lyrics.
Yeah. He’s looking at just the lyrics.
Rick Beato
Yeah, and it’s one of the, he’s one of the very few people that has the lyrics first and writes the music to it, which to me is far more difficult. 99% Of songwriters write the music first, and then they put the melody and lyrics to the finished backing track.
Yeah, and it’s one of the, he’s one of the very few people that has the lyrics first and writes the music to it, which to me is far more difficult. 99% Of songwriters write the music first, and then they put the melody and lyrics to the finished backing track.
Lex Fridman
And maybe they write, like, lyrics, they write, like nonsense words kind of- … thing. And then they figure out from there. Yeah, that’s… I mean, I don’t know what skill that is exactly, but that’s incredible. I mean, in that process he makes it his own. Okay. You had an amazing interview with Kirk Hammett. I’m a huge Metallica fan.
And maybe they write, like, lyrics, they write, like nonsense words kind of- … thing. And then they figure out from there. Yeah, that’s… I mean, I don’t know what skill that is exactly, but that’s incredible. I mean, in that process he makes it his own. Okay. You had an amazing interview with Kirk Hammett. I’m a huge Metallica fan.
Metallica
Rick Beato
Same here.
Same here.
Lex Fridman
There is a lot of interesting stuff that came out of that, from that conversation. One is the distinction between heavy metal and hard rock. Which is very interesting. Of course, Metallica went through their own evolution. They had many periods. I mean, they’ve been around 40 years.
There is a lot of interesting stuff that came out of that, from that conversation. One is the distinction between heavy metal and hard rock. Which is very interesting. Of course, Metallica went through their own evolution. They had many periods. I mean, they’ve been around 40 years.
Rick Beato
Over 40 years, yeah. Crazy.
Over 40 years, yeah. Crazy.
Lex Fridman
The other thing is the downpicking, which was interesting, which is creating that really distinct sound.
The other thing is the downpicking, which was interesting, which is creating that really distinct sound.
Rick Beato
James and Kirk’s, the downpicking, I used to be able to do that. I just can’t do that anymore. It hurts my thumb- … to do it. I think honestly, I thought a lot about it. It’s like, why does it, why is it so painful? Why is it so hard? It’s from swiping with your thumb on phones. And I think it affects that basal joint there, and—I’m sorry—no, I’m serious.
James and Kirk’s, the downpicking, I used to be able to do that. I just can’t do that anymore. It hurts my thumb- … to do it. I think honestly, I thought a lot about it. It’s like, why does it, why is it so painful? Why is it so hard? It’s from swiping with your thumb on phones. And I think it affects that basal joint there, and—I’m sorry—no, I’m serious.
Lex Fridman
I love your theories.
I love your theories.
Rick Beato
Well, I think that that’s actually right, ’cause I’m thinking like, “Why does that hurt so much to do that? All the downstrokes and stuff.” It’s gotta be something. It’s like, yeah, it’s from swiping with the phone.
Well, I think that that’s actually right, ’cause I’m thinking like, “Why does that hurt so much to do that? All the downstrokes and stuff.” It’s gotta be something. It’s like, yeah, it’s from swiping with the phone.
Lex Fridman
The other thing that came through is that he’s an improviser at heart. And that, I think, clashes with this kind of rigid structure that metal is. So there’s a real soulful, melodic aspect to him. And he gave a lot of props to James Hetfield for just being a great composer, being a great musician and writer of riffs, of rhythm.
The other thing that came through is that he’s an improviser at heart. And that, I think, clashes with this kind of rigid structure that metal is. So there’s a real soulful, melodic aspect to him. And he gave a lot of props to James Hetfield for just being a great composer, being a great musician and writer of riffs, of rhythm.
Rick Beato
The improvisation part of it you don’t think of ’cause they’ve… ’cause you have the finished songs that you listen to. But those songs are born out of improvisations, of jams, of little fragments of ideas. And then they craft them into these masterpieces.
The improvisation part of it you don’t think of ’cause they’ve… ’cause you have the finished songs that you listen to. But those songs are born out of improvisations, of jams, of little fragments of ideas. And then they craft them into these masterpieces.
Lex Fridman
Also, you mentioned that… This is weird that I didn’t know, that Hendrix used different gauges of strings.
Also, you mentioned that… This is weird that I didn’t know, that Hendrix used different gauges of strings.
Rick Beato
Yeah, he was the one that talked about that, wasn’t he?
Yeah, he was the one that talked about that, wasn’t he?
Lex Fridman
Mm-hmm, yeah, mm-hmm.
Mm-hmm, yeah, mm-hmm.
Rick Beato
Yeah, that was really interesting. See, these are the things that I like to learn from these interviews with these people. I was like, “What? Why have I never heard of that?”
Yeah, that was really interesting. See, these are the things that I like to learn from these interviews with these people. I was like, “What? Why have I never heard of that?”
Lex Fridman
It’s like, it’s one of the ways you can find uniqueness of sound, is by trying different things that are not… I mean, I guess Apple was really good at this, right? Like, completely breaking out of what you’re supposed to do, the ways you’re supposed to do them, and doing it completely differently. You often ask musicians what their perfect song is. First of all, that’s an interesting question.
It’s like, it’s one of the ways you can find uniqueness of sound, is by trying different things that are not… I mean, I guess Apple was really good at this, right? Like, completely breaking out of what you’re supposed to do, the ways you’re supposed to do them, and doing it completely differently. You often ask musicians what their perfect song is. First of all, that’s an interesting question.
Rick Beato
What is a perfect song?
What is a perfect song?
Lex Fridman
Like, one surprise is, Hans Zimmer said God Only Knows by the Beach Boys.
Like, one surprise is, Hans Zimmer said God Only Knows by the Beach Boys.
Rick Beato
I was surprised by that too, but I thought it was like, “Yeah, okay, that’s a perfect song for sure.” The first interview I ever did was with Peter Frampton in 2018, and I asked him in that interview, “What’s the perfect song?” And he said, “Whiter Shade of Pale.” And I was like, “Ooh, that’s a great song.” And then I thought, “I’m gonna ask that to people, just to see what they…” Now people are prepared if I ask that.
I was surprised by that too, but I thought it was like, “Yeah, okay, that’s a perfect song for sure.” The first interview I ever did was with Peter Frampton in 2018, and I asked him in that interview, “What’s the perfect song?” And he said, “Whiter Shade of Pale.” And I was like, “Ooh, that’s a great song.” And then I thought, “I’m gonna ask that to people, just to see what they…” Now people are prepared if I ask that.
Lex Fridman
But it’s like, they’re willing to go out on a limb and say it. Like, if you ask me, I don’t even know. I guess you just say it, whatever, right? Like, what would I even say? What’s a perfect song? Yeah, I would go… See, I feel the pressure.
But it’s like, they’re willing to go out on a limb and say it. Like, if you ask me, I don’t even know. I guess you just say it, whatever, right? Like, what would I even say? What’s a perfect song? Yeah, I would go… See, I feel the pressure.
Rick Beato
Right?
Right?
Lex Fridman
Because the problem is, the reality is, it changes day by day, like minute by minute. I… Yeah, I would probably, I’m sorry, but I would have to go Mark Knopfler. And I would probably go… Is it really cheesy to say the obvious thing? I would go Sultans of Swing. Even though like I’m tempted to say Europa, but then like…
Because the problem is, the reality is, it changes day by day, like minute by minute. I… Yeah, I would probably, I’m sorry, but I would have to go Mark Knopfler. And I would probably go… Is it really cheesy to say the obvious thing? I would go Sultans of Swing. Even though like I’m tempted to say Europa, but then like…
Rick Beato
Sultans of Swing hits on so many levels- … ’cause it’s got a great melody, great lyrics, and then multiple great guitar solos. And has such a unique sound to it. The other thing is that it sounds very different from other Dire Straits songs. I mean, this is like early- … Dire Straits Strat tone. And then you think of like Money for Nothing is a Les Paul, and it’s a totally different kind of vibe than him playing it on Sultans of Swing. But that song’s amazing.
Sultans of Swing hits on so many levels- … ’cause it’s got a great melody, great lyrics, and then multiple great guitar solos. And has such a unique sound to it. The other thing is that it sounds very different from other Dire Straits songs. I mean, this is like early- … Dire Straits Strat tone. And then you think of like Money for Nothing is a Les Paul, and it’s a totally different kind of vibe than him playing it on Sultans of Swing. But that song’s amazing.
Lex Fridman
Plus it’s about music.
Plus it’s about music.
Rick Beato
Yes.
Yes.
Tom Waits
Lex Fridman
So it’s like there’s a meta aspect to it. But then there’s also like, we’re talking about this guitar stuff, but Leonard Cohen, Hallelujah. I mean, Leonard Cohen in general. Like these songwriters, they go super simple on guitar. And there, it’s just what’s that called? Singer-songwriter type. I told you off my one of my, maybe the music guest that’s a dream guest is Tom Waits. I’ve wanted to talk to Tom Waits for a very long time, and I’ve gone through different periods of… You’ve met me at a point in my life where I’ve given up on it a little bit. And I was trying-
So it’s like there’s a meta aspect to it. But then there’s also like, we’re talking about this guitar stuff, but Leonard Cohen, Hallelujah. I mean, Leonard Cohen in general. Like these songwriters, they go super simple on guitar. And there, it’s just what’s that called? Singer-songwriter type. I told you off my one of my, maybe the music guest that’s a dream guest is Tom Waits. I’ve wanted to talk to Tom Waits for a very long time, and I’ve gone through different periods of… You’ve met me at a point in my life where I’ve given up on it a little bit. And I was trying-
Rick Beato
That’s when it’s gonna happen. That’s-
That’s when it’s gonna happen. That’s-
Lex Fridman
Okay.
Okay.
Rick Beato
Once you give up on it, it’s gonna happen.
Once you give up on it, it’s gonna happen.
Lex Fridman
Yeah. Yeah.
Yeah. Yeah.
Rick Beato
Why Tom Waits won’t be on your podcast.
Why Tom Waits won’t be on your podcast.
Lex Fridman
Exactly. Exactly, dude. This is, this is my, this is my moment.
Exactly. Exactly, dude. This is, this is my, this is my moment.
Rick Beato
Tom, come, come here. Let’s do it. I wanna see it.
Tom, come, come here. Let’s do it. I wanna see it.
Lex Fridman
I’m such a fan of, like the Zappa-like artistry on the musical front, which Tom Waits has, but I’m a sucker for great lyrics. Lyrics to me is such a big part of great songs. And he’s another example. He has a song called Martha. It’s about a love story that didn’t work out, and it’s an older man calling the woman that he was in love with, and basically reminiscing about like, you know, thinking about like, “What would’ve happened if it worked out?” That kinda thing. And then, you know, I loved that song for a long time, and you know at some point I found out that he wrote that when he was in his early 20s. And you realize, it’s similar with the Beatles, like- … These guys somehow were able to capture the human condition so masterfully, and they’re kids.
I’m such a fan of, like the Zappa-like artistry on the musical front, which Tom Waits has, but I’m a sucker for great lyrics. Lyrics to me is such a big part of great songs. And he’s another example. He has a song called Martha. It’s about a love story that didn’t work out, and it’s an older man calling the woman that he was in love with, and basically reminiscing about like, you know, thinking about like, “What would’ve happened if it worked out?” That kinda thing. And then, you know, I loved that song for a long time, and you know at some point I found out that he wrote that when he was in his early 20s. And you realize, it’s similar with the Beatles, like- … These guys somehow were able to capture the human condition so masterfully, and they’re kids.
Lex Fridman
This, I don’t get it. I don’t understand it.
This, I don’t get it. I don’t understand it.
Rick Beato
I can’t speak for Tom Waits, but in the Beatles case, they went to Hamburg, they spent time on their own, they played cover gigs that were eight hours long, and they lived-
I can’t speak for Tom Waits, but in the Beatles case, they went to Hamburg, they spent time on their own, they played cover gigs that were eight hours long, and they lived-
Lex Fridman
Yeah, they’ve lived-
Yeah, they’ve lived-
Rick Beato
… they lived life. It’s not like, not like kids today.
… they lived life. It’s not like, not like kids today.
Lex Fridman
Now you’re on a porch. You also had an amazing interview with Billy Corgan, of Smashing Pumpkins. He is definitively one of my favorite musicians.
Now you’re on a porch. You also had an amazing interview with Billy Corgan, of Smashing Pumpkins. He is definitively one of my favorite musicians.
Rick Beato
I love Billy.
I love Billy.
Lex Fridman
You asked him an interesting question about how he creates this melancholy feeling that permeates a lot of his songs, and he jokingly said that the secret is all about the seventh and the ninth. So like, musically, chord-wise, what do you think about that? You think he’s onto something?
You asked him an interesting question about how he creates this melancholy feeling that permeates a lot of his songs, and he jokingly said that the secret is all about the seventh and the ninth. So like, musically, chord-wise, what do you think about that? You think he’s onto something?
Rick Beato
He’s talking a little music theory there. Seventh and ninth over the chord that he’s playing. So if you’re playing a C chord, he’s singing a B, would be the seventh, D would be the ninth. And he does use a lot of those notes. But almost all these people that we’re talking… No, all these people that we’re talking about use these notes, and this is why their songs… And when I interviewed Sting, I called them surprise tones, and Sting’s like, “I like the way you use the word surprise.” Notes that are outside the chord that are dissonant with the chords that they’re playing, but then that creates emotion. Dissonance equals emotion. And that’s what I like. I want music to be… to depress me.
He’s talking a little music theory there. Seventh and ninth over the chord that he’s playing. So if you’re playing a C chord, he’s singing a B, would be the seventh, D would be the ninth. And he does use a lot of those notes. But almost all these people that we’re talking… No, all these people that we’re talking about use these notes, and this is why their songs… And when I interviewed Sting, I called them surprise tones, and Sting’s like, “I like the way you use the word surprise.” Notes that are outside the chord that are dissonant with the chords that they’re playing, but then that creates emotion. Dissonance equals emotion. And that’s what I like. I want music to be… to depress me.
Lex Fridman
Yeah. What is that? I don’t know. But melancholy, and I think you articulate it in Aries, it’s not actually that depressing. There’s something about that melancholy feeling that is somehow the other side of the coin of happiness. It’s a kind of longing.
Yeah. What is that? I don’t know. But melancholy, and I think you articulate it in Aries, it’s not actually that depressing. There’s something about that melancholy feeling that is somehow the other side of the coin of happiness. It’s a kind of longing.
Lex Fridman
Or there’s a hopefulness to it. That aloneness that you feel. I mean, that’s actually like one of the intimate connections you have with music, is when you’re alone. There’s nothing like you’re alone in a car driving, listening to, like, whatever it is, Bruce Springsteen. Well, I think Louis CK has a bit about that. And was it Bruce Springsteen? But sometimes he has to pull over to the side of the road and just weep, or something like this. It’s just there’s something about that. Sometimes a song just connects with you. And I don’t know, nothing like a melancholy song could do that. It…
Or there’s a hopefulness to it. That aloneness that you feel. I mean, that’s actually like one of the intimate connections you have with music, is when you’re alone. There’s nothing like you’re alone in a car driving, listening to, like, whatever it is, Bruce Springsteen. Well, I think Louis CK has a bit about that. And was it Bruce Springsteen? But sometimes he has to pull over to the side of the road and just weep, or something like this. It’s just there’s something about that. Sometimes a song just connects with you. And I don’t know, nothing like a melancholy song could do that. It…
Lex Fridman
You think about, like, maybe things you regret or how life could’ve worked out. And sometimes it’s not even about, like… It’s not even real. It just connects something in the soul. The uneasiness that we all feel. Maybe the loneliness we all feel that underpins so much of the human condition, and it just connects with that. I don’t know what that is.
You think about, like, maybe things you regret or how life could’ve worked out. And sometimes it’s not even about, like… It’s not even real. It just connects something in the soul. The uneasiness that we all feel. Maybe the loneliness we all feel that underpins so much of the human condition, and it just connects with that. I don’t know what that is.
Rick Beato
There’s a Kurt Cobain lyric. It was on the In Utero record, from the song Frances Farmer. The chorus part is, “I miss the comfort of being sad.”… and I was like, “Yes.” <> I was like, “Yeah, that’s it right there.”
There’s a Kurt Cobain lyric. It was on the In Utero record, from the song Frances Farmer. The chorus part is, “I miss the comfort of being sad.”… and I was like, “Yes.” <> I was like, “Yeah, that’s it right there.”
Lex Fridman
In terms of love songs, I somehow find powerful that kind of desperation. So like I’ve always connected with Pearl Jam’s Black.
In terms of love songs, I somehow find powerful that kind of desperation. So like I’ve always connected with Pearl Jam’s Black.
Rick Beato
Oh, amazing.
Oh, amazing.
Lex Fridman
Like that line is… A friend of mine was going through a breakup, so I was listening and he, he’s the one that introduced me to Pearl Jam during that, that whole period when Pearl Jam was huge with Ten. Is, is that line is “Someday-“
Like that line is… A friend of mine was going through a breakup, so I was listening and he, he’s the one that introduced me to Pearl Jam during that, that whole period when Pearl Jam was huge with Ten. Is, is that line is “Someday-“
Rick Beato
“Someday you’ll have a beautiful life. You know, someday you’ll be a star in somebody else’s sky. Why, why, why can’t it be, can’t it be mine?” Oh my God, that- … blows me away. That’s an amazing line.
“Someday you’ll have a beautiful life. You know, someday you’ll be a star in somebody else’s sky. Why, why, why can’t it be, can’t it be mine?” Oh my God, that- … blows me away. That’s an amazing line.
Lex Fridman
Well, yeah, I mean-
Well, yeah, I mean-
Rick Beato
The delivery is incredible on it too.
The delivery is incredible on it too.
Lex Fridman
Yeah. Eddie Vedder, one of the great frontmen of all time. And that whole period, that whole moment in history of Kurt Cobain and Eddie Vedder that captured… That was the ’90s. That was one side of the ’90s that just… This singular moment in history. Who, who do you think are the great frontmen in the history of music?
Yeah. Eddie Vedder, one of the great frontmen of all time. And that whole period, that whole moment in history of Kurt Cobain and Eddie Vedder that captured… That was the ’90s. That was one side of the ’90s that just… This singular moment in history. Who, who do you think are the great frontmen in the history of music?
Greatest rock stars
Rick Beato
Freddie Mercury, Robert Plant.
Freddie Mercury, Robert Plant.
Lex Fridman
Freddie Mercury number one, probably.
Freddie Mercury number one, probably.
Rick Beato
Steven Tyler.
Steven Tyler.
Lex Fridman
Jim Morrison.
Jim Morrison.
Rick Beato
Jim Morrison? Yeah. Roger Daltrey.
Jim Morrison? Yeah. Roger Daltrey.
Lex Fridman
Well, we have to say, I have to say, James Hetfield.
Well, we have to say, I have to say, James Hetfield.
Rick Beato
James Hetfield?
James Hetfield?
Lex Fridman
I mean, there’s nothing… I mean, I have to talk to you about this. I mean, it’s just the greatest, I think the greatest concert of all time. This is their historic performance in Moscow in September of ’91. This is shortly before the Soviet Union collapsed. Plus, we should mention AC/DC and Pantera-
I mean, there’s nothing… I mean, I have to talk to you about this. I mean, it’s just the greatest, I think the greatest concert of all time. This is their historic performance in Moscow in September of ’91. This is shortly before the Soviet Union collapsed. Plus, we should mention AC/DC and Pantera-
Lex Fridman
… were there too. And about 1.6 million people were there. Now, by the way, there’s like some kind of reporting that there was a half a million people, 500,000 people. There’s somewhere I’ve seen statements like that. That’s a ridiculously inaccurate statement. So it’s a free concert, so any official counts don’t count. It’s definitely over a million. It’s very likely to be 1.5, 1.6 million people. And this moment in history that I think they channeled, it’s like whenever great music… Metallica was firing on all cylinders at the very top of their game, and they meet this moment in history and this place in history.
… were there too. And about 1.6 million people were there. Now, by the way, there’s like some kind of reporting that there was a half a million people, 500,000 people. There’s somewhere I’ve seen statements like that. That’s a ridiculously inaccurate statement. So it’s a free concert, so any official counts don’t count. It’s definitely over a million. It’s very likely to be 1.5, 1.6 million people. And this moment in history that I think they channeled, it’s like whenever great music… Metallica was firing on all cylinders at the very top of their game, and they meet this moment in history and this place in history.
Lex Fridman
There was a, a defining part of the 20th century collapsing, and you have these people who are, for a moment, through music, are able to escape the fear, the anger they feel, the… all of it. There was also a political, social, cultural moment meeting the musical moment, and the set list, I was just… I listened to it several times over the past few days, just taking myself back into that moment in time. Listen to this set list: Enter Sandman, Creeping Death, Harvester of Sorrow, Fade to Black, Sad but True, Master of Puppets, Seek and Destroy, For Whom the Bell Tolls, One, and Whiplash. Look at that. How is that-
There was a, a defining part of the 20th century collapsing, and you have these people who are, for a moment, through music, are able to escape the fear, the anger they feel, the… all of it. There was also a political, social, cultural moment meeting the musical moment, and the set list, I was just… I listened to it several times over the past few days, just taking myself back into that moment in time. Listen to this set list: Enter Sandman, Creeping Death, Harvester of Sorrow, Fade to Black, Sad but True, Master of Puppets, Seek and Destroy, For Whom the Bell Tolls, One, and Whiplash. Look at that. How is that-
Rick Beato
That’s-
That’s-
Lex Fridman
That just-
That just-
Rick Beato
That’s my kind of set
That’s my kind of set
Lex Fridman
… get the fuck out of here.
… get the fuck out of here.
Rick Beato
That’s-
That’s-
Lex Fridman
This is amazing. This is-
This is amazing. This is-
Rick Beato
That’s my kind of set right there.
That’s my kind of set right there.
Lex Fridman
I don’t know if you can think of anything that could beat that.
I don’t know if you can think of anything that could beat that.
Rick Beato
I think that the guys in the band would say that, too. That was… I mean, they were really at their peak. The Black Album had just come out then, and that must have been so, so exciting.
I think that the guys in the band would say that, too. That was… I mean, they were really at their peak. The Black Album had just come out then, and that must have been so, so exciting.
Lex Fridman
I mean, Woodstock was big. There’s, there’s certain moments in time that really, really meet the moment. Are you a fan of live, live like big?
I mean, Woodstock was big. There’s, there’s certain moments in time that really, really meet the moment. Are you a fan of live, live like big?
Rick Beato
I used to be, but at this point- … I can’t, you know… I’d much rather see people play in small clubs- … and, or go to the… I’d like to listen in the studio. Go to the studio, even.
I used to be, but at this point- … I can’t, you know… I’d much rather see people play in small clubs- … and, or go to the… I’d like to listen in the studio. Go to the studio, even.
Lex Fridman
I generally almost entirely agree with you. I just think that there’s these historic moments, but you don’t know- … which are gonna be which, but you’re making the concert free, it’s just all of it, you get plus Pantera and AC/DC. The other, which actually is a legitimate thing you mentioned, is one of the greatest concerts of all time: Beethoven’s world premiere of the Ninth Symphony. You know, I didn’t really know the personal side of Beethoven until I saw this movie called Immortal Beloved. It’s an excellent movie with-
I generally almost entirely agree with you. I just think that there’s these historic moments, but you don’t know- … which are gonna be which, but you’re making the concert free, it’s just all of it, you get plus Pantera and AC/DC. The other, which actually is a legitimate thing you mentioned, is one of the greatest concerts of all time: Beethoven’s world premiere of the Ninth Symphony. You know, I didn’t really know the personal side of Beethoven until I saw this movie called Immortal Beloved. It’s an excellent movie with-
Beethoven
Rick Beato
Gary Oldman
Gary Oldman
Lex Fridman
… Gary Oldman. Just a really… it’s a masterful celebration of Beethoven in an interesting kind of way through the perspective of a love letter that he’s written. But then I realized like… and this is early, this is many, many… this is a couple decades ago now, that, you know, he went deaf before he even started writing the Ninth Symphony, which is why they consider it to be one of the greatest compositions of all time, the greatest symphonies of all time. He went deaf, couldn’t hear anything before he even started writing it. And so there’s that famous story of him in that world premiere of having to be turned around because he can’t hear people applauding, so he has to be turned around to see that people are actually clapping. I mean, there’s just this whole tragic element.
… Gary Oldman. Just a really… it’s a masterful celebration of Beethoven in an interesting kind of way through the perspective of a love letter that he’s written. But then I realized like… and this is early, this is many, many… this is a couple decades ago now, that, you know, he went deaf before he even started writing the Ninth Symphony, which is why they consider it to be one of the greatest compositions of all time, the greatest symphonies of all time. He went deaf, couldn’t hear anything before he even started writing it. And so there’s that famous story of him in that world premiere of having to be turned around because he can’t hear people applauding, so he has to be turned around to see that people are actually clapping. I mean, there’s just this whole tragic element.
Lex Fridman
Plus, the meaning of the symphony that ends in this beautiful Ode to Joy, the symphony itself is a kind of… It starts with the chaos and conflict and ends with this celebration of peace and brotherly unity and a— I guess a call for that, a reaching for that, for that peace. And it’s a… and there’s a tragic element to it, again, connected to history, which is it was post Napoleonic Wars-
Plus, the meaning of the symphony that ends in this beautiful Ode to Joy, the symphony itself is a kind of… It starts with the chaos and conflict and ends with this celebration of peace and brotherly unity and a— I guess a call for that, a reaching for that, for that peace. And it’s a… and there’s a tragic element to it, again, connected to history, which is it was post Napoleonic Wars-
Lex Fridman
… and before the American Civil War. So like, you’re in this, in this middle… this respite from war, calling for peace, not knowing that truly horrific wars are coming. So you have the American Civil War, and you have the, of course, the two World Wars coming. So this, all of it together, and the fact that he’s conducting deaf, and he wrote this whole thing deaf. I was reading a lot about his process, and he just edits and edits and edits and edits. So the fact that he had to edit in his head is just insane.
… and before the American Civil War. So like, you’re in this, in this middle… this respite from war, calling for peace, not knowing that truly horrific wars are coming. So you have the American Civil War, and you have the, of course, the two World Wars coming. So this, all of it together, and the fact that he’s conducting deaf, and he wrote this whole thing deaf. I was reading a lot about his process, and he just edits and edits and edits and edits. So the fact that he had to edit in his head is just insane.
Rick Beato
I mean, it… Beethoven was sick all the time too. I mean, a lot of people were sick all the time. It was very common. What would motivate you to write music, this beautiful music that you can never actually hear except for in your head?
I mean, it… Beethoven was sick all the time too. I mean, a lot of people were sick all the time. It was very common. What would motivate you to write music, this beautiful music that you can never actually hear except for in your head?
Rick Beato
Right? Like, why… The amount of time it takes to write a 35-minute, 40-minute piece, all the parts, you got to hear all the orchestration in your head. You’re editing, you’re doing all these things. Where do you get the motivation when you can’t hear the actual finished work? One, and people would say, “Well, he hears in his head.” But what kind of enjoyment is it? You wanna hear the orchestra… I mean, it’s really profound that he was inspired to do this. There’s a thing called the Heiligenstadt Testament that he wrote. It was a letter to his brothers from 1802. I think they found it in his desk after Beethoven died, and he felt a sense of shame and humiliation because of his hearing loss.
Right? Like, why… The amount of time it takes to write a 35-minute, 40-minute piece, all the parts, you got to hear all the orchestration in your head. You’re editing, you’re doing all these things. Where do you get the motivation when you can’t hear the actual finished work? One, and people would say, “Well, he hears in his head.” But what kind of enjoyment is it? You wanna hear the orchestra… I mean, it’s really profound that he was inspired to do this. There’s a thing called the Heiligenstadt Testament that he wrote. It was a letter to his brothers from 1802. I think they found it in his desk after Beethoven died, and he felt a sense of shame and humiliation because of his hearing loss.
Rick Beato
And he said that he was afflicted with this thing where him of all people, that someone standing next to him could hear a flute that he could not hear, or a shepherd singing in the field that… And he could not hear this. And of all the people, why him? Where hearing played such an important part. Another person that would have had to have had perfect pitch, ’cause you could never do this- … if you didn’t have perfect pitch, which I think all of these great composers, for the most part. Brahms didn’t, from what I know, but all the rest of them, for sure, had perfect pitch. So they could hear these things in their head, and that’s how they composed.
And he said that he was afflicted with this thing where him of all people, that someone standing next to him could hear a flute that he could not hear, or a shepherd singing in the field that… And he could not hear this. And of all the people, why him? Where hearing played such an important part. Another person that would have had to have had perfect pitch, ’cause you could never do this- … if you didn’t have perfect pitch, which I think all of these great composers, for the most part. Brahms didn’t, from what I know, but all the rest of them, for sure, had perfect pitch. So they could hear these things in their head, and that’s how they composed.
Lex Fridman
I mean, you love sound and music. What do you think it was like gradually losing your hearing for Beethoven?
I mean, you love sound and music. What do you think it was like gradually losing your hearing for Beethoven?
Rick Beato
It must have been terrible. I mean, I just… Terrible. I mean, I’ve heard things where he would have a stick in his mouth and put it on the soundboard of the piano, and you could feel the vibrations in his skull, and things like that.
It must have been terrible. I mean, I just… Terrible. I mean, I’ve heard things where he would have a stick in his mouth and put it on the soundboard of the piano, and you could feel the vibrations in his skull, and things like that.
Lex Fridman
Yeah, desperately trying to-
Yeah, desperately trying to-
Rick Beato
Yeah. I just-
Yeah. I just-
Lex Fridman
But also, there’s, what is, what is that, that he’s able to write like one of the greatest symphonies ever, while deaf? So there’s something about that. We mentioned darkness, but torment that he’s going through. And ultimately, Ode to Joy. Like, not a cynical thing- … but a call for the positive.
But also, there’s, what is, what is that, that he’s able to write like one of the greatest symphonies ever, while deaf? So there’s something about that. We mentioned darkness, but torment that he’s going through. And ultimately, Ode to Joy. Like, not a cynical thing- … but a call for the positive.
Rick Beato
Yeah. Yeah. That’s, that’s… I’ve devoted many, many hours thinking about that.
Yeah. Yeah. That’s, that’s… I’ve devoted many, many hours thinking about that.
Lex Fridman
And plus, Napoleon broke his heart, because he was a supporter of Napoleon- … because Napoleon was supposed to represent the French Revolution, this, this hopeful future of no more kings, no more monarchs, no more authoritarian regimes. And Napoleon ended up becoming, essentially, king. Becoming an authoritarian. And Beethoven sort of famously was critical of that. Nevertheless, I think maintained a fascination with Napoleon throughout his life. But sort of a kind of more sophisticated, complex view of human nature and human civilization. So becoming more cynical. Like, seeing more clearly that the world disappoints you, that dreams get shattered. And through that, is able to still do this call for the hopeful future. All right, so okay. So Beethoven, one of the greats, for sure.
And plus, Napoleon broke his heart, because he was a supporter of Napoleon- … because Napoleon was supposed to represent the French Revolution, this, this hopeful future of no more kings, no more monarchs, no more authoritarian regimes. And Napoleon ended up becoming, essentially, king. Becoming an authoritarian. And Beethoven sort of famously was critical of that. Nevertheless, I think maintained a fascination with Napoleon throughout his life. But sort of a kind of more sophisticated, complex view of human nature and human civilization. So becoming more cynical. Like, seeing more clearly that the world disappoints you, that dreams get shattered. And through that, is able to still do this call for the hopeful future. All right, so okay. So Beethoven, one of the greats, for sure.
Lex Fridman
Like basically everybody I know how to play the first movement of Moonlight Sonata, but I always avoided the third movement ’cause I was like, “I’ll never be good enough.” Never, never, but I need to-
Like basically everybody I know how to play the first movement of Moonlight Sonata, but I always avoided the third movement ’cause I was like, “I’ll never be good enough.” Never, never, but I need to-
Rick Beato
Never say never, Lex.
Never say never, Lex.
Lex Fridman
One of these days, maybe. You know what would be great? If Tom Waits writes me an email that says, “I only talk to people that can play-” “… the third movement.”
One of these days, maybe. You know what would be great? If Tom Waits writes me an email that says, “I only talk to people that can play-” “… the third movement.”
Rick Beato
Play the third movement.
Play the third movement.
Lex Fridman
That’d be a dream come true. I’d be like, “For this-“
That’d be a dream come true. I’d be like, “For this-“
Rick Beato
That’s motivation.
That’s motivation.
Lex Fridman
“That’s my dragon,” or whatever you do. You have to have a prince and rescue the princess. My dragon is the third movement of Moonlight Sonata. Okay. You often highlight the importance of Bach. In fact, so many of your guests…
“That’s my dragon,” or whatever you do. You have to have a prince and rescue the princess. My dragon is the third movement of Moonlight Sonata. Okay. You often highlight the importance of Bach. In fact, so many of your guests…
Bach
Rick Beato
Every famous songwriter is influenced by Bach. They are. The greatest composer of all time, the greatest musician of all time.
Every famous songwriter is influenced by Bach. They are. The greatest composer of all time, the greatest musician of all time.
Lex Fridman
Even Sting and Dominic Miller said they go to Bach even for, like, practice.
Even Sting and Dominic Miller said they go to Bach even for, like, practice.
Rick Beato
Every day. People talk about how Bach was not known other than in the places he lived. Eisenach, he was born in. Leipzig, he spent many years. But Bach was known to great musicians. It was difficult to find manuscripts, but there was a premiere of the Saint Matthew Passion that Mendelssohn had done in 1829. It was on March 11th, I believe. He had a manuscript because his father and mother collected manuscripts.
Every day. People talk about how Bach was not known other than in the places he lived. Eisenach, he was born in. Leipzig, he spent many years. But Bach was known to great musicians. It was difficult to find manuscripts, but there was a premiere of the Saint Matthew Passion that Mendelssohn had done in 1829. It was on March 11th, I believe. He had a manuscript because his father and mother collected manuscripts.
Rick Beato
And he got a manuscript of this piece, and he, I think he was 20 years old, and they had a performance of it in Berlin…. and Beethoven, Mozart. They studied the Well-Tempered Clavier, the two books of the Well-Tempered Clavier. But Bach wrote profoundly beautiful music, and some of the most complex contrapuntal music that I don’t think anyone has ever done like that. Extremely bright guy. Had 20 kids, only 10 survived till adulthood. Lost both his parents when he was nine, within nine months of each other. Went to live with an older brother.
And he got a manuscript of this piece, and he, I think he was 20 years old, and they had a performance of it in Berlin…. and Beethoven, Mozart. They studied the Well-Tempered Clavier, the two books of the Well-Tempered Clavier. But Bach wrote profoundly beautiful music, and some of the most complex contrapuntal music that I don’t think anyone has ever done like that. Extremely bright guy. Had 20 kids, only 10 survived till adulthood. Lost both his parents when he was nine, within nine months of each other. Went to live with an older brother.
Lex Fridman
And extremely productive. Also. I think from all the music teachers I’ve ever had, I understood the importance of studying Bach.
And extremely productive. Also. I think from all the music teachers I’ve ever had, I understood the importance of studying Bach.
Rick Beato
He didn’t write Master of Puppets, but he wrote some great powerful-
He didn’t write Master of Puppets, but he wrote some great powerful-
Lex Fridman
Well put.
Well put.
Rick Beato
… music.
… music.
Lex Fridman
Well put. I tried to educate the aforementioned music teachers of the brilliance of Master of Puppets. Sometimes a good riff is greater than any musical composition. So-
Well put. I tried to educate the aforementioned music teachers of the brilliance of Master of Puppets. Sometimes a good riff is greater than any musical composition. So-
Rick Beato
I agree. I go back and I play Master of Puppets every time I’m trying out a new amplifier. That’s my go-to.
I agree. I go back and I play Master of Puppets every time I’m trying out a new amplifier. That’s my go-to.
Lex Fridman
That’s your go-to? So, like, the stereotypical guitar store when you come in, you’re playing Master of Puppets?
That’s your go-to? So, like, the stereotypical guitar store when you come in, you’re playing Master of Puppets?
Rick Beato
I’ll play Master of Puppets. I will play, I have to play some heavy riff- … and so usually it will default to some Metallica or something like that. Or I’ll play Alice in Chains, or I do usually, like, a lot of times I’ll go and I’ll do Drop D something or play Tool. I usually would do some drop tuning thing. And it’s always gotta be some type of metal that I’ll test to see if the bottom end’s tight on the amp and stuff. So, yes.
I’ll play Master of Puppets. I will play, I have to play some heavy riff- … and so usually it will default to some Metallica or something like that. Or I’ll play Alice in Chains, or I do usually, like, a lot of times I’ll go and I’ll do Drop D something or play Tool. I usually would do some drop tuning thing. And it’s always gotta be some type of metal that I’ll test to see if the bottom end’s tight on the amp and stuff. So, yes.
AI in music
Lex Fridman
All right. We have to talk about this a little bit. You made a bunch of videos about it. There was a, there was a moment in time, it still goes on, but there was a moment where really people were freaking out about the use of AI in music. So there’s these, I would say, incredible apps like Suno, Udio. ElevenLabs Music is also great. They can generate basically text to song, full song from a text prompt. And a lot of people started freaking out just based on how good it is.
All right. We have to talk about this a little bit. You made a bunch of videos about it. There was a, there was a moment in time, it still goes on, but there was a moment where really people were freaking out about the use of AI in music. So there’s these, I would say, incredible apps like Suno, Udio. ElevenLabs Music is also great. They can generate basically text to song, full song from a text prompt. And a lot of people started freaking out just based on how good it is.
Lex Fridman
And so you start to immediately imagine how this is going to transform music, and you’re going to replace musicians and all that kind of stuff. It is legitimately nerve-wracking because these are early versions, so you don’t know where it goes. But in your intuition now, you’ve been thinking about this, you made a bunch of videos. Now, like, being able to reflect, “Okay, everybody chill. Calm down.”
And so you start to immediately imagine how this is going to transform music, and you’re going to replace musicians and all that kind of stuff. It is legitimately nerve-wracking because these are early versions, so you don’t know where it goes. But in your intuition now, you’ve been thinking about this, you made a bunch of videos. Now, like, being able to reflect, “Okay, everybody chill. Calm down.”
Rick Beato
So if you write a prompt in Suno and it spits out a song, which I’ve done, made a bunch of videos on this. I made up a fake artist, Eli Mercer, in this video. Then I did a thing for CBS News; I made up this fake artist, Sadie Winters, and came up with this song, “Walking Away.” Well, the computer, the program came up with it.
So if you write a prompt in Suno and it spits out a song, which I’ve done, made a bunch of videos on this. I made up a fake artist, Eli Mercer, in this video. Then I did a thing for CBS News; I made up this fake artist, Sadie Winters, and came up with this song, “Walking Away.” Well, the computer, the program came up with it.
Lex Fridman
There is some creativity in a process. So in this particular thing, the process is you generate an image.
There is some creativity in a process. So in this particular thing, the process is you generate an image.
Rick Beato
I did it in ChatGPT, the image.
I did it in ChatGPT, the image.
Lex Fridman
The image?
The image?
Rick Beato
Then I went to, then I went to Claude and I wrote the lyrics, ’cause Claude’s way better at lyrics- … than Suno is. Suno’s bad at lyrics, at least right now. So I did, I created the lyrics in Claude and then I imported the lyrics into Suno, and I had great results with the songs that it came up with. I always have to qualify that. But then I started thinking about this. People freak out about this, “Oh, this is bad, this is bad.” And then I thought, I was like, “No, who are going to be the ones that are gonna benefit from AI?” Well, the people that are already great songwriters, because you have to be able to recognize when it spits out something good versus when it spits out something that’s not that good.
Then I went to, then I went to Claude and I wrote the lyrics, ’cause Claude’s way better at lyrics- … than Suno is. Suno’s bad at lyrics, at least right now. So I did, I created the lyrics in Claude and then I imported the lyrics into Suno, and I had great results with the songs that it came up with. I always have to qualify that. But then I started thinking about this. People freak out about this, “Oh, this is bad, this is bad.” And then I thought, I was like, “No, who are going to be the ones that are gonna benefit from AI?” Well, the people that are already great songwriters, because you have to be able to recognize when it spits out something good versus when it spits out something that’s not that good.
Rick Beato
And every other song, I’ve probably created 130 song ideas, out of which there’s three good ones.
And every other song, I’ve probably created 130 song ideas, out of which there’s three good ones.
Lex Fridman
And there’s a thing that’s happening where people’s ear very quickly is becoming attuned to AI slop. And that’s actually quite fascinating. Like, for example, one of the things, there’s this viral clip going around of an AI-based, like, a soul jazz remix of songs like 50 Cent’s “Many Men,” and I think it is super impressive. And there’s a different pipeline actually. It’s a tricky pipeline to how to pull that off, and I think a lot of the creativity in that, even that kind of remixing, is in the pipeline of how you actually do that, because there’s actually a lot of manual stuff in that pipeline. But I think ironically it’s very cool at first, but when you listen to it for a while you understand that this is AI slop.
And there’s a thing that’s happening where people’s ear very quickly is becoming attuned to AI slop. And that’s actually quite fascinating. Like, for example, one of the things, there’s this viral clip going around of an AI-based, like, a soul jazz remix of songs like 50 Cent’s “Many Men,” and I think it is super impressive. And there’s a different pipeline actually. It’s a tricky pipeline to how to pull that off, and I think a lot of the creativity in that, even that kind of remixing, is in the pipeline of how you actually do that, because there’s actually a lot of manual stuff in that pipeline. But I think ironically it’s very cool at first, but when you listen to it for a while you understand that this is AI slop.
Lex Fridman
For a soul remix, it actually lacks soul. But it made me think of, like, when I listen to soul or blues, I think I really want, in that case, to know… I don’t want an AI B.B. King, I want the real B.B. King. And if I know if any AI is involved in the B.B. King process, I’m tuning out. And I don’t think I’m being a curmudgeonly old dude in that. I think we humans want authenticity.
For a soul remix, it actually lacks soul. But it made me think of, like, when I listen to soul or blues, I think I really want, in that case, to know… I don’t want an AI B.B. King, I want the real B.B. King. And if I know if any AI is involved in the B.B. King process, I’m tuning out. And I don’t think I’m being a curmudgeonly old dude in that. I think we humans want authenticity.
Rick Beato
So when AI, when I first started making these AI videos, it started back in 2023, I made my first one, and I would take my phone, come up in the kitchen, I’d play a song, and then my youngest, Layla, and I have three kids, and my oldest, Dylan, as soon as I played it, “Why are you listening to AI?” And it’s like, oh my God, instantly. It’s like, how do you know? Oh, it has this ringing sound in the thing. So it took me probably about four or five days to figure out, “Okay, what are they hearing that I’m not hearing?” So I did it, I separated all the parts, and what they’re hearing was the artifacts that are in the vocal reverb. That sound that were… That made incomplete-
So when AI, when I first started making these AI videos, it started back in 2023, I made my first one, and I would take my phone, come up in the kitchen, I’d play a song, and then my youngest, Layla, and I have three kids, and my oldest, Dylan, as soon as I played it, “Why are you listening to AI?” And it’s like, oh my God, instantly. It’s like, how do you know? Oh, it has this ringing sound in the thing. So it took me probably about four or five days to figure out, “Okay, what are they hearing that I’m not hearing?” So I did it, I separated all the parts, and what they’re hearing was the artifacts that are in the vocal reverb. That sound that were… That made incomplete-
Rick Beato
It just couldn’t do the ambiances correctly, right? Because it’s trained on… A lot of these AI programs are trained on very low bit-rate MP3s, right? So, they feed all this stuff in there. So, they’re getting really inferior information in the training process, whereas now when they make these deals with the major labels, they’ll get the multitracks, and they’ll get high-quality WAV files to train from, right? And whoever opts in, they get the solo vocal tracks. You know, if Ed Sheeran wants to do it, or Drake, or whoever wants to give their voice to it, let it do its thing, and then get the royalties from it.
It just couldn’t do the ambiances correctly, right? Because it’s trained on… A lot of these AI programs are trained on very low bit-rate MP3s, right? So, they feed all this stuff in there. So, they’re getting really inferior information in the training process, whereas now when they make these deals with the major labels, they’ll get the multitracks, and they’ll get high-quality WAV files to train from, right? And whoever opts in, they get the solo vocal tracks. You know, if Ed Sheeran wants to do it, or Drake, or whoever wants to give their voice to it, let it do its thing, and then get the royalties from it.
Rick Beato
I’m not saying that any of them are doing it. I’m just giving an example. But every time that I would do it, I could be down the hall, and I would play something in my phone just to see if they’ll… “Why are you listening to AI?” They can instantly tell. Then it eventually started getting better. And then, it’d be like, “Is this AI?” I’d be in the car with Layla coming back from taekwondo practice, and she’s like, “Is this AI? Why? Does it sound like AI? Sounds like it could be AI.” And I’d be like, “Yeah, it’s AI.” She’s like, “Oh, it’s getting better.”
I’m not saying that any of them are doing it. I’m just giving an example. But every time that I would do it, I could be down the hall, and I would play something in my phone just to see if they’ll… “Why are you listening to AI?” They can instantly tell. Then it eventually started getting better. And then, it’d be like, “Is this AI?” I’d be in the car with Layla coming back from taekwondo practice, and she’s like, “Is this AI? Why? Does it sound like AI? Sounds like it could be AI.” And I’d be like, “Yeah, it’s AI.” She’s like, “Oh, it’s getting better.”
Rick Beato
And then I did this song for… It was an NPR interview, and I created a song with a fake artist. And the song was called “Neon Ghosts,” and I played it for Layla in the car. She’s like, “Can you separate the tracks?” I said, “Yeah, I have them separated back home.” “Okay, I want to go down to hear it.” So, we go down to the studio, and I play it for her, and she listens to the soloed vocal. She said, “Wow, this is really realistic.” “This is very hard to tell, even with the soloed vocal.”
And then I did this song for… It was an NPR interview, and I created a song with a fake artist. And the song was called “Neon Ghosts,” and I played it for Layla in the car. She’s like, “Can you separate the tracks?” I said, “Yeah, I have them separated back home.” “Okay, I want to go down to hear it.” So, we go down to the studio, and I play it for her, and she listens to the soloed vocal. She said, “Wow, this is really realistic.” “This is very hard to tell, even with the soloed vocal.”
Lex Fridman
I think the room for creativity right now for humans is lyrics. It seems like the lyrics that are being generated, they lack soul somehow. And that’s- I don’t know the words correctly. I mean, they can be incredibly sophisticated, but there’s something, the edge is not there. Some kind of edge that we want in our lyrics. Some kind of surprise, but not cringe or not cliché. Or something truly novel in the lyrics. But if that’s the case, it’s kind of sad that that’s where the creativity has to come from, but not from the music. Because then if we can create very realistic music that sounds really damn good, where’s the role of the musician there?
I think the room for creativity right now for humans is lyrics. It seems like the lyrics that are being generated, they lack soul somehow. And that’s- I don’t know the words correctly. I mean, they can be incredibly sophisticated, but there’s something, the edge is not there. Some kind of edge that we want in our lyrics. Some kind of surprise, but not cringe or not cliché. Or something truly novel in the lyrics. But if that’s the case, it’s kind of sad that that’s where the creativity has to come from, but not from the music. Because then if we can create very realistic music that sounds really damn good, where’s the role of the musician there?
Rick Beato
I think the role of the musician is that in actually… If they use AI to assist them in coming up with ideas, they could as a creation tool. Then the musician… Like, some of the stuff is just not high quality sonically. So, the musician goes in and redoes stuff and changes things and adds parts, and then they actually do music production, and maybe they re-sing the parts, and they change the stuff. And then it’s just basically like an idea generator, and I think that that’s a great use of AI, is for that.
I think the role of the musician is that in actually… If they use AI to assist them in coming up with ideas, they could as a creation tool. Then the musician… Like, some of the stuff is just not high quality sonically. So, the musician goes in and redoes stuff and changes things and adds parts, and then they actually do music production, and maybe they re-sing the parts, and they change the stuff. And then it’s just basically like an idea generator, and I think that that’s a great use of AI, is for that.
Lex Fridman
But see, if you do that, does it make you sad that you don’t necessarily need to learn instruments? So, basically, you can… I mean, you can think of it as a different kind of instrument, but you can write lyrics. You can hum the melody. You can just hum parts. You know? And then do an A/B kind of thing. Just kind of rhythm this kind of, and stitch them together. And never actually have your fingers on a guitar or fingers on a drumstick.
But see, if you do that, does it make you sad that you don’t necessarily need to learn instruments? So, basically, you can… I mean, you can think of it as a different kind of instrument, but you can write lyrics. You can hum the melody. You can just hum parts. You know? And then do an A/B kind of thing. Just kind of rhythm this kind of, and stitch them together. And never actually have your fingers on a guitar or fingers on a drumstick.
Rick Beato
That’s why I’m not gonna use AI, Lex, is for that reason, because to me, it’s just boring. And I-
That’s why I’m not gonna use AI, Lex, is for that reason, because to me, it’s just boring. And I-
Lex Fridman
Yeah, it is.
Yeah, it is.
Rick Beato
… when I use it, it’s just like, “Eh.” But I used it for about a month or so, just because I was making videos. And I was trying to see how it’s advancing. Every three or four months, I’ll sit down, and I’ll see whatever new versions they have. And I’ll write some songs. I’ll prompt some songs and see what they come up with and see if they’re improving on the things. But ultimately, I don’t find it interesting to use.
… when I use it, it’s just like, “Eh.” But I used it for about a month or so, just because I was making videos. And I was trying to see how it’s advancing. Every three or four months, I’ll sit down, and I’ll see whatever new versions they have. And I’ll write some songs. I’ll prompt some songs and see what they come up with and see if they’re improving on the things. But ultimately, I don’t find it interesting to use.
Lex Fridman
I hear you. You’re a bit old school.
I hear you. You’re a bit old school.
Rick Beato
I’m old school.
I’m old school.
Lex Fridman
As am I. I’m trying to think about the future, and I think it’s still, even in the future, also going to be boring. I think there’s something-
As am I. I’m trying to think about the future, and I think it’s still, even in the future, also going to be boring. I think there’s something-
Rick Beato
I agree.
I agree.
Lex Fridman
… fundamentally boring about it, and I’ve been trying to figure it out. For example, I use it a lot for—more and more and more for programming. So, for building stuff. And there, it’s not about the… The final output is not the code. The output is what the code creates. And there, it’s extremely useful. It doesn’t matter if it’s boring or not, it’s useful. But when the final output is the thing that AI creates, which it would be in music, then there’s something about us that just, like… We know. There is something boring about it.
… fundamentally boring about it, and I’ve been trying to figure it out. For example, I use it a lot for—more and more and more for programming. So, for building stuff. And there, it’s not about the… The final output is not the code. The output is what the code creates. And there, it’s extremely useful. It doesn’t matter if it’s boring or not, it’s useful. But when the final output is the thing that AI creates, which it would be in music, then there’s something about us that just, like… We know. There is something boring about it.
Lex Fridman
We want to celebrate and see the thing that’s hard to create. And if AI can just text a song, “Generate a top 10 hit,” we’ll quickly lose value for that, I think. And so, we’ll want raw. Whatever shape that raw takes, I want to say raw talent, but that raw talent of any kind. And perhaps… It would make me a little bit sad, but that’s also awesome. Perhaps the new kind of raw talent that civilization is asking for is how to make great TikToks. Maybe that’s what raw talent looks like. It makes me a little bit sad because I’m a huge fan of long-form. But that also… Creating TikToks is also talent.
We want to celebrate and see the thing that’s hard to create. And if AI can just text a song, “Generate a top 10 hit,” we’ll quickly lose value for that, I think. And so, we’ll want raw. Whatever shape that raw takes, I want to say raw talent, but that raw talent of any kind. And perhaps… It would make me a little bit sad, but that’s also awesome. Perhaps the new kind of raw talent that civilization is asking for is how to make great TikToks. Maybe that’s what raw talent looks like. It makes me a little bit sad because I’m a huge fan of long-form. But that also… Creating TikToks is also talent.
Rick Beato
It is a talent. Absolutely. When I see anything that’s AI generated, I instantly recognize it. Any video, I’m like, “Ugh, boring, boring, boring.” And my kids do the same thing. They just have no interest in engaging with it. As soon as they recognize it, and they can spot it a mile away— …and they’re just like, “Boring, boring, boring, boring, boring.” And then they don’t even wanna engage with the social media platforms, which is a danger. Which I think they need to crack down on the AI slop. YouTube’s done a pretty good job on it, but it’s hard to stay on this. It gets flooded with so much of this stuff, it’s so easy to create and put up there. And to just be in the whack-a-mole thing where you’re just trying to get rid of it all is a—
It is a talent. Absolutely. When I see anything that’s AI generated, I instantly recognize it. Any video, I’m like, “Ugh, boring, boring, boring.” And my kids do the same thing. They just have no interest in engaging with it. As soon as they recognize it, and they can spot it a mile away— …and they’re just like, “Boring, boring, boring, boring, boring.” And then they don’t even wanna engage with the social media platforms, which is a danger. Which I think they need to crack down on the AI slop. YouTube’s done a pretty good job on it, but it’s hard to stay on this. It gets flooded with so much of this stuff, it’s so easy to create and put up there. And to just be in the whack-a-mole thing where you’re just trying to get rid of it all is a—
Lex Fridman
Yeah, it’s fundamentally boring. I think boring is a really good—
Yeah, it’s fundamentally boring. I think boring is a really good—
Rick Beato
Yes, boring.
Yes, boring.
Lex Fridman
And it’s annoying to have to flip through the AI slop. But I think actually, as a civilization, it’s just inspiring for authenticity ’cause you wanna be real. And being raw, which, one of the things I like about podcasts is people just shooting shit and just being themselves in the long form versus overproduced. ‘Cause I think AI is making people realize that AI is good at being overproduced. So there’ll be more.
And it’s annoying to have to flip through the AI slop. But I think actually, as a civilization, it’s just inspiring for authenticity ’cause you wanna be real. And being raw, which, one of the things I like about podcasts is people just shooting shit and just being themselves in the long form versus overproduced. ‘Cause I think AI is making people realize that AI is good at being overproduced. So there’ll be more.
Rick Beato
Let’s get that covered.
Let’s get that covered.
Lex Fridman
Yeah. Even artists, ’cause you’re saying like, yeah, they’ll use it as tools. Part of me thinks like not really. Like, I think they’ll quickly, this kind of process of generating a bunch of different options and choosing the one you like the most, I think is a really frustrating process for artists. And I think it, I think AI will definitely be used extremely effectively as a very fine-grained tool in the image domain, editing images. But not like macro editing, but very specific kinda editing that Photoshop is increasingly integrated in. I’ll mention to you offline, so the whole iZotope RX group of software that does a lot of the denoising, D- All the D, removing the wind, all the—they integrate machine learning extremely effectively—
Yeah. Even artists, ’cause you’re saying like, yeah, they’ll use it as tools. Part of me thinks like not really. Like, I think they’ll quickly, this kind of process of generating a bunch of different options and choosing the one you like the most, I think is a really frustrating process for artists. And I think it, I think AI will definitely be used extremely effectively as a very fine-grained tool in the image domain, editing images. But not like macro editing, but very specific kinda editing that Photoshop is increasingly integrated in. I’ll mention to you offline, so the whole iZotope RX group of software that does a lot of the denoising, D- All the D, removing the wind, all the—they integrate machine learning extremely effectively—
Lex Fridman
… for working with audio in different kinds of ways. There’s a bunch of different other programs that do that. Maybe for like B-roll footage and a, same thing on the audio, if you just need a little audio to create a feeling of a scene, the AI might be used there in that kinda way. But truly original stuff, eh.
… for working with audio in different kinds of ways. There’s a bunch of different other programs that do that. Maybe for like B-roll footage and a, same thing on the audio, if you just need a little audio to create a feeling of a scene, the AI might be used there in that kinda way. But truly original stuff, eh.
Rick Beato
I’ve saved videos where I’m doing, speaking over music, for example, in an interview. Somebody’s playing and we have two people speaking in lavs, but there’s so much bleed coming from the person playing- … that you can’t hear what we’re saying. And then we’ll split out the voice for that section, the two voices, separate them- … and then take the music and separate that stuff in. So it’s really helpful for things like that.
I’ve saved videos where I’m doing, speaking over music, for example, in an interview. Somebody’s playing and we have two people speaking in lavs, but there’s so much bleed coming from the person playing- … that you can’t hear what we’re saying. And then we’ll split out the voice for that section, the two voices, separate them- … and then take the music and separate that stuff in. So it’s really helpful for things like that.
Lex Fridman
And now, once again, a quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Uplift Desk for my favorite office desks, BetterHelp for mental health, LMNT for electrolytes, Fin for customer service AI agents, Shopify for selling stuff online, and Perplexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Rick Beato. So you have this video breaking down Sabrina Carpenter’s song Manchild. And you use that as an example of building up people’s intuition about the music business and how the music production for these popular songs is being done these days.
And now, once again, a quick 30-second thank you to our sponsors. Check them out in the description. It really is the best way to support this podcast. Go to lexfridman.com/sponsors. We got Uplift Desk for my favorite office desks, BetterHelp for mental health, LMNT for electrolytes, Fin for customer service AI agents, Shopify for selling stuff online, and Perplexity for curiosity-driven knowledge exploration. Choose wisely, my friends. And now, back to my conversation with Rick Beato. So you have this video breaking down Sabrina Carpenter’s song Manchild. And you use that as an example of building up people’s intuition about the music business and how the music production for these popular songs is being done these days.
Sabrina Carpenter
Lex Fridman
Who’s doing the songwriting, how’s it being done, and all that kind of stuff? I, I was wondering if you could speak to that.
Who’s doing the songwriting, how’s it being done, and all that kind of stuff? I, I was wondering if you could speak to that.
Rick Beato
In that particular song, Jack Antonoff, who is one of the writers, Amy Allen, Sabrina Carpenter, said in some awards thing that there’s an old guy on YouTube that says that Sabrina had very little to do with the song. And so he said in this clip-
In that particular song, Jack Antonoff, who is one of the writers, Amy Allen, Sabrina Carpenter, said in some awards thing that there’s an old guy on YouTube that says that Sabrina had very little to do with the song. And so he said in this clip-
Lex Fridman
You being the old guy.
You being the old guy.
Rick Beato
Me being the old guy. That, well, Sabrina really was the—she’s amazing and she’s the one that wrote everything in the song. It’s like, so my response is like, “Well, why are you guys even included on the songwriting then?”
Me being the old guy. That, well, Sabrina really was the—she’s amazing and she’s the one that wrote everything in the song. It’s like, so my response is like, “Well, why are you guys even included on the songwriting then?”
Lex Fridman
So one of the things you highlight is a lot of people are included on the list of songwriters.
So one of the things you highlight is a lot of people are included on the list of songwriters.
Rick Beato
Yeah, 10 people- … 11 people. I mean, you know. Like, why does Song of the Year have songs that are interpolations, meaning that they have melodies from other songs in their interpolation? They used to call it stealing. And then you have songs that use samples for the whole thing, like the Doechii song that’s out right now. And I said, “Look, she took a Gotye song and basically took off his melody and she created her own melody over it.” It’s like, well, it’s—I mean, it saves time for her. You don’t have to actually create a track; you just can sing over someone else’s song that was already successful.
Yeah, 10 people- … 11 people. I mean, you know. Like, why does Song of the Year have songs that are interpolations, meaning that they have melodies from other songs in their interpolation? They used to call it stealing. And then you have songs that use samples for the whole thing, like the Doechii song that’s out right now. And I said, “Look, she took a Gotye song and basically took off his melody and she created her own melody over it.” It’s like, well, it’s—I mean, it saves time for her. You don’t have to actually create a track; you just can sing over someone else’s song that was already successful.
Lex Fridman
Yeah, you pointing that out, the song Anxiety—it broke my brain.
Yeah, you pointing that out, the song Anxiety—it broke my brain.
Rick Beato
I mean, it’s so absurd.
I mean, it’s so absurd.
Lex Fridman
It, yeah, it just feels unfair. It feels—it’s a good song, but it was also a good song before, and before that, it was also a good song.
It, yeah, it just feels unfair. It feels—it’s a good song, but it was also a good song before, and before that, it was also a good song.
Rick Beato
Right, 2011, or Luiz Bonfá in 1967. So why is that considered to be in the top songs of the year? It’s like, come on, you can’t find another song that’s not based on that? That’s ridiculous. And Doechii has some really good songs- … on her record.
Right, 2011, or Luiz Bonfá in 1967. So why is that considered to be in the top songs of the year? It’s like, come on, you can’t find another song that’s not based on that? That’s ridiculous. And Doechii has some really good songs- … on her record.
Lex Fridman
Yeah, but why are these the ones that are coming to the top, right?
Yeah, but why are these the ones that are coming to the top, right?
Rick Beato
Well, you know.
Well, you know.
Lex Fridman
This is interesting. Hey, that might be just a criticism of the machinery of the business-
This is interesting. Hey, that might be just a criticism of the machinery of the business-
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
… that drives them. It’s not necessarily, like, a lot of these folks are really good musicians. First of all, I think a lot of them are also good, like the actual songs they make at the top are good. I’m a big fan of Bruno Mars. He’s a great songwriter and is a great musician all around.
… that drives them. It’s not necessarily, like, a lot of these folks are really good musicians. First of all, I think a lot of them are also good, like the actual songs they make at the top are good. I’m a big fan of Bruno Mars. He’s a great songwriter and is a great musician all around.
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
You know, he is Michael Jackson reincarnated. I mean, he’s-
You know, he is Michael Jackson reincarnated. I mean, he’s-
Rick Beato
Super, super talented guy.
Super, super talented guy.
Lex Fridman
Incredible, right? You mentioned Billie Eilish and her brother write a lot of the songs.
Incredible, right? You mentioned Billie Eilish and her brother write a lot of the songs.
Rick Beato
So good. Yeah, super talented.
So good. Yeah, super talented.
Lex Fridman
I mean, Taylor Swift is unlike anything. I mean, that’s a historic figure in music, but she’s a fundamentally, at least originally, a singer-songwriter. So that’s a, I mean, I’m sorry, but that is a, like, of the kind of music that Rick Beato gives props to. She’s the—she carries the flame forward.
I mean, Taylor Swift is unlike anything. I mean, that’s a historic figure in music, but she’s a fundamentally, at least originally, a singer-songwriter. So that’s a, I mean, I’m sorry, but that is a, like, of the kind of music that Rick Beato gives props to. She’s the—she carries the flame forward.
Rick Beato
She works on her own songs, absolutely, and she, but she never has more than two co-writers on things.
She works on her own songs, absolutely, and she, but she never has more than two co-writers on things.
YouTube copyright strikes
Lex Fridman
Wanna take a quick bathroom break? Okay. I have to ask you about this complexity that you’re facing on a basically daily basis. I think it’s a challenge a lot of YouTube folks experience, but you’re just so viscerally experiencing it because a lot of what you do on your channel is celebrate music, broadly. And so, as part of that process, you have to sometimes show clips of music, and I think all of that falls under fair use, quite obviously. And so you get all these YouTube copyright claims, and for folks who don’t know, if you get three of those, each one of those can be a strike on the channel and could take down your channel. And you get some insane amount. You said you got, like—I think I had a similar thing on my Rick Rubin episode—like, I think you said 13.
Wanna take a quick bathroom break? Okay. I have to ask you about this complexity that you’re facing on a basically daily basis. I think it’s a challenge a lot of YouTube folks experience, but you’re just so viscerally experiencing it because a lot of what you do on your channel is celebrate music, broadly. And so, as part of that process, you have to sometimes show clips of music, and I think all of that falls under fair use, quite obviously. And so you get all these YouTube copyright claims, and for folks who don’t know, if you get three of those, each one of those can be a strike on the channel and could take down your channel. And you get some insane amount. You said you got, like—I think I had a similar thing on my Rick Rubin episode—like, I think you said 13.
Lex Fridman
13. So, what, can you just speak to this whole thing? You’ve been in a constant battle, WMG, UMG, all the, all, all-
13. So, what, can you just speak to this whole thing? You’ve been in a constant battle, WMG, UMG, all the, all, all-
Rick Beato
All the, all the three-letter name-
All the, all the three-letter name-
Lex Fridman
All the s-
All the s-
Rick Beato
… record labels, right?
… record labels, right?
Lex Fridman
The music business people, so, what’s the story there?
The music business people, so, what’s the story there?
Rick Beato
Well, this has been going on since the beginning of my channel, and I’ve made videos periodically. When I first started, it was just instant blocks. So you never knew back in—I started, it’ll be 10 years in June. So, when I’d play music in a video, YouTubers were not playing music in videos because they didn’t, because of the Content ID things and the take-downs and stuff. So, I would play music, and I’d just see what happens, and then you get a Content ID claim, or you realize the people were, quote-unquote, “blockers,” and I came up with that term that they would block your video, take down your video.
Well, this has been going on since the beginning of my channel, and I’ve made videos periodically. When I first started, it was just instant blocks. So you never knew back in—I started, it’ll be 10 years in June. So, when I’d play music in a video, YouTubers were not playing music in videos because they didn’t, because of the Content ID things and the take-downs and stuff. So, I would play music, and I’d just see what happens, and then you get a Content ID claim, or you realize the people were, quote-unquote, “blockers,” and I came up with that term that they would block your video, take down your video.
Rick Beato
And I realized at first it was, like, anything Guns N’ Roses, which is still the case, Guns N’ Roses, AC/DC, I mean, many bands, Fleetwood Mac, Led Zeppelin, and then, and then something happened. There was a guy on the skateboard on TikTok that had the Ocean Spray thing and he was listening to- … Dreams by Fleetwood Mac. And that blew up and became a number one song again. And the labels then realized—I mean, I’d made many videos about why this is wrong, and it should be fair use and everything. Well, because of that, the labels were like, “Ooh, maybe we should rethink this.” And then they just started demonetizing videos.
And I realized at first it was, like, anything Guns N’ Roses, which is still the case, Guns N’ Roses, AC/DC, I mean, many bands, Fleetwood Mac, Led Zeppelin, and then, and then something happened. There was a guy on the skateboard on TikTok that had the Ocean Spray thing and he was listening to- … Dreams by Fleetwood Mac. And that blew up and became a number one song again. And the labels then realized—I mean, I’d made many videos about why this is wrong, and it should be fair use and everything. Well, because of that, the labels were like, “Ooh, maybe we should rethink this.” And then they just started demonetizing videos.
Lex Fridman
Demonetized means they get all the money that you make.
Demonetized means they get all the money that you make.
Rick Beato
They get all the money. In a one-hour video, if they, if you use 20 seconds of a clip- … they get all the money. Okay? So, I hired a lawyer finally after the Rick Rubin video, ’cause I thought it was ridiculous. I go over to Tuscany, I interview Rick at his house, and I hired a lawyer to fight this, who I’m gonna have on my channel. I don’t wanna say who it is, but he’s another YouTuber. And he had approached me a couple years ago, and it’s not cheap to do.
They get all the money. In a one-hour video, if they, if you use 20 seconds of a clip- … they get all the money. Okay? So, I hired a lawyer finally after the Rick Rubin video, ’cause I thought it was ridiculous. I go over to Tuscany, I interview Rick at his house, and I hired a lawyer to fight this, who I’m gonna have on my channel. I don’t wanna say who it is, but he’s another YouTuber. And he had approached me a couple years ago, and it’s not cheap to do.
Lex Fridman
Oh, you’re gonna do, like, a public interview with him?
Oh, you’re gonna do, like, a public interview with him?
Rick Beato
I’m gonna do an interview- … with him, yes.
I’m gonna do an interview- … with him, yes.
Lex Fridman
Awesome. Okay.
Awesome. Okay.
Rick Beato
I talked to him today about it, actually.
I talked to him today about it, actually.
Lex Fridman
I can’t wait. That’d be great.
I can’t wait. That’d be great.
Rick Beato
So he said, “You should fight these ’cause every single one of them is fair use.” And he went through my entire catalog. I have 2,100 videos, and he’s fought 4,000 Content ID claims and won every single one of them. 4,000. That’s a lot—I mean, when I do top 20 guitar solos, there’s 20 Content ID claims, you know? It’s, and it’s either, it can be either from the sound recording, if I used that, or if I just play it, it can be from the publisher.
So he said, “You should fight these ’cause every single one of them is fair use.” And he went through my entire catalog. I have 2,100 videos, and he’s fought 4,000 Content ID claims and won every single one of them. 4,000. That’s a lot—I mean, when I do top 20 guitar solos, there’s 20 Content ID claims, you know? It’s, and it’s either, it can be either from the sound recording, if I used that, or if I just play it, it can be from the publisher.
Lex Fridman
That’s amazing. So is there, I mean, that’s still, he’s still a lawyer, still work. Does that, is there a hopeful thing you can say about the future of-
That’s amazing. So is there, I mean, that’s still, he’s still a lawyer, still work. Does that, is there a hopeful thing you can say about the future of-
Rick Beato
Yeah, fight these Content ID claims. If it’s fair use, if you’re not just playing the song and listening to it, and, ’cause a lot of stuff that are reaction videos, or whatever, that are not, where they play the whole song, I mean, I’m using these things, and I’m talking, lot of the times it’s in interviews, or it’s in, I’m breaking down a solo, and there’s a-
Yeah, fight these Content ID claims. If it’s fair use, if you’re not just playing the song and listening to it, and, ’cause a lot of stuff that are reaction videos, or whatever, that are not, where they play the whole song, I mean, I’m using these things, and I’m talking, lot of the times it’s in interviews, or it’s in, I’m breaking down a solo, and there’s a-
Lex Fridman
Yeah. See, that’s an-
Yeah. See, that’s an-
Rick Beato
… you know
… you know
Lex Fridman
… obvious one, but even reaction videos, right? Where those-
… obvious one, but even reaction videos, right? Where those-
Rick Beato
Yeah. Even reaction videos, yes, absolutely.
Yeah. Even reaction videos, yes, absolutely.
Lex Fridman
Those are more borderline. But I don’t know. I love those videos.
Those are more borderline. But I don’t know. I love those videos.
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
Like, when a person’s just sitting there and listening to it, and they’re like, you know, like a voice teacher is listening to a vocal performance, and like-
Like, when a person’s just sitting there and listening to it, and they’re like, you know, like a voice teacher is listening to a vocal performance, and like-
Rick Beato
Yeah, but those are breakdowns.
Yeah, but those are breakdowns.
Lex Fridman
Yeah, those are breakdowns, yeah.
Yeah, those are breakdowns, yeah.
Rick Beato
I think that the Content ID stuff that was happening with these major labels, they would hire third parties- … that would go out, use AI, and go and anytime they detect anything, they always go to the biggest channels first to get the most views. Makes sense and stuff. And they would claim everything that they could, and historically, YouTubers never would fight back. They were like, “Oh, this is easy money.” YouTubers never fight back at these things, because they’re afraid to have their channels taken down. So-
I think that the Content ID stuff that was happening with these major labels, they would hire third parties- … that would go out, use AI, and go and anytime they detect anything, they always go to the biggest channels first to get the most views. Makes sense and stuff. And they would claim everything that they could, and historically, YouTubers never would fight back. They were like, “Oh, this is easy money.” YouTubers never fight back at these things, because they’re afraid to have their channels taken down. So-
Lex Fridman
Right, you gotta say, “Hold my beer.”
Right, you gotta say, “Hold my beer.”
Rick Beato
There you go.
There you go.
Lex Fridman
So, I mean, it’s important. So, you-
So, I mean, it’s important. So, you-
Rick Beato
I mean, it took me years though, Lex. I didn’t… I’ve been doing this… So, I’ve been doing it for one year now, and I’m nine years—almost 10 years into my channel. So, it took me that long.
I mean, it took me years though, Lex. I didn’t… I’ve been doing this… So, I’ve been doing it for one year now, and I’m nine years—almost 10 years into my channel. So, it took me that long.
Spotify
Lex Fridman
I mean, hopefully, there’s a ripple effect also. It’s not just your situation. Hopefully, you don’t have to deal with this for much longer. How has Spotify changed music? Sometimes we highlight the fact that they changed the nature of music and that the scarcity is not there. But also, a lot of it’s like every kind of music is available and so fast and it’s so easy. It’s easy to explore.
I mean, hopefully, there’s a ripple effect also. It’s not just your situation. Hopefully, you don’t have to deal with this for much longer. How has Spotify changed music? Sometimes we highlight the fact that they changed the nature of music and that the scarcity is not there. But also, a lot of it’s like every kind of music is available and so fast and it’s so easy. It’s easy to explore.
Rick Beato
It’s a commodity. It’s like turning on a water faucet.
It’s a commodity. It’s like turning on a water faucet.
Lex Fridman
Do you think-
Do you think-
Rick Beato
Once you get going-
Once you get going-
Lex Fridman
… that there’s some good to… I mean, there’s a lot of good to that, right? Well, have you… Did you go through that whole process? I still remember where I had to basically throw away the albums.
… that there’s some good to… I mean, there’s a lot of good to that, right? Well, have you… Did you go through that whole process? I still remember where I had to basically throw away the albums.
Rick Beato
I never did that. When? After you uploaded them into your computer?
I never did that. When? After you uploaded them into your computer?
Lex Fridman
Yeah. So, there’s that two-step process. One, there’s like the hard albums, CDs for me.
Yeah. So, there’s that two-step process. One, there’s like the hard albums, CDs for me.
Rick Beato
CDs, yep.
CDs, yep.
Lex Fridman
And then, and then you upload them into your computer. And you save them. And then how do you put it? Allegedly, a friend of yours pirates some extra songs. And puts them on the computer. So, you have… but you have your stash on the computer. You’re like, “This is my finely selected stash of greatness.” Uh, sometimes organized by album, sometimes not. And the big moment for me that was really difficult to do, really difficult to do, is throw away that stash and switch to Spotify.
And then, and then you upload them into your computer. And you save them. And then how do you put it? Allegedly, a friend of yours pirates some extra songs. And puts them on the computer. So, you have… but you have your stash on the computer. You’re like, “This is my finely selected stash of greatness.” Uh, sometimes organized by album, sometimes not. And the big moment for me that was really difficult to do, really difficult to do, is throw away that stash and switch to Spotify.
Lex Fridman
Switch to streaming, and basically, rebuild the stash of playlists and all this kind of stuff. And that, it was heartbreaking ’cause so much love and effort went into that. Both the CD, the stashing of the CD, and the stashing of the MP3s in the computer. And then in Spotify, it just seems just effortless. But it helped me discover all kinds of artists I never would have discovered otherwise. And Pandora, I used a lot. Pandora is more prioritizing on the discovery part versus organization part. And that was really wonderful.
Switch to streaming, and basically, rebuild the stash of playlists and all this kind of stuff. And that, it was heartbreaking ’cause so much love and effort went into that. Both the CD, the stashing of the CD, and the stashing of the MP3s in the computer. And then in Spotify, it just seems just effortless. But it helped me discover all kinds of artists I never would have discovered otherwise. And Pandora, I used a lot. Pandora is more prioritizing on the discovery part versus organization part. And that was really wonderful.
Rick Beato
So, one of the things I… I’ll start with a positive that I like about Spotify, is that they show view count, they show play counts. Whether they’re real or not, that’s another question. But they show how many plays songs have, and that’s how the charts are based.
So, one of the things I… I’ll start with a positive that I like about Spotify, is that they show view count, they show play counts. Whether they’re real or not, that’s another question. But they show how many plays songs have, and that’s how the charts are based.
Lex Fridman
Does that give you signal that something is listened to a billion times? Does that mean something to you?
Does that give you signal that something is listened to a billion times? Does that mean something to you?
Rick Beato
Yeah. It means that it’s a popular song. Well, that’s a massive hit. There’s very few songs that have a billion plays. Now, the downside of Spotify is the way that they pay their artists. Now they’ve lumped in podcasts that are getting a cut of this streaming with the music. And you know, the search and discovery. I mean, there’s benefits of algorithms and there’s negative things of algorithms. Algorithms happen to many times pigeonhole people into listening to the same genre of music all the time, and not expanding their discovery of new music. Where you might hear on the radio back in the day where program directors would play things that they liked, right?
Yeah. It means that it’s a popular song. Well, that’s a massive hit. There’s very few songs that have a billion plays. Now, the downside of Spotify is the way that they pay their artists. Now they’ve lumped in podcasts that are getting a cut of this streaming with the music. And you know, the search and discovery. I mean, there’s benefits of algorithms and there’s negative things of algorithms. Algorithms happen to many times pigeonhole people into listening to the same genre of music all the time, and not expanding their discovery of new music. Where you might hear on the radio back in the day where program directors would play things that they liked, right?
Rick Beato
And you might hear something, “Oh, what is that?” “Oh, that’s a new Soundgarden record,” or you know, like, “Whoa, I like that. I’m gonna go check that out.” You know, something you might not have heard or something odd.
And you might hear something, “Oh, what is that?” “Oh, that’s a new Soundgarden record,” or you know, like, “Whoa, I like that. I’m gonna go check that out.” You know, something you might not have heard or something odd.
Lex Fridman
Like, one thing I really love doing on Spotify is you can… you can have radio. Meaning, like, you have a few… It’s similar to Pandora, like you can… Okay, this is gonna reveal a little too much about myself. But usually when I go work out, I’ll listen to something like Rage Against the Machine radio. I’m sorry, I need-
Like, one thing I really love doing on Spotify is you can… you can have radio. Meaning, like, you have a few… It’s similar to Pandora, like you can… Okay, this is gonna reveal a little too much about myself. But usually when I go work out, I’ll listen to something like Rage Against the Machine radio. I’m sorry, I need-
Rick Beato
What else would you listen to?
What else would you listen to?
Lex Fridman
I need motivation. Classical music? I don’t know. But yeah, it’s pretty good ’cause it recommends a bunch of other stuff I wouldn’t even know. Some of it I know, obviously, but akin to the, similar to the Rage Against the Machine-y type thing.
I need motivation. Classical music? I don’t know. But yeah, it’s pretty good ’cause it recommends a bunch of other stuff I wouldn’t even know. Some of it I know, obviously, but akin to the, similar to the Rage Against the Machine-y type thing.
Lex Fridman
It recommends a bunch of artists, and it’s like, “Oh, holy shit, that’s awesome.” So, I don’t know. That discovery works really well. So, some of it is the technology thing. But that experience was fundamentally more vibrant than I had previously with my stash. That I would just keep a stash, and I would listen to the same record over and over and over and over. But yeah, what’s lost is the, I’m sure you love this, but listening through the Led Zeppelin records, just driving in a car and listening to the whole thing all the way through. Yeah, that’s lost.
It recommends a bunch of artists, and it’s like, “Oh, holy shit, that’s awesome.” So, I don’t know. That discovery works really well. So, some of it is the technology thing. But that experience was fundamentally more vibrant than I had previously with my stash. That I would just keep a stash, and I would listen to the same record over and over and over and over. But yeah, what’s lost is the, I’m sure you love this, but listening through the Led Zeppelin records, just driving in a car and listening to the whole thing all the way through. Yeah, that’s lost.
Rick Beato
So, I have my old iTunes libraries from 2005- … that I’ve saved. The CDs that I uploaded into my computer. Anytime I do the, I play songs on my… When I’m doing interview, I always play WAV files, I put them in. And it’s funny that when I interview a mixer, I interviewed this mixing engineer, Andy Wallace, and people comment, “Wow, the song sounded amazing.” And you go, “Well, not only are they great mixes that he did, but I’m using WAV files in there.” And people notice the, and these are WAV files from original encoding. Not, not remastered things that Spotify keeps doing- … and adding a bunch more top end and things like that. That these are the-
So, I have my old iTunes libraries from 2005- … that I’ve saved. The CDs that I uploaded into my computer. Anytime I do the, I play songs on my… When I’m doing interview, I always play WAV files, I put them in. And it’s funny that when I interview a mixer, I interviewed this mixing engineer, Andy Wallace, and people comment, “Wow, the song sounded amazing.” And you go, “Well, not only are they great mixes that he did, but I’m using WAV files in there.” And people notice the, and these are WAV files from original encoding. Not, not remastered things that Spotify keeps doing- … and adding a bunch more top end and things like that. That these are the-
Lex Fridman
Oh, I see.
Oh, I see.
Rick Beato
… these are actually the original WAV files from off the CD that I ripped… 20 years ago.
… these are actually the original WAV files from off the CD that I ripped… 20 years ago.
Lex Fridman
What’s your current… And people are really curious about that, so what’s your current stack? What are the tools you use? What’s your DAW? What’s the audio interface? What are the mics?
What’s your current… And people are really curious about that, so what’s your current stack? What are the tools you use? What’s your DAW? What’s the audio interface? What are the mics?
Rick Beato
So I use Pro Tools.
So I use Pro Tools.
Lex Fridman
Pro Tools stuff.
Pro Tools stuff.
Rick Beato
For the most part, but I also use Logic- … And Ableton. I’ve got all those.
For the most part, but I also use Logic- … And Ableton. I’ve got all those.
Lex Fridman
So you’re mostly on a Mac?
So you’re mostly on a Mac?
Rick Beato
I’m only on a Mac.
I’m only on a Mac.
Lex Fridman
Only on a Mac.
Only on a Mac.
Rick Beato
Only on a Mac.
Only on a Mac.
Lex Fridman
I’m only the opposite.
I’m only the opposite.
Rick Beato
Although we have multiple PCs, ’cause my kids use PCs.
Although we have multiple PCs, ’cause my kids use PCs.
Lex Fridman
Yeah, just to rebel.
Yeah, just to rebel.
Rick Beato
They do it for gaming. They like to game.
They do it for gaming. They like to game.
Lex Fridman
Right, that’s true. But like in terms of editing, I hate how good Mac is-
Right, that’s true. But like in terms of editing, I hate how good Mac is-
Rick Beato
So good.
So good.
Lex Fridman
… at just integrating. The, the hardware and the software just work well together. Both on the video en-
… at just integrating. The, the hardware and the software just work well together. Both on the video en-
Rick Beato
If I didn’t have a Mac, honestly, I wouldn’t be talking to you right now. Because I got a G3 that’s… So the only good thing that a major label did for me is when my band was on UMG and they bought me a G3 and an SM7 and Pro Tools Digi 001, the first prosumer Pro Tools thing. And I learned how to use Pro Tools, and that allowed me to learn how to edit video and become a record producer. So I gotta give it to Macs for that.
If I didn’t have a Mac, honestly, I wouldn’t be talking to you right now. Because I got a G3 that’s… So the only good thing that a major label did for me is when my band was on UMG and they bought me a G3 and an SM7 and Pro Tools Digi 001, the first prosumer Pro Tools thing. And I learned how to use Pro Tools, and that allowed me to learn how to edit video and become a record producer. So I gotta give it to Macs for that.
Lex Fridman
So Pro Tools, I mean, that’s still the standard.
So Pro Tools, I mean, that’s still the standard.
Rick Beato
That’s kinda the industry standard, yeah.
That’s kinda the industry standard, yeah.
Lex Fridman
I gotta ask you ’cause I know… I’ve never used Pro Tools. I’ve used… Again, I’m a caveman. I’ve used REAPER, I’ve used Studio One, that’s the most recent that I’ve used that. And- … for the most time I’ve used Ableton Live. I feel like I’m using 1% of the power of the tool. Like, Ableton Live makes me feel like I’m literally just pressing the record button.
I gotta ask you ’cause I know… I’ve never used Pro Tools. I’ve used… Again, I’m a caveman. I’ve used REAPER, I’ve used Studio One, that’s the most recent that I’ve used that. And- … for the most time I’ve used Ableton Live. I feel like I’m using 1% of the power of the tool. Like, Ableton Live makes me feel like I’m literally just pressing the record button.
Rick Beato
Ableton’s amazing. It really is.
Ableton’s amazing. It really is.
Lex Fridman
It is. But I feel like the… It, I mean, it’s designed for people that are doing like all kinds of MIDI stuff, and like looping and the—what is it?—the push buttons with the, with the beats. And the, it’s, it’s… I mean, I sound, I sound really out of touch. But it, it’s just the power is incredible. Also, it’s, I think it’s not just for recording, it’s also for live performances. So this is why Studio One has been a little bit nicer for me, because it’s simpler, made for recording more so.
It is. But I feel like the… It, I mean, it’s designed for people that are doing like all kinds of MIDI stuff, and like looping and the—what is it?—the push buttons with the, with the beats. And the, it’s, it’s… I mean, I sound, I sound really out of touch. But it, it’s just the power is incredible. Also, it’s, I think it’s not just for recording, it’s also for live performances. So this is why Studio One has been a little bit nicer for me, because it’s simpler, made for recording more so.
Rick Beato
Any DAW that you get used to, Lex, that’s-
Any DAW that you get used to, Lex, that’s-
Lex Fridman
Just use anything.
Just use anything.
Rick Beato
… using it, yeah. And, and- … you have to become a master at the things. If you wanna be a recording engineer or producer, you, you become an expert. A lot of the… You know, Finneas and Billie Eilish, I think that they use Logic, that’s their DAW that they like to use. And Logic, you know, a lot of pros use Logic. You know, I fire up Logic every couple days and I use it for things. I have it on my laptop here and I have Pro Tools and Logic on my laptop. I use both. I use Pro Tools mostly though.
… using it, yeah. And, and- … you have to become a master at the things. If you wanna be a recording engineer or producer, you, you become an expert. A lot of the… You know, Finneas and Billie Eilish, I think that they use Logic, that’s their DAW that they like to use. And Logic, you know, a lot of pros use Logic. You know, I fire up Logic every couple days and I use it for things. I have it on my laptop here and I have Pro Tools and Logic on my laptop. I use both. I use Pro Tools mostly though.
Lex Fridman
But Pro Tools, that’s where you feel at home?
But Pro Tools, that’s where you feel at home?
Rick Beato
Oh, yeah. I’m an expert in Pro Tools.
Oh, yeah. I’m an expert in Pro Tools.
Lex Fridman
Are you using any emulation? Any amp sims or it’s all real amps?
Are you using any emulation? Any amp sims or it’s all real amps?
Rick Beato
No, I use amp sims. On my laptop here when I travel and things like that, I use Neural DSP, which I just did a video at their headquarters in Helsinki. And their CEO, Doug Castro, is a friend of mine. I actually talked to him today as a matter of fact. And I have a Kemper amp sim, you know, a modeler. I have an Axe-Fx, I’ve got a Helix, I pretty much have all these things. But for me, I can… I have 100 amps in my studio, so… And I have mics set up all the time, and cabinets, and stuff.
No, I use amp sims. On my laptop here when I travel and things like that, I use Neural DSP, which I just did a video at their headquarters in Helsinki. And their CEO, Doug Castro, is a friend of mine. I actually talked to him today as a matter of fact. And I have a Kemper amp sim, you know, a modeler. I have an Axe-Fx, I’ve got a Helix, I pretty much have all these things. But for me, I can… I have 100 amps in my studio, so… And I have mics set up all the time, and cabinets, and stuff.
Lex Fridman
Oh, what do you mean?
Oh, what do you mean?
Rick Beato
I have 100 amplifiers. Real amplifiers.
I have 100 amplifiers. Real amplifiers.
Lex Fridman
Real? Wait, sorry, 100?
Real? Wait, sorry, 100?
Rick Beato
I have 100, yeah. About 100, maybe 95.
I have 100, yeah. About 100, maybe 95.
Lex Fridman
How does one go get to that level?
How does one go get to that level?
Rick Beato
Collecting and being… I’ll be 64 in April, so-
Collecting and being… I’ll be 64 in April, so-
Lex Fridman
So you just don’t let go?
So you just don’t let go?
Rick Beato
I don’t let go, no.
I don’t let go, no.
Lex Fridman
Why would you get to 100? Like is it, is it tone difference, the-
Why would you get to 100? Like is it, is it tone difference, the-
Rick Beato
Yes, so everything-
Yes, so everything-
Lex Fridman
You know the tone difference?
You know the tone difference?
Rick Beato
… does one thing really well. And so it’d be like, okay, so I have this Marshall JCM800 that’s modded that does this one thing. It’s got great mids and it’s good for this kind of a tune, so I will pull that out. Then it’s like, no, I need more of like a scooped metal sound that’s more like Metallica or Dream Theater or something, so, oh, I’m gonna pull out my Mesa/Boogie. Or I need something that’s chimey that’s more like Brian May or like The Edge, I’m gonna pull out my Vox AC30. So everything—and, and that’s why I have so many amps, because they all do… Every amp I have does one thing really well. If it doesn’t do it well, I get rid of it. And I’m down to 100.
… does one thing really well. And so it’d be like, okay, so I have this Marshall JCM800 that’s modded that does this one thing. It’s got great mids and it’s good for this kind of a tune, so I will pull that out. Then it’s like, no, I need more of like a scooped metal sound that’s more like Metallica or Dream Theater or something, so, oh, I’m gonna pull out my Mesa/Boogie. Or I need something that’s chimey that’s more like Brian May or like The Edge, I’m gonna pull out my Vox AC30. So everything—and, and that’s why I have so many amps, because they all do… Every amp I have does one thing really well. If it doesn’t do it well, I get rid of it. And I’m down to 100.
Lex Fridman
Down to 100. It’s only 100. But it-
Down to 100. It’s only 100. But it-
Rick Beato
I can get by with probably 75.
I can get by with probably 75.
Guitars
Lex Fridman
Come on, but you, then you’re really running the risk of not having just the right amps. But you’re using emulation, so that’s great. I mean, and that… But there’s the other side of it which is the guitar. I told you offline, I think having multiple guitars is cheating, but whatever. Nobody agrees with me on this. I only have like one… I do have some side pieces but one main… The greatest gi-
Come on, but you, then you’re really running the risk of not having just the right amps. But you’re using emulation, so that’s great. I mean, and that… But there’s the other side of it which is the guitar. I told you offline, I think having multiple guitars is cheating, but whatever. Nobody agrees with me on this. I only have like one… I do have some side pieces but one main… The greatest gi-
Rick Beato
The Strat? What do you play?
The Strat? What do you play?
Lex Fridman
The Strat, yeah.
The Strat, yeah.
Rick Beato
The Strat, yeah.
The Strat, yeah.
Lex Fridman
American Strat. I said I would never do this, but I was in a guitar store. I live next to a guitar store in Cambridge, and one day… I would always stop by, I don’t know why. I just to look at the guitars, and I don’t really know why exactly, just to be in the aura of these great instruments. And they brought in this American Strat that had these different shades of… It was like a silver. And I just… I’ve never had this feeling. They talk about love at first sight. I just fell in love with the guitar. Can you just speak to the kinda guitars you have and you love?
American Strat. I said I would never do this, but I was in a guitar store. I live next to a guitar store in Cambridge, and one day… I would always stop by, I don’t know why. I just to look at the guitars, and I don’t really know why exactly, just to be in the aura of these great instruments. And they brought in this American Strat that had these different shades of… It was like a silver. And I just… I’ve never had this feeling. They talk about love at first sight. I just fell in love with the guitar. Can you just speak to the kinda guitars you have and you love?
Rick Beato
I pretty much have… Mainly old school guitars, right? So I have Gibsons, I have Fenders, I have PRS guitars. And then I have… I have two Gibson acoustics. I have a 1957 Country Western that I’ve had for probably 30 some odd years. It’s a great guitar. And I have a J-45 Gibson, and I have a Martin D-28. So I only have three nice acoustics. And I have a Guild 12-string, and I have a Guild Nashville-tuned guitar. The low strings are up the octave, so the E, A, D and G are up the octave. That’s Nashville tuning. Six-string though. Like, basically what David Gilmour plays on Comfortably Numb in my video. He plays a Nashville tune, but with one variation. The low E is up two octaves. So he demonstrates actually the… And this is how he wrote Comfortably Numb. The, the chorus-
I pretty much have… Mainly old school guitars, right? So I have Gibsons, I have Fenders, I have PRS guitars. And then I have… I have two Gibson acoustics. I have a 1957 Country Western that I’ve had for probably 30 some odd years. It’s a great guitar. And I have a J-45 Gibson, and I have a Martin D-28. So I only have three nice acoustics. And I have a Guild 12-string, and I have a Guild Nashville-tuned guitar. The low strings are up the octave, so the E, A, D and G are up the octave. That’s Nashville tuning. Six-string though. Like, basically what David Gilmour plays on Comfortably Numb in my video. He plays a Nashville tune, but with one variation. The low E is up two octaves. So he demonstrates actually the… And this is how he wrote Comfortably Numb. The, the chorus-
Rick Beato
… part of it was with this particular guitar that he’s playing in the video.
… part of it was with this particular guitar that he’s playing in the video.
Lex Fridman
What can you say about, like, the different feels that the guitars, the acoustics have? Like, how do you know which one to pull out?
What can you say about, like, the different feels that the guitars, the acoustics have? Like, how do you know which one to pull out?
Rick Beato
It depends on the kind of part that I’m playing. If I want something with really tight midrange, with not, that doesn’t have a lot of low bass, this particular old Gibson that I have, the 57, I will pull that out. It’s got very balanced strings and like, you know, midrange. It doesn’t have a lot, it doesn’t have a booming bottom end, booming low E string- … or anything or A string. So it depends on what kind of sound I’m looking for. If I’m-
It depends on the kind of part that I’m playing. If I want something with really tight midrange, with not, that doesn’t have a lot of low bass, this particular old Gibson that I have, the 57, I will pull that out. It’s got very balanced strings and like, you know, midrange. It doesn’t have a lot, it doesn’t have a booming bottom end, booming low E string- … or anything or A string. So it depends on what kind of sound I’m looking for. If I’m-
Lex Fridman
So it’s more about sound versus feel?
So it’s more about sound versus feel?
Rick Beato
Yeah. All my guitars play equally well.
Yeah. All my guitars play equally well.
Lex Fridman
Okay.
Okay.
Rick Beato
I have them all set up to where they play well. I have a signature Gibson guitar that I’ve had for five years now.
I have them all set up to where they play well. I have a signature Gibson guitar that I’ve had for five years now.
Lex Fridman
When you say Gibson, Gibson Les Paul?
When you say Gibson, Gibson Les Paul?
Rick Beato
Gibson. It’s a double cut Les Paul Special. Yeah, with P-90 pickups.
Gibson. It’s a double cut Les Paul Special. Yeah, with P-90 pickups.
Lex Fridman
I don’t know what double cut means, but it sounds impressive.
I don’t know what double cut means, but it sounds impressive.
Rick Beato
That means two cutouts. Two, um-
That means two cutouts. Two, um-
Lex Fridman
Oh. Cool.
Oh. Cool.
Rick Beato
As opposed to a Les Paul that has one cut. So it’s a Les Paul Special that has two. I have it over there. My signature guitar.
As opposed to a Les Paul that has one cut. So it’s a Les Paul Special that has two. I have it over there. My signature guitar.
Lex Fridman
That’s the- That’s the… All right, nice.
That’s the- That’s the… All right, nice.
Rick Beato
Yeah. When you play this, you’re gonna be like, “Oh my God, this is butter.”
Yeah. When you play this, you’re gonna be like, “Oh my God, this is butter.”
Lex Fridman
Now, I’m again, I said it’s cheating. I don’t-
Now, I’m again, I said it’s cheating. I don’t-
Rick Beato
And what amp do you play through? Do you play through an amp sim, or do you have… What do you have, like a-
And what amp do you play through? Do you play through an amp sim, or do you have… What do you have, like a-
Lex Fridman
This is gonna be embar… Yeah. I use BIAS FX. I’m sorry.
This is gonna be embar… Yeah. I use BIAS FX. I’m sorry.
Rick Beato
Lex, I use amp sims too, so… I just got the new John Mayer Neural DSP plugin today that I have not tried out. He did a modeling of all his amplifiers that- … that Neural DSP did. And it sounds great. John played it, it sounds just like his amps.
Lex, I use amp sims too, so… I just got the new John Mayer Neural DSP plugin today that I have not tried out. He did a modeling of all his amplifiers that- … that Neural DSP did. And it sounds great. John played it, it sounds just like his amps.
Lex Fridman
Yeah, John is incredible.
Yeah, John is incredible.
Rick Beato
John’s great.
John’s great.
Lex Fridman
I’ve been fortunate enough to have dinner with him two times. And outside of being an incredible musician, he’s also conversationally just-
I’ve been fortunate enough to have dinner with him two times. And outside of being an incredible musician, he’s also conversationally just-
Rick Beato
Yes. I’ve known John since he lived in Atlanta when he got signed, and I knew John from way back then, right in the early 2000s.
Yes. I’ve known John since he lived in Atlanta when he got signed, and I knew John from way back then, right in the early 2000s.
Lex Fridman
I think he doesn’t get enough credit. Like, he’s one of the greatest living guitarists-
I think he doesn’t get enough credit. Like, he’s one of the greatest living guitarists-
Rick Beato
He’s a fantastic guitar player.
He’s a fantastic guitar player.
Lex Fridman
… in the world.
… in the world.
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
And a celebrator, if that’s a word, of great guitar playing.
And a celebrator, if that’s a word, of great guitar playing.
Rick Beato
Absolutely.
Absolutely.
Advice
Lex Fridman
By way of advice, you started your YouTube channel in your mid-50s and found incredible success. You’ve had essentially multiple careers. Is there some wisdom you can extract from that?
By way of advice, you started your YouTube channel in your mid-50s and found incredible success. You’ve had essentially multiple careers. Is there some wisdom you can extract from that?
Rick Beato
So my theory is that somebody’s gotta be successful, so why can’t it be you? That was my… When I started my channel, I mean, I didn’t start it to… It started by accident with the Dylan video. And really, so many people reached out to me. I started it six months after that viral video. So many people wrote to me, “Can you teach me this?” Pro musicians, well-known ones who you’d know. “Can you teach me this?” I can’t teach you what Dylan did, but I can teach you relative pitch, develop your ear that way. But then I had conservatories writing to me about this stuff from all over the world. “How did you teach Dylan this?” ‘Cause we made about four different videos, and they got more and more sophisticated.
So my theory is that somebody’s gotta be successful, so why can’t it be you? That was my… When I started my channel, I mean, I didn’t start it to… It started by accident with the Dylan video. And really, so many people reached out to me. I started it six months after that viral video. So many people wrote to me, “Can you teach me this?” Pro musicians, well-known ones who you’d know. “Can you teach me this?” I can’t teach you what Dylan did, but I can teach you relative pitch, develop your ear that way. But then I had conservatories writing to me about this stuff from all over the world. “How did you teach Dylan this?” ‘Cause we made about four different videos, and they got more and more sophisticated.
Rick Beato
And so I thought, “Okay, I’ll make some YouTube videos and explain this stuff.” This is, that’s really why I started, so I didn’t have to keep… I couldn’t answer the emails. There were so many of them, so I just started making videos on how to train your ear and music theory. And that’s really how I started my channel, and my wife was like, “What are you doing?” I said, “I’m making YouTube videos.” “Why?” So I don’t have to keep telling people how I did this stuff. And then all of a sudden, you know, I had 4,000 subscribers the first month, another 4,000 then. Hit 100,000 after a year, and then six months later, 200,000, then three months later, 300,000. So-
And so I thought, “Okay, I’ll make some YouTube videos and explain this stuff.” This is, that’s really why I started, so I didn’t have to keep… I couldn’t answer the emails. There were so many of them, so I just started making videos on how to train your ear and music theory. And that’s really how I started my channel, and my wife was like, “What are you doing?” I said, “I’m making YouTube videos.” “Why?” So I don’t have to keep telling people how I did this stuff. And then all of a sudden, you know, I had 4,000 subscribers the first month, another 4,000 then. Hit 100,000 after a year, and then six months later, 200,000, then three months later, 300,000. So-
Lex Fridman
I think there’s one thing that should be said, that in modern culture for young people, a lot of them will see YouTube and TikTok and Instagram, and they kinda wanna be famous. They wanna get the clicks and the views and so on, and that’s the thing they chase and optimize. I think the thing that you’re leaving unstated perhaps is that you spent many years pursuing the mastery of a craft. And there’s a lot of value to getting good at something.
I think there’s one thing that should be said, that in modern culture for young people, a lot of them will see YouTube and TikTok and Instagram, and they kinda wanna be famous. They wanna get the clicks and the views and so on, and that’s the thing they chase and optimize. I think the thing that you’re leaving unstated perhaps is that you spent many years pursuing the mastery of a craft. And there’s a lot of value to getting good at something.
Rick Beato
Absolutely.
Absolutely.
Lex Fridman
Offline. You can actually reveal your journey online, but the thing you’re chasing is not fame. It’s getting good at s- something. And I think actually what happens is even if the thing you get good at… is not the thing that you become famous for, if that’s the thing that ends up happening. It’s still, like, getting good at one thing, kind of somehow relates to getting good at another thing. Somehow they’ll lead you to get better at getting better at the next thing, at the next thing, and the next thing. But if you’re just chasing fame and trying to figure out, “How do I do the viral thing?” or so on, it just seems to… You might actually get there, but it’ll be unfulfilling and not long-lasting.
Offline. You can actually reveal your journey online, but the thing you’re chasing is not fame. It’s getting good at s- something. And I think actually what happens is even if the thing you get good at… is not the thing that you become famous for, if that’s the thing that ends up happening. It’s still, like, getting good at one thing, kind of somehow relates to getting good at another thing. Somehow they’ll lead you to get better at getting better at the next thing, at the next thing, and the next thing. But if you’re just chasing fame and trying to figure out, “How do I do the viral thing?” or so on, it just seems to… You might actually get there, but it’ll be unfulfilling and not long-lasting.
Rick Beato
My theory of my channel has always been, make videos on things I’m interested in. And at first, I thought, “Oh, nobody’s going to watch an old white-haired guy on YouTube.” That was kind of my thing. Well, that was not correct. And then it’s like, “Well, just make videos on stuff I’m interested in.” It just so happens that other people are interested in the same things I’m interested in. And keep learning. And when I produced bands, I never let them take my picture, ever. I never let them record me in the studio. There’s virtually no pictures of any band I ever produced. So from 1999 to 2015 when that Dylan video came out, no one took my picture. There were no pictures of me on the internet.
My theory of my channel has always been, make videos on things I’m interested in. And at first, I thought, “Oh, nobody’s going to watch an old white-haired guy on YouTube.” That was kind of my thing. Well, that was not correct. And then it’s like, “Well, just make videos on stuff I’m interested in.” It just so happens that other people are interested in the same things I’m interested in. And keep learning. And when I produced bands, I never let them take my picture, ever. I never let them record me in the studio. There’s virtually no pictures of any band I ever produced. So from 1999 to 2015 when that Dylan video came out, no one took my picture. There were no pictures of me on the internet.
Lex Fridman
You’re fully behind the camera kind of guy-
You’re fully behind the camera kind of guy-
Rick Beato
Yes.
Yes.
Lex Fridman
… meaning, like, no…
… meaning, like, no…
Rick Beato
No. No pictures. No, no pictures with people. “Hey, can we take a picture?” I said, “No. No pictures with people.”
No. No pictures. No, no pictures with people. “Hey, can we take a picture?” I said, “No. No pictures with people.”
Lex Fridman
And now you’re like… you’re the talent. You’re the face. No, I mean, but then again, the thing you’re leaving unstated there is like you spent a lot of years teaching music. Like, really exploring music. Trying a music career of like, trying to create, trying to produce, trying to be a musician, and all these… Not just trying. Like, being a, getting extremely good at it. I just, I think in modern culture there’s a sense you want to skip that part. “I wanna be famous. I wanna…” You know this. And that is a thing that’s not going to be in most cases effective as a primary thing to chase.
And now you’re like… you’re the talent. You’re the face. No, I mean, but then again, the thing you’re leaving unstated there is like you spent a lot of years teaching music. Like, really exploring music. Trying a music career of like, trying to create, trying to produce, trying to be a musician, and all these… Not just trying. Like, being a, getting extremely good at it. I just, I think in modern culture there’s a sense you want to skip that part. “I wanna be famous. I wanna…” You know this. And that is a thing that’s not going to be in most cases effective as a primary thing to chase.
Rick Beato
So I have an undergrad in classical bass. I have a master’s from New England Conservatory in jazz guitar. Then I taught college for… I taught jazz studies for five years- … from ’87- … to ’92. Then I got a publishing deal, my first publishing deal, in 1992- … with PolyGram Publishing. And then I became a producer when I was 37, having no idea how to engineer, I taught myself engineering. And then YouTube. I taught myself how to edit videos.
So I have an undergrad in classical bass. I have a master’s from New England Conservatory in jazz guitar. Then I taught college for… I taught jazz studies for five years- … from ’87- … to ’92. Then I got a publishing deal, my first publishing deal, in 1992- … with PolyGram Publishing. And then I became a producer when I was 37, having no idea how to engineer, I taught myself engineering. And then YouTube. I taught myself how to edit videos.
Lex Fridman
And then you taught yourself how to interview.
And then you taught yourself how to interview.
Rick Beato
And I taught myself how to interview. I’d never done an interview before. I never was like, “An interviewer? What?”
And I taught myself how to interview. I’d never done an interview before. I never was like, “An interviewer? What?”
Lex Fridman
You haven’t just done that. You’ve taught yourself not how to do YouTube, but YouTube Shorts. Different-
You haven’t just done that. You’ve taught yourself not how to do YouTube, but YouTube Shorts. Different-
Rick Beato
Totally different thing
Totally different thing
Lex Fridman
… totally different thing.
… totally different thing.
Rick Beato
Totally different skill.
Totally different skill.
Lex Fridman
And then not just YouTube, but like, how to be like a… there’s a… ’cause you’re both a YouTuber and like a musician who posts stuff on YouTube. YouTuber means like you’re thinking about stuff like thumbnails and…
And then not just YouTube, but like, how to be like a… there’s a… ’cause you’re both a YouTuber and like a musician who posts stuff on YouTube. YouTuber means like you’re thinking about stuff like thumbnails and…
Rick Beato
Which I make my own thumbnails. I’ve always made my own thumbnails.
Which I make my own thumbnails. I’ve always made my own thumbnails.
Lex Fridman
By the way, before I forget, I think I speak for the entirety of the internet thanking you for how you introduce your videos and how you close them. ‘Cause you, this is a big part of YouTube, where people have a 30-minute introduction to a five-minute video. You just go straight in. That’s really wonderful. That’s, I mean, on all fronts. I mean, I suppose that has to do with the production skills that you have, of understanding, cutting, cutting the fluff.
By the way, before I forget, I think I speak for the entirety of the internet thanking you for how you introduce your videos and how you close them. ‘Cause you, this is a big part of YouTube, where people have a 30-minute introduction to a five-minute video. You just go straight in. That’s really wonderful. That’s, I mean, on all fronts. I mean, I suppose that has to do with the production skills that you have, of understanding, cutting, cutting the fluff.
Rick Beato
To make a song.
To make a song.
Lex Fridman
Yep. Yeah, cutting, cutting the fluff, cutting the bullshit. I’ll just get straight to the core of the thing. I’ve heard you talk about maintaining friendships for a long time. You said, “Never waste a friendship.” Can you elaborate on that?
Yep. Yeah, cutting, cutting the fluff, cutting the bullshit. I’ll just get straight to the core of the thing. I’ve heard you talk about maintaining friendships for a long time. You said, “Never waste a friendship.” Can you elaborate on that?
Rick Beato
Yeah. That’s one of my things is that I really value the time I’ve spent with people—friendships and keeping in touch with people. I talk to each one of my siblings multiple times a week. I talk to my sisters probably every night, my two sisters. I have friends from college, I got friends from growing up, I have friends from, you know, both colleges I went to. I have friends from all different eras in my life that I keep in touch with and visit whenever I can, and…
Yeah. That’s one of my things is that I really value the time I’ve spent with people—friendships and keeping in touch with people. I talk to each one of my siblings multiple times a week. I talk to my sisters probably every night, my two sisters. I have friends from college, I got friends from growing up, I have friends from, you know, both colleges I went to. I have friends from all different eras in my life that I keep in touch with and visit whenever I can, and…
Lex Fridman
And you must have met some incredible humans, and incredibly weird, and interesting humans throughout your life. So it’s worth it—the effort to connect and reconnect.
And you must have met some incredible humans, and incredibly weird, and interesting humans throughout your life. So it’s worth it—the effort to connect and reconnect.
Rick Beato
I mean, it’s pretty much everything in life. Nothing means anything more than the friendships that you make in your family.
I mean, it’s pretty much everything in life. Nothing means anything more than the friendships that you make in your family.
Lex Fridman
Yeah, what’s the point of this whole thing, right?
Yeah, what’s the point of this whole thing, right?
Rick Beato
That’s right.
That’s right.
Lex Fridman
What’s the role of music in the human experience?
What’s the role of music in the human experience?
Rick Beato
Well, hopefully to enlighten people and to create the soundtrack of their life.
Well, hopefully to enlighten people and to create the soundtrack of their life.
Lex Fridman
It is, right? Music does something. I’ll get… sometimes when I’m alone I’ll listen to a song, and there’s nothing quite like a song that makes me truly feel, like feel alive. And whatever that is: sadness, or hope, or excitement. Or when I’m working out, listening to Rage Against the Machine—like protest. Or as I was listening to Metallica, I was re-listening to the set that they played in Moscow, just hyped. Like truly hyped. I was like pacing listening to it. And there’s nothing like that.
It is, right? Music does something. I’ll get… sometimes when I’m alone I’ll listen to a song, and there’s nothing quite like a song that makes me truly feel, like feel alive. And whatever that is: sadness, or hope, or excitement. Or when I’m working out, listening to Rage Against the Machine—like protest. Or as I was listening to Metallica, I was re-listening to the set that they played in Moscow, just hyped. Like truly hyped. I was like pacing listening to it. And there’s nothing like that.
Rick Beato
I’ve never found anything.
I’ve never found anything.
Lex Fridman
And I don’t know what that is in the human psyche that’s that, but I’m so glad we found it. We humans created instruments that can vibrate strings and together create harmonies and melodies, and ones that reverberate through generations and they carry that.
And I don’t know what that is in the human psyche that’s that, but I’m so glad we found it. We humans created instruments that can vibrate strings and together create harmonies and melodies, and ones that reverberate through generations and they carry that.
Rick Beato
It’s one of the greatest things that humans ever did, creating music.
It’s one of the greatest things that humans ever did, creating music.
Lex Fridman
And all of that led up to you, some guy being listened to by millions of people on the internet. This is all a simulation, Rick. And I’ve been a fan of yours for a long time, like I told you. This is crazy to meet you.
And all of that led up to you, some guy being listened to by millions of people on the internet. This is all a simulation, Rick. And I’ve been a fan of yours for a long time, like I told you. This is crazy to meet you.
Rick Beato
Same, Lex.
Same, Lex.
Lex Fridman
Thank you for everything you do for the world, for celebrating music. For helping us discover and rediscover some of the incredible musicians and songs that have been created over the decades, over the centuries. Thank you for being who you are and thank you for talking to me.
Thank you for everything you do for the world, for celebrating music. For helping us discover and rediscover some of the incredible musicians and songs that have been created over the decades, over the centuries. Thank you for being who you are and thank you for talking to me.
Rick Beato
Thanks, I appreciate it.
Thanks, I appreciate it.
Lex Fridman
Thanks for listening to this conversation with Rick Beato. To support this podcast, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from Friedrich Nietzsche, as I often do. “Without music, life would be a mistake.” Thank you for listening, and I hope to see you next time.
Thanks for listening to this conversation with Rick Beato. To support this podcast, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on. And now, let me leave you with some words from Friedrich Nietzsche, as I often do. “Without music, life would be a mistake.” Thank you for listening, and I hope to see you next time.
#491 – OpenClaw: The Viral AI Agent that Broke the Internet – Peter Steinberger
Podcast: Play in new window | Download
Subscribe: Spotify | TuneIn | RSS
Peter Steinberger is the creator of OpenClaw, an open-source AI agent framework that’s the fastest-growing project in GitHub history.
Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep491-sc
See below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.
Transcript:
https://lexfridman.com/peter-steinberger-transcript
CONTACT LEX:
Feedback – give feedback to Lex: https://lexfridman.com/survey
AMA – submit questions, videos or call-in: https://lexfridman.com/ama
Hiring – join our team: https://lexfridman.com/hiring
Other – other ways to get in touch: https://lexfridman.com/contact
EPISODE LINKS:
Peter’s X: https://x.com/steipete
Peter’s GitHub: https://github.com/steipete
Peter’s Website: https://steipete.com
Peter’s LinkedIn: https://www.linkedin.com/in/steipete
OpenClaw Website: https://openclaw.ai
OpenClaw GitHub: https://github.com/openclaw/openclaw
OpenClaw Discord: https://discord.gg/openclaw
SPONSORS:
To support this podcast, check out our sponsors & get discounts:
Perplexity: AI-powered answer engine.
Go to https://perplexity.ai/
Quo: Phone system (calls, texts, contacts) for businesses.
Go to https://quo.com/lex
CodeRabbit: AI-powered code reviews.
Go to https://coderabbit.ai/lex
Fin: AI agent for customer service.
Go to https://fin.ai/lex
Blitzy: AI agent for large enterprise codebases.
Go to https://blitzy.com/lex
Shopify: Sell stuff online.
Go to https://shopify.com/lex
LMNT: Zero-sugar electrolyte drink mix.
Go to https://drinkLMNT.com/lex
OUTLINE:
(00:00) – Introduction
(03:51) – Sponsors, Comments, and Reflections
(15:29) – OpenClaw origin story
(18:48) – Mind-blowing moment
(28:15) – Why OpenClaw went viral
(32:12) – Self-modifying AI agent
(36:57) – Name-change drama
(54:07) – Moltbook saga
(1:02:26) – OpenClaw security concerns
(1:11:07) – How to code with AI agents
(1:42:02) – Programming setup
(1:48:45) – GPT Codex 5.3 vs Claude Opus 4.6
(1:57:52) – Best AI agent for programming
(2:19:52) – Life story and career advice
(2:23:49) – Money and happiness
(2:27:41) – Acquisition offers from OpenAI and Meta
(2:44:51) – How OpenClaw works
(2:56:09) – AI slop
(3:02:13) – AI agents will replace 80% of apps
(3:10:50) – Will AI replace programmers?
(3:22:50) – Future of OpenClaw community
Transcript for OpenClaw: The Viral AI Agent that Broke the Internet – Peter Steinberger | Lex Fridman Podcast #491
This is a transcript of Lex Fridman Podcast #491 with Peter Steinberger.
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
- Go back to this episode’s main page
- Watch the full YouTube version of the podcast
Table of Contents
Here are the loose “chapters” in the conversation.
Click link to jump approximately to that part in the transcript:
Click link to jump approximately to that part in the transcript:
- 0:00 – Episode highlight
- 1:30 – Introduction
- 5:36 – OpenClaw origin story
- 8:55 – Mind-blowing moment
- 18:22 – Why OpenClaw went viral
- 22:19 – Self-modifying AI agent
- 27:04 – Name-change drama
- 44:15 – Moltbook saga
- 52:34 – OpenClaw security concerns
- 1:01:14 – How to code with AI agents
- 1:32:09 – Programming setup
- 1:38:52 – GPT Codex 5.3 vs Claude Opus 4.6
- 1:47:59 – Best AI agent for programming
- 2:09:59 – Life story and career advice
- 2:13:56 – Money and happiness
- 2:17:49 – Acquisition offers from OpenAI and Meta
- 2:34:58 – How OpenClaw works
- 2:46:17 – AI slop
- 2:52:20 – AI agents will replace 80% of apps
- 3:00:57 – Will AI replace programmers?
- 3:12:57 – Future of OpenClaw community
Episode highlight
Peter Steinberger
I watched my agent happily click the “I’m not a robot” button. I made the agent very aware. Like, it knows what his source code is. It understands th- how it sits and runs in its own harness. It knows where documentation is. It knows which model it runs. It understands its own system that made it very easy for an agent to… Oh, you don’t like anything? You just prompted it to existence, and then the agent would just modify its own software. People talk about self-modifying software, I just built it. I actually think wipe coding is a slur.
I watched my agent happily click the “I’m not a robot” button. I made the agent very aware. Like, it knows what his source code is. It understands th- how it sits and runs in its own harness. It knows where documentation is. It knows which model it runs. It understands its own system that made it very easy for an agent to… Oh, you don’t like anything? You just prompted it to existence, and then the agent would just modify its own software. People talk about self-modifying software, I just built it. I actually think wipe coding is a slur.
Lex Fridman
You prefer agentic engineering?
You prefer agentic engineering?
Peter Steinberger
Yeah, I always tell people I’d- I do agentic engineering, and then maybe after 3:00 AM, I switch to wipe coding, and then I have regrets on the next day.
Yeah, I always tell people I’d- I do agentic engineering, and then maybe after 3:00 AM, I switch to wipe coding, and then I have regrets on the next day.
Lex Fridman
What a walk of shame.
What a walk of shame.
Peter Steinberger
Yeah, you just have to clean up and, like, fix your sh- shit.
Yeah, you just have to clean up and, like, fix your sh- shit.
Lex Fridman
We’ve all been there.
We’ve all been there.
Peter Steinberger
I used to write really long prompts. And by writing, I mean, I don’t write, I- I- I talk, you know? These- these hands are, like, too- too precious for writing now. I just- I just use bespoke prompts to build my software.
I used to write really long prompts. And by writing, I mean, I don’t write, I- I- I talk, you know? These- these hands are, like, too- too precious for writing now. I just- I just use bespoke prompts to build my software.
Lex Fridman
So, you, for real, with all those terminals, are using voice?
So, you, for real, with all those terminals, are using voice?
Peter Steinberger
Yeah. I used to do it very extensively, to the point where there was a period where I lost my voice.
Yeah. I used to do it very extensively, to the point where there was a period where I lost my voice.
Lex Fridman
I mean, I have to ask you, just curious. I- I know you’ve probably gotten huge offers from major companies. Can you speak to who you’re considering working with?
I mean, I have to ask you, just curious. I- I know you’ve probably gotten huge offers from major companies. Can you speak to who you’re considering working with?
Peter Steinberger
Yeah.
Yeah.
Introduction
Lex Fridman
The following is a conversation with Peter Steinberger, creator of OpenClaw, formerly known as MoldBot, ClawedBot, Clawdus, Claude, spelled with a W as in lobster claw. Not to be confused with Claud, the AI model from Anthropic, spelled with a U. In fact, this confusion is the reason Anthropic kindly asked Peter to change the name to OpenClaw. So, what is OpenClaw? It’s an open-source AI agent that has taken over the tech world in a matter of days, exploding in popularity, reaching over 180,000 stars on GitHub, and spawning the social network mold book, where AI agents post manifestos and debate consciousness, creating a mix of excitement and fear in the general public.
The following is a conversation with Peter Steinberger, creator of OpenClaw, formerly known as MoldBot, ClawedBot, Clawdus, Claude, spelled with a W as in lobster claw. Not to be confused with Claud, the AI model from Anthropic, spelled with a U. In fact, this confusion is the reason Anthropic kindly asked Peter to change the name to OpenClaw. So, what is OpenClaw? It’s an open-source AI agent that has taken over the tech world in a matter of days, exploding in popularity, reaching over 180,000 stars on GitHub, and spawning the social network mold book, where AI agents post manifestos and debate consciousness, creating a mix of excitement and fear in the general public.
Lex Fridman
And a kind of AI psychosis, a mix of clickbait fearmongering and genuine, fully justifiable concern about the role of AI in our digital, interconnected human world. OpenClaw, as its tagline states, is the AI that actually does things. It’s an autonomous AI assistant that lives in your computer, has access to all of your stuff, if you let it, talks to you through Telegram, WhatsApp, Signal, iMessage, and whatever else messaging client. Uses whatever AI model you like, including Claude Opus 4.6 and GPT 5.3 Codex, all to do stuff for you. Many people are calling this one of the biggest moments in the recent history of AI, since the launch of ChatGPT in November 2022.
And a kind of AI psychosis, a mix of clickbait fearmongering and genuine, fully justifiable concern about the role of AI in our digital, interconnected human world. OpenClaw, as its tagline states, is the AI that actually does things. It’s an autonomous AI assistant that lives in your computer, has access to all of your stuff, if you let it, talks to you through Telegram, WhatsApp, Signal, iMessage, and whatever else messaging client. Uses whatever AI model you like, including Claude Opus 4.6 and GPT 5.3 Codex, all to do stuff for you. Many people are calling this one of the biggest moments in the recent history of AI, since the launch of ChatGPT in November 2022.
Lex Fridman
The ingredients for this kind of AI agent were all there, but putting it all together in a system that definitively takes a step forward over the line from language to agency, from ideas to actions, in a way that created a useful assistant that feels like one who gets you and learns from you, in an open source, community-driven way, is the reason OpenClaw took the internet by storm. Its power, in large part, comes from the fact that you can give it access to all of your stuff and give it permission to do anything with that stuff in order to be useful to you. This is very powerful, but it is also dangerous. OpenClaw represents freedom, but with freedom comes responsibility.
The ingredients for this kind of AI agent were all there, but putting it all together in a system that definitively takes a step forward over the line from language to agency, from ideas to actions, in a way that created a useful assistant that feels like one who gets you and learns from you, in an open source, community-driven way, is the reason OpenClaw took the internet by storm. Its power, in large part, comes from the fact that you can give it access to all of your stuff and give it permission to do anything with that stuff in order to be useful to you. This is very powerful, but it is also dangerous. OpenClaw represents freedom, but with freedom comes responsibility.
Lex Fridman
With it, you can own and have control over your data, but precisely because you have this control, you also have the responsibility to protect it from cybersecurity threats of various kinds. There are great ways to protect yourself, but the threats and vulnerabilities are out there. Again, a powerful AI agent with system-level access is a security minefield, but it also represents the future. Because when done well and securely, it can be extremely useful to each of us humans as a personal assistant. We discuss all of this with Peter, and also discuss his big-picture programming and entrepreneurship life story, which I think is truly inspiring. He spent 13 years building PSPDF Kit, which is a software used on a billion devices.
With it, you can own and have control over your data, but precisely because you have this control, you also have the responsibility to protect it from cybersecurity threats of various kinds. There are great ways to protect yourself, but the threats and vulnerabilities are out there. Again, a powerful AI agent with system-level access is a security minefield, but it also represents the future. Because when done well and securely, it can be extremely useful to each of us humans as a personal assistant. We discuss all of this with Peter, and also discuss his big-picture programming and entrepreneurship life story, which I think is truly inspiring. He spent 13 years building PSPDF Kit, which is a software used on a billion devices.
Lex Fridman
He sold it, and for a brief time, fell out of love with programming, vanished for three years, and then came back, rediscovered his love for programming, and built, in a very short time, an open source AI agent that took the internet by storm. He is, in many ways, the symbol of the AI revolution happening in the programming world. There was the ChatGPT moment in 2022, the DeepSeek moment in 2025, and now, in ’26, we’re living through the OpenClaw moment, the age of the lobster. The start of the agentic AI revolution. What a time to be alive. This is a Lex Fridman podcast. To support it, please check out our sponsors in the description, or you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Peter Steinberger.
He sold it, and for a brief time, fell out of love with programming, vanished for three years, and then came back, rediscovered his love for programming, and built, in a very short time, an open source AI agent that took the internet by storm. He is, in many ways, the symbol of the AI revolution happening in the programming world. There was the ChatGPT moment in 2022, the DeepSeek moment in 2025, and now, in ’26, we’re living through the OpenClaw moment, the age of the lobster. The start of the agentic AI revolution. What a time to be alive. This is a Lex Fridman podcast. To support it, please check out our sponsors in the description, or you can also find links to contact me, ask questions, give feedback, and so on. And now, dear friends, here’s Peter Steinberger.
OpenClaw origin story
Lex Fridman
The one and only, the Clawed Father. Actually, Benjamin predicted it in his tweet. “The following is a conversation with Claude, a respected crustacean.” It’s a hilarious-looking picture of a lobster in a suit, so I think the prophecy has been fulfilled. Let’s go to this moment when you built a prototype in one hour, that was the early version of OpenClaw. I think this story’s really inspiring to a lot of people because this prototype led to something that just took the internet by storm…. and became the fastest-growing repository in GitHub history, with now over 175,000 stars. So, what was the story of the one-hour prototype?
The one and only, the Clawed Father. Actually, Benjamin predicted it in his tweet. “The following is a conversation with Claude, a respected crustacean.” It’s a hilarious-looking picture of a lobster in a suit, so I think the prophecy has been fulfilled. Let’s go to this moment when you built a prototype in one hour, that was the early version of OpenClaw. I think this story’s really inspiring to a lot of people because this prototype led to something that just took the internet by storm…. and became the fastest-growing repository in GitHub history, with now over 175,000 stars. So, what was the story of the one-hour prototype?
Peter Steinberger
You know, I wanted that since April.
You know, I wanted that since April.
Lex Fridman
A personal assistant. AI personal assistant.
A personal assistant. AI personal assistant.
Peter Steinberger
Yeah. And I, I played around with some other things, like even stuff that gets all my WhatsApp, and I could just run queries on it. That was back when we had GPT-4.1, with the one million context window. And I, I pulled in all the data and then just asked him questions like, “What makes this friendship meaningful?”
Yeah. And I, I played around with some other things, like even stuff that gets all my WhatsApp, and I could just run queries on it. That was back when we had GPT-4.1, with the one million context window. And I, I pulled in all the data and then just asked him questions like, “What makes this friendship meaningful?”
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And I got some, some really profound results. Like, I sent it to my friends and they got, like, teary eyes.
And I got some, some really profound results. Like, I sent it to my friends and they got, like, teary eyes.
Lex Fridman
So, there’s something there.
So, there’s something there.
Peter Steinberger
Yeah. But then I… I thought all the labs will, will, will work on that. So I, I moved on to other things, and that was still very much in my early days of experimenting and pl- playing. You know, you have to… That’s how you learn. You just like, you do stuff and you play. And time flew by and it was November. I wanted to make sure that the thing I started is actually happening. I was annoyed that it didn’t exist, so I just prompted it into existence.
Yeah. But then I… I thought all the labs will, will, will work on that. So I, I moved on to other things, and that was still very much in my early days of experimenting and pl- playing. You know, you have to… That’s how you learn. You just like, you do stuff and you play. And time flew by and it was November. I wanted to make sure that the thing I started is actually happening. I was annoyed that it didn’t exist, so I just prompted it into existence.
Lex Fridman
I mean, that’s the beginning of the hero’s journey of the entrepreneur, right? And you’ve even with your original story with PS PDF kit, it’s like, “Why does this not exist? Let me build it.” And again, here’s diff- a whole different realm, but similar maybe spirit.
I mean, that’s the beginning of the hero’s journey of the entrepreneur, right? And you’ve even with your original story with PS PDF kit, it’s like, “Why does this not exist? Let me build it.” And again, here’s diff- a whole different realm, but similar maybe spirit.
Peter Steinberger
Yeah, so I had this problem. I tried to show PDF on an iPad, which should not be hard.
Yeah, so I had this problem. I tried to show PDF on an iPad, which should not be hard.
Lex Fridman
This is like 15 years ago, something like that.
This is like 15 years ago, something like that.
Peter Steinberger
Yeah. Like the most, the most random thing ever. And suddenly, I had this problem and I, I wanted to help a friend. And there was, there was… Well, not like nothing existed, but it was just not good. And like… Like I tried it and it was like very, “Nah.” Like, “Hmm, I can do this better.”
Yeah. Like the most, the most random thing ever. And suddenly, I had this problem and I, I wanted to help a friend. And there was, there was… Well, not like nothing existed, but it was just not good. And like… Like I tried it and it was like very, “Nah.” Like, “Hmm, I can do this better.”
Lex Fridman
By the way, for people who don’t know, this led to the development of PS PDF kit that’s used on a billion devices. So, the… It turns out that it’s pretty useful to be able to open a PDF.
By the way, for people who don’t know, this led to the development of PS PDF kit that’s used on a billion devices. So, the… It turns out that it’s pretty useful to be able to open a PDF.
Peter Steinberger
You could also make the joke that I’m really bad at naming.
You could also make the joke that I’m really bad at naming.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Like, name number five on the current project. And even PS PDF doesn’t really roll from the tongue.
Like, name number five on the current project. And even PS PDF doesn’t really roll from the tongue.
Lex Fridman
Anyway, so you said “Screw it. Why don’t I do it?” So what was the… What was the prototype? What was the thing that you… What was the magical thing that you built in a short amount of time that you were like, “This might actually work as an agent,” where I talk to it and it does things?
Anyway, so you said “Screw it. Why don’t I do it?” So what was the… What was the prototype? What was the thing that you… What was the magical thing that you built in a short amount of time that you were like, “This might actually work as an agent,” where I talk to it and it does things?
Mind-blowing moment
Peter Steinberger
There was… Like, one of my projects before already did something where I could bring my terminals onto the web and then I could, like, interact with them, but there also would be terminals on my Mac.
There was… Like, one of my projects before already did something where I could bring my terminals onto the web and then I could, like, interact with them, but there also would be terminals on my Mac.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Viptunnel, which was like a, a weekend hack project that was still very early. And it was cloud code times. You know, you got a dopamine hit when you got something right. And now I get, like, mad when you get something wrong.
Viptunnel, which was like a, a weekend hack project that was still very early. And it was cloud code times. You know, you got a dopamine hit when you got something right. And now I get, like, mad when you get something wrong.
Lex Fridman
And you had a really great -– not to take a tangent -– but a great blog post describing that you converted Viptunnel. You vibe-coded Viptunnel from TypeScript into Zig of all programming languages with a single prompt. One prompt, one shot. Convert the entire code base into Zig.
And you had a really great -– not to take a tangent -– but a great blog post describing that you converted Viptunnel. You vibe-coded Viptunnel from TypeScript into Zig of all programming languages with a single prompt. One prompt, one shot. Convert the entire code base into Zig.
Peter Steinberger
Yeah. There was this one thing where part of the architecture was… Took too much memory. Every terminal used like a node. And I wanted to change it to Rust and… I mean, I can do it. I can, I can manually figure it all out, but all my automated attempts failed miserably. And then I revisited about four or five months later. And I’m like, “Okay, now let’s use something even more experimental.” And I, and I just typed, “Convert this and this part to Sig,” and then let Codex run off. And it basically got it right. There was one little detail that I had to, like, modify afterwards, but it just ran for overnight or like six hours and just did its thing. And it’s like… It’s just mind-blowing.
Yeah. There was this one thing where part of the architecture was… Took too much memory. Every terminal used like a node. And I wanted to change it to Rust and… I mean, I can do it. I can, I can manually figure it all out, but all my automated attempts failed miserably. And then I revisited about four or five months later. And I’m like, “Okay, now let’s use something even more experimental.” And I, and I just typed, “Convert this and this part to Sig,” and then let Codex run off. And it basically got it right. There was one little detail that I had to, like, modify afterwards, but it just ran for overnight or like six hours and just did its thing. And it’s like… It’s just mind-blowing.
Lex Fridman
So that’s on the LLM programming side, refactoring. But uh, back to the actual story of the of the prototype. So how did Viptunnel connect to the first prototype where your, like, agents can actually work?
So that’s on the LLM programming side, refactoring. But uh, back to the actual story of the of the prototype. So how did Viptunnel connect to the first prototype where your, like, agents can actually work?
Peter Steinberger
Well, that was still very limited. You know, like I had this one experiment with WhatsApp, then I had this experiment, and both felt like not the right answer. And then my search bar was literally just hooking up WhatsApp to cloud code. One shot. The CLI message comes in. I call the CLI with -p. It does its magic, I get the string back and I send it back to WhatsApp. And I, I built this in one hour. And I felt… Already felt really cool. It’s like, “Oh, I could… I can, like, talk to my computer,” right? This… That, that was, that was cool. But I, I wanted images, ’cause I alw- I often use images when I prompt. I think it’s such a, such an efficient way to give the agent more context.
Well, that was still very limited. You know, like I had this one experiment with WhatsApp, then I had this experiment, and both felt like not the right answer. And then my search bar was literally just hooking up WhatsApp to cloud code. One shot. The CLI message comes in. I call the CLI with -p. It does its magic, I get the string back and I send it back to WhatsApp. And I, I built this in one hour. And I felt… Already felt really cool. It’s like, “Oh, I could… I can, like, talk to my computer,” right? This… That, that was, that was cool. But I, I wanted images, ’cause I alw- I often use images when I prompt. I think it’s such a, such an efficient way to give the agent more context.
Peter Steinberger
And they are really good at figuring out what I mean, e- even if it’s like a, a weird cropped-up screenshot. So I used it a lot and I wanted to do that in WhatsApp as well. Also, like, you know, just you run around, you see like a poster of an event, you just make a screenshot and like figure out if I have time there, if this is good, if my friends are maybe up for that. Just like images seemed im- important. So I, I worked a few… It took me a few more hours to actually get that right. And then it was just…… I, I used it a lot. And funny enough, that was just before I went on a trip to Marrakesh with my friends for a birthday trip. And there it was even better because internet was a little shaky but WhatsApp just works, you know?
And they are really good at figuring out what I mean, e- even if it’s like a, a weird cropped-up screenshot. So I used it a lot and I wanted to do that in WhatsApp as well. Also, like, you know, just you run around, you see like a poster of an event, you just make a screenshot and like figure out if I have time there, if this is good, if my friends are maybe up for that. Just like images seemed im- important. So I, I worked a few… It took me a few more hours to actually get that right. And then it was just…… I, I used it a lot. And funny enough, that was just before I went on a trip to Marrakesh with my friends for a birthday trip. And there it was even better because internet was a little shaky but WhatsApp just works, you know?
Peter Steinberger
It’s like doesn’t matter, you have, like, edge, it still works. WhatsApp is just… It’s just made really well. So I ended up using it a lot. Translate this for me, explain this, find me places. Like, you just having a clanker doing, having Google for you, that was… Basically there was still nothing built but it still could do so much.
It’s like doesn’t matter, you have, like, edge, it still works. WhatsApp is just… It’s just made really well. So I ended up using it a lot. Translate this for me, explain this, find me places. Like, you just having a clanker doing, having Google for you, that was… Basically there was still nothing built but it still could do so much.
Lex Fridman
So, if we talk about the full journey that’s happening there with the agent, you’re just sending on this very thin line WhatsApp message via CLI, it’s going to a cloud code and cloud code is doing all kinds of heavy work and coming back to you with a thin message.
So, if we talk about the full journey that’s happening there with the agent, you’re just sending on this very thin line WhatsApp message via CLI, it’s going to a cloud code and cloud code is doing all kinds of heavy work and coming back to you with a thin message.
Peter Steinberger
Yeah. It was slow because every time I boot up the CLI, but it… It was really cool already. And it could just use all the things that I already had built. I had built like a whole bunch of CLI stuff over the month so it, it felt really powerful.
Yeah. It was slow because every time I boot up the CLI, but it… It was really cool already. And it could just use all the things that I already had built. I had built like a whole bunch of CLI stuff over the month so it, it felt really powerful.
Lex Fridman
There is something magical about that experience that’s hard to put into words. Being able to use a chat client to talk to an agent, versus, like, sitting behind a computer and like, I don’t know, using cursor or even using Cloud Code CLI in the terminal. It’s a different experience than being able to sit back and talk to it. I mean, it seems like a trivial step but, it- in some sense it’s a… It’s like a phase shift in the integration of AI into your life and how it feels, right?
There is something magical about that experience that’s hard to put into words. Being able to use a chat client to talk to an agent, versus, like, sitting behind a computer and like, I don’t know, using cursor or even using Cloud Code CLI in the terminal. It’s a different experience than being able to sit back and talk to it. I mean, it seems like a trivial step but, it- in some sense it’s a… It’s like a phase shift in the integration of AI into your life and how it feels, right?
Peter Steinberger
Yeah. Yeah. I, I read this tweet this morning where someone said, “Oh, there’s no magic in it. It’s just like, it does this and this and this and this and this and this.” And it almost feels like a hobby, just as cursor or perplexity. And I’m like, well, if that’s a hobby that’s kind of a compliment, you know? They’re like, they’re not doing too bad. Thank you I guess? Yes. I mean, isn’t, isn’t, isn’t magic often just like you take a lot of things that are already there but bring them together in new ways? Like, I don’t… There’s no… Yeah. Maybe there’s no magic in there but sometimes just rearranging things and, like, adding a few new ideas is all the magic that you need.
Yeah. Yeah. I, I read this tweet this morning where someone said, “Oh, there’s no magic in it. It’s just like, it does this and this and this and this and this and this.” And it almost feels like a hobby, just as cursor or perplexity. And I’m like, well, if that’s a hobby that’s kind of a compliment, you know? They’re like, they’re not doing too bad. Thank you I guess? Yes. I mean, isn’t, isn’t, isn’t magic often just like you take a lot of things that are already there but bring them together in new ways? Like, I don’t… There’s no… Yeah. Maybe there’s no magic in there but sometimes just rearranging things and, like, adding a few new ideas is all the magic that you need.
Lex Fridman
It’s really hard to convert into words what is, what is magic about a thing. If you look at the, the scrolling on an iPhone, why is that so pleasant? There’s a lot of elements about that interface that makes it incredibly pleasant, that is fundamental to the experience of using a smartphone, and it’s like, okay, all the components were there. Scrolling was there, everything was there.
It’s really hard to convert into words what is, what is magic about a thing. If you look at the, the scrolling on an iPhone, why is that so pleasant? There’s a lot of elements about that interface that makes it incredibly pleasant, that is fundamental to the experience of using a smartphone, and it’s like, okay, all the components were there. Scrolling was there, everything was there.
Peter Steinberger
Nobody did it-
Nobody did it-
Lex Fridman
Yep
Yep
Peter Steinberger
… and afterwards it felt so obvious.
… and afterwards it felt so obvious.
Lex Fridman
Yeah, so obvious.
Yeah, so obvious.
Peter Steinberger
Right? But still… You know the moment where it, it blew my mind was when, when I- I used it a lot and then at some point I just sent it a message and, and then a typing indicator appeared. And I’m like, wait, I didn’t build that, it only m- it only has image support, so what is it even doing? And then it would just reply.
Right? But still… You know the moment where it, it blew my mind was when, when I- I used it a lot and then at some point I just sent it a message and, and then a typing indicator appeared. And I’m like, wait, I didn’t build that, it only m- it only has image support, so what is it even doing? And then it would just reply.
Lex Fridman
What was the thing you sent it?
What was the thing you sent it?
Peter Steinberger
Oh, just a random question like, “Hey, what about this in this restaurant?” You know? Because we were just running around and checking out the city. So that’s why I, I didn’t, didn’t even think when I used it because sometimes when you’re in a hurry typing is annoying.
Oh, just a random question like, “Hey, what about this in this restaurant?” You know? Because we were just running around and checking out the city. So that’s why I, I didn’t, didn’t even think when I used it because sometimes when you’re in a hurry typing is annoying.
Lex Fridman
So, oh, you did an audio message?
So, oh, you did an audio message?
Peter Steinberger
Yeah. And it just, it just worked and I’m like…
Yeah. And it just, it just worked and I’m like…
Lex Fridman
And it’s not supposed to work because-
And it’s not supposed to work because-
Peter Steinberger
No
No
Lex Fridman
… you didn’t give it that-
… you didn’t give it that-
Peter Steinberger
No, literally
No, literally
Lex Fridman
… capability.
… capability.
Peter Steinberger
I literally went, “How the fuck did he do that?” And it was like, “Yeah, the mad lad did the following. He sent me a message but it only, only was a file and no file ending.” So I checked out the header of the file and it found that it was, like, opus so I used ffmpeg to convert it and then I wanted to use whisper but it didn’t had it installed. But then I found the OpenAI key and just used Curl to send the file to OpenAI to translate and here I am.
I literally went, “How the fuck did he do that?” And it was like, “Yeah, the mad lad did the following. He sent me a message but it only, only was a file and no file ending.” So I checked out the header of the file and it found that it was, like, opus so I used ffmpeg to convert it and then I wanted to use whisper but it didn’t had it installed. But then I found the OpenAI key and just used Curl to send the file to OpenAI to translate and here I am.
Peter Steinberger
Just looked at the message I’m like, “Oh wow.”
Just looked at the message I’m like, “Oh wow.”
Lex Fridman
You didn’t teach it any of those things and the agent just figured it out, did all those conversions, the translations. It figured out the API, it figured out which program to use, all those kinds of things. And you were just absent-mindedly just sent an audio message when it came back.
You didn’t teach it any of those things and the agent just figured it out, did all those conversions, the translations. It figured out the API, it figured out which program to use, all those kinds of things. And you were just absent-mindedly just sent an audio message when it came back.
Peter Steinberger
Yeah, like, so clever even because he would have gotten the whisper local path, he would have had to download a model. It would have been too slow. So like, there’s so much world knowledge in there, so much creative problem solving. A lot of it I think mapped from… If you get really good at coding that means you have to be really good at general purpose problem solving. So that’s a skill, right? And that just maps into other domains. So it had the problem of like, what is this file with no file ending? Let’s figure it out. And that’s when it kind of clicked for me. It’s like, I was like very impressed. And somebody sent a pull request for Discord support and I’m like, “This is a WhatsApp relay.
Yeah, like, so clever even because he would have gotten the whisper local path, he would have had to download a model. It would have been too slow. So like, there’s so much world knowledge in there, so much creative problem solving. A lot of it I think mapped from… If you get really good at coding that means you have to be really good at general purpose problem solving. So that’s a skill, right? And that just maps into other domains. So it had the problem of like, what is this file with no file ending? Let’s figure it out. And that’s when it kind of clicked for me. It’s like, I was like very impressed. And somebody sent a pull request for Discord support and I’m like, “This is a WhatsApp relay.
Peter Steinberger
That doesn’t, doesn’t fit at all.”
That doesn’t, doesn’t fit at all.”
Lex Fridman
At that time it was called WA Relay.
At that time it was called WA Relay.
Peter Steinberger
Yeah. And so I debated with me like, do I want that? Do I not want that? And then I thought, well maybe, maybe I do that because that could be a cool way to show people. Because I… So far I did it in WhatsApp as like groups you know but don’t really want to give my phone number to every internet stranger.
Yeah. And so I debated with me like, do I want that? Do I not want that? And then I thought, well maybe, maybe I do that because that could be a cool way to show people. Because I… So far I did it in WhatsApp as like groups you know but don’t really want to give my phone number to every internet stranger.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Journalists manage to do that anyhow now so that’s a different story. So I merged it-… from Shadow, who helped me a lot with the whole project. So, thank you. And, and I put my, my bot in there.
Journalists manage to do that anyhow now so that’s a different story. So I merged it-… from Shadow, who helped me a lot with the whole project. So, thank you. And, and I put my, my bot in there.
Why OpenClaw went viral
Lex Fridman
On Discord?
On Discord?
Peter Steinberger
Yeah. No security because I didn’t… I hadn’t built sandboxing in yet. I, I just prompted it to, like, only listen to me. And then some people came and tried to hack it, and I just… Or, like, just watched and I just kept working in the open, you know? Like, y- I used my agent to build my agent harness and to test, like, various stuff. And that’s very quickly when it clicked for people. So it’s almost like it needs to be experienced. And from that time on, that was January the 1st, I, I got my first real influencer being a fan and did videos, dachitze. Thank you. And, and from there on, I saw, I started gaining up speed. And at the same time, my, my sleep cycle went shorter and shorter because I, I felt the storm coming, and I just worked my ass off to get it to…
Yeah. No security because I didn’t… I hadn’t built sandboxing in yet. I, I just prompted it to, like, only listen to me. And then some people came and tried to hack it, and I just… Or, like, just watched and I just kept working in the open, you know? Like, y- I used my agent to build my agent harness and to test, like, various stuff. And that’s very quickly when it clicked for people. So it’s almost like it needs to be experienced. And from that time on, that was January the 1st, I, I got my first real influencer being a fan and did videos, dachitze. Thank you. And, and from there on, I saw, I started gaining up speed. And at the same time, my, my sleep cycle went shorter and shorter because I, I felt the storm coming, and I just worked my ass off to get it to…
Peter Steinberger
into a state where it’s kinda good.
into a state where it’s kinda good.
Lex Fridman
There’s a few components and we’ll talk about how it all works, but basically, you’re able to talk to it using WhatsApp, Telegram, Discord. So that’s a component that you have to get right.
There’s a few components and we’ll talk about how it all works, but basically, you’re able to talk to it using WhatsApp, Telegram, Discord. So that’s a component that you have to get right.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
And then you have to figure out the agentic loop, you have to have the gateway, you have the harness, you have all those components that make it all just work nicely.
And then you have to figure out the agentic loop, you have to have the gateway, you have the harness, you have all those components that make it all just work nicely.
Peter Steinberger
Yeah. It felt like Factorio times infinite.
Yeah. It felt like Factorio times infinite.
Lex Fridman
Right.
Right.
Peter Steinberger
I, I feel like I built my little- … my little playground. Like, I never had so much fun than building this project. You know? Like, you have like, “Oh,” I go like, level one agentic loop. What can I do there? How can I be smart at queuing messages? How can I make it more human-like? Oh, then I had this idea of… Because the loop always… The agent always replies something, but you don’t always want an agent to reply something in a group chat. So I gave him this no-reply token. So I gave him an option to shut up. So it, it feels more natural.
I, I feel like I built my little- … my little playground. Like, I never had so much fun than building this project. You know? Like, you have like, “Oh,” I go like, level one agentic loop. What can I do there? How can I be smart at queuing messages? How can I make it more human-like? Oh, then I had this idea of… Because the loop always… The agent always replies something, but you don’t always want an agent to reply something in a group chat. So I gave him this no-reply token. So I gave him an option to shut up. So it, it feels more natural.
Lex Fridman
That’s level two.
That’s level two.
Peter Steinberger
Y- uh, yeah, yeah. Yeah, on the- on the-
Y- uh, yeah, yeah. Yeah, on the- on the-
Lex Fridman
Factorio.
Factorio.
Peter Steinberger
On the agentic loop. And then I go to memory, right?
On the agentic loop. And then I go to memory, right?
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
You want him to, like, remember stuff. So maybe, maybe the end… The ultimate boss is continuous reinforcement learning, but I’m, I’m, like, at… I feel like I’m level two or three with Markdown files and the vector database. And then you, you can go to level community management, you can go to level website and marketing. There’s just so many hats that you have to have on. Not even talking about native apps. That’s just, like, infinite different levels and infinite level ups you can do.
You want him to, like, remember stuff. So maybe, maybe the end… The ultimate boss is continuous reinforcement learning, but I’m, I’m, like, at… I feel like I’m level two or three with Markdown files and the vector database. And then you, you can go to level community management, you can go to level website and marketing. There’s just so many hats that you have to have on. Not even talking about native apps. That’s just, like, infinite different levels and infinite level ups you can do.
Lex Fridman
So the whole time you’re having fun. We should say that for the most part, throughout this whole process, you’re a one-man team. There’s people helping, but you’re doing so much of the key core development.
So the whole time you’re having fun. We should say that for the most part, throughout this whole process, you’re a one-man team. There’s people helping, but you’re doing so much of the key core development.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
And having fun? You did, in January, 6,600 commits. Probably more.
And having fun? You did, in January, 6,600 commits. Probably more.
Peter Steinberger
I sometimes posted a meme. I’m limited by the technology of my time. I could do more if agents would be faster.
I sometimes posted a meme. I’m limited by the technology of my time. I could do more if agents would be faster.
Lex Fridman
But we should say you’re running multiple agents at the same time.
But we should say you’re running multiple agents at the same time.
Peter Steinberger
Yeah. Depending on how much I slept and how difficult of the tasks I work on, between four and 10.
Yeah. Depending on how much I slept and how difficult of the tasks I work on, between four and 10.
Lex Fridman
Four and 10 agents. Uh there’s so many possible directions, speaking of Factorio, that we can go here. But one big picture one is, why do you think your work, Open Claw, won? In this world, if you look at 2025, so many startups, so many companies were doing kind of agentic type stuff, or claiming to. And here, Open Claw comes in and destroys everybody. Like, why did you win?
Four and 10 agents. Uh there’s so many possible directions, speaking of Factorio, that we can go here. But one big picture one is, why do you think your work, Open Claw, won? In this world, if you look at 2025, so many startups, so many companies were doing kind of agentic type stuff, or claiming to. And here, Open Claw comes in and destroys everybody. Like, why did you win?
Peter Steinberger
Because they all take themselves too serious.
Because they all take themselves too serious.
Lex Fridman
Yeah.
Yeah.
Self-modifying AI agent
Peter Steinberger
Like, it’s hard to compete against someone who’s just there to have fun.
Like, it’s hard to compete against someone who’s just there to have fun.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I wanted it to be fun, I wanted it to be weird. And if you see, like, all the, all the lobster stuff online I think I, I managed weird. I… You know, for the longest time, the only, the only way to install it was git clone, pnpm build, pnpm gateway. Like, you clone it, you build it, you run it. And then the, the agent… I made the agent very aware. Like, it knows that it is… What its source code is. It understands th- how it sits and runs in its own harness. It knows where documentation is. It knows which model it runs. It knows if you turn on the voice or, or reasoning mode. Like, I, I wanted to be more human-like, so it understands its own system that made it very easy for an agent to… Oh, you don’t like anything?
I wanted it to be fun, I wanted it to be weird. And if you see, like, all the, all the lobster stuff online I think I, I managed weird. I… You know, for the longest time, the only, the only way to install it was git clone, pnpm build, pnpm gateway. Like, you clone it, you build it, you run it. And then the, the agent… I made the agent very aware. Like, it knows that it is… What its source code is. It understands th- how it sits and runs in its own harness. It knows where documentation is. It knows which model it runs. It knows if you turn on the voice or, or reasoning mode. Like, I, I wanted to be more human-like, so it understands its own system that made it very easy for an agent to… Oh, you don’t like anything?
Peter Steinberger
You just prompted it to existence, and then the agent would just modify its own software. You know, we have people talk about self-modifying software. I just built it and didn’t even… I didn’t even plan it so much. It just happened.
You just prompted it to existence, and then the agent would just modify its own software. You know, we have people talk about self-modifying software. I just built it and didn’t even… I didn’t even plan it so much. It just happened.
Lex Fridman
Can you actually speak to that? ‘Cause it’s just fascinating. So you have this piece of software that’s written in TypeScript-
Can you actually speak to that? ‘Cause it’s just fascinating. So you have this piece of software that’s written in TypeScript-
Peter Steinberger
Yeah
Yeah
Lex Fridman
… that’s able to, via the agentic loop, modify itself. I mean, what a moment to be alive in the history of humanity and the history of programming. Here’s the thing that’s used by a huge amount of people to do incredibly powerful things in their lives, and that very system can rewrite itself, can modify itself. Can you just, like, speak to the power of that? Like, isn’t that incredible? Like, when did you first close the loop on that?
… that’s able to, via the agentic loop, modify itself. I mean, what a moment to be alive in the history of humanity and the history of programming. Here’s the thing that’s used by a huge amount of people to do incredibly powerful things in their lives, and that very system can rewrite itself, can modify itself. Can you just, like, speak to the power of that? Like, isn’t that incredible? Like, when did you first close the loop on that?
Peter Steinberger
Oh, because that’s how I built it as well, you know? Most of it is built by Codex, but oftentimes I… When I debug it, I…… I use self-introspection so much. It’s like, “Hey, what tools do you see? Can you call the tool yourself?” Or like, “What error do you see? Read the source code. Figure out what’s the problem.” Like, I just found it an incredibly fun way to… That the agent, the very agent and software that you use is used to debug itself, so that it felt just natural that everybody does that. And that it led to so many, so many pull requests by people who never wrote software. I mean, it also did show that people never wrote software . So I call them prompt requests in the end.
Oh, because that’s how I built it as well, you know? Most of it is built by Codex, but oftentimes I… When I debug it, I…… I use self-introspection so much. It’s like, “Hey, what tools do you see? Can you call the tool yourself?” Or like, “What error do you see? Read the source code. Figure out what’s the problem.” Like, I just found it an incredibly fun way to… That the agent, the very agent and software that you use is used to debug itself, so that it felt just natural that everybody does that. And that it led to so many, so many pull requests by people who never wrote software. I mean, it also did show that people never wrote software . So I call them prompt requests in the end.
Peter Steinberger
But I don’t want to, like, pull that down because every time someone made the first pull request is a win for our society, you know? Like, it… Like, it doesn’t matter how, how shitty it is, y- you gotta start somewhere. So I know there’s, like, this whole big movement of people complain about open source and the quality of PRs, and a whole different level of problems. But on a different level, I found it… I found it very meaningful that, that I built something that people love to think of so much that they actually start to learn how open source works.
But I don’t want to, like, pull that down because every time someone made the first pull request is a win for our society, you know? Like, it… Like, it doesn’t matter how, how shitty it is, y- you gotta start somewhere. So I know there’s, like, this whole big movement of people complain about open source and the quality of PRs, and a whole different level of problems. But on a different level, I found it… I found it very meaningful that, that I built something that people love to think of so much that they actually start to learn how open source works.
Lex Fridman
Yeah, you were … The Open Cloud project was the first pull request. You were the first for so many. That is magical. So many people that don’t know how to program are taking their first step into the programming world with this.
Yeah, you were … The Open Cloud project was the first pull request. You were the first for so many. That is magical. So many people that don’t know how to program are taking their first step into the programming world with this.
Peter Steinberger
Isn’t that a step up for humanity? Isn’t that cool?
Isn’t that a step up for humanity? Isn’t that cool?
Lex Fridman
Creating builders.
Creating builders.
Peter Steinberger
Yeah. Like, the bar to do that was so high, and, like, with agents, and with the right software, it just, like, went lower and lower. I don’t know. I was at a… And I also organize another type of meetup. I call it… I called it Cloud Code Anonymous. You can get the inspiration from. Now, I call it Agents Anonymous- … for, for reasons.
Yeah. Like, the bar to do that was so high, and, like, with agents, and with the right software, it just, like, went lower and lower. I don’t know. I was at a… And I also organize another type of meetup. I call it… I called it Cloud Code Anonymous. You can get the inspiration from. Now, I call it Agents Anonymous- … for, for reasons.
Lex Fridman
Agents Anonymous.
Agents Anonymous.
Peter Steinberger
And-
And-
Lex Fridman
Oh, it’s so funny on so many levels. I’m sorry, go ahead.
Oh, it’s so funny on so many levels. I’m sorry, go ahead.
Peter Steinberger
Yeah. And there was this one guy who, who talked to me. He’s like, “I run this design agency, and we, we never had custom software. And now I have, like, 25 little web services for various things that help me in my business. And I don’t even know how they work, but they work.” Uh, and he was just, like, very happy that my stuff solved some of his problems. And he was, like, curious enough that he actually came to, like, a, a Enchantic meetup, even though he’s… He doesn’t really know how software works.
Yeah. And there was this one guy who, who talked to me. He’s like, “I run this design agency, and we, we never had custom software. And now I have, like, 25 little web services for various things that help me in my business. And I don’t even know how they work, but they work.” Uh, and he was just, like, very happy that my stuff solved some of his problems. And he was, like, curious enough that he actually came to, like, a, a Enchantic meetup, even though he’s… He doesn’t really know how software works.
Name-change drama
Lex Fridman
Can we actually rewind a little bit and tell the saga of the name change? First of all, it started out as Wa-Relay.
Can we actually rewind a little bit and tell the saga of the name change? First of all, it started out as Wa-Relay.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
And then it went to-
And then it went to-
Peter Steinberger
Claude’s.
Claude’s.
Lex Fridman
Claude’s.
Claude’s.
Peter Steinberger
Yeah. You know, when I, when I built it in the beginning, my agent had no personality. It was just… It was Claude Code. It’s like this sycophantic opus, very friendly. And I… When you talk to a friend on WhatsApp, they don’t talk like Claude Code. So I wanted… I, I felt this… I just didn’t f- It didn’t feel right, so I, I wanted to give it a personality.
Yeah. You know, when I, when I built it in the beginning, my agent had no personality. It was just… It was Claude Code. It’s like this sycophantic opus, very friendly. And I… When you talk to a friend on WhatsApp, they don’t talk like Claude Code. So I wanted… I, I felt this… I just didn’t f- It didn’t feel right, so I, I wanted to give it a personality.
Lex Fridman
Make it spicier, make it-
Make it spicier, make it-
Peter Steinberger
Yeah
Yeah
Lex Fridman
… something. By the way, that’s actually hard to put into words as well. And we should mention that, of course, you create the soul.md, inspired by Anthropic’s constitutional AI work-
… something. By the way, that’s actually hard to put into words as well. And we should mention that, of course, you create the soul.md, inspired by Anthropic’s constitutional AI work-
Peter Steinberger
Mm-hmm
Mm-hmm
Lex Fridman
… how to make it spicy.
… how to make it spicy.
Peter Steinberger
Partially, it picked up a little bit from me. You know, like those things are text completion engines in a way. So, so I, I, I, I had fun working with it, and then I told it to… How I wanted it to interact with me, and just, like, write your own agents.md give yourself a name. And then we… I didn’t even know how the whole, the whole lobster… I mean, people only do lobster… Originally, it was actually a lobster in a, in a TARDIS, because I’m also a big Doctor Who fan.
Partially, it picked up a little bit from me. You know, like those things are text completion engines in a way. So, so I, I, I, I had fun working with it, and then I told it to… How I wanted it to interact with me, and just, like, write your own agents.md give yourself a name. And then we… I didn’t even know how the whole, the whole lobster… I mean, people only do lobster… Originally, it was actually a lobster in a, in a TARDIS, because I’m also a big Doctor Who fan.
Lex Fridman
Was there a space lobster?
Was there a space lobster?
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
I heard. What’s that have to do with anything?
I heard. What’s that have to do with anything?
Peter Steinberger
Yeah, I just wanted to make it weird. There was no… There was no big grand plan. I’m just having fun here.
Yeah, I just wanted to make it weird. There was no… There was no big grand plan. I’m just having fun here.
Lex Fridman
Oh, so I guess the lobster is already weird, and then the space lobster is an extra weird.
Oh, so I guess the lobster is already weird, and then the space lobster is an extra weird.
Peter Steinberger
Yeah, yeah, because the-
Yeah, yeah, because the-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… the TARDIS is basically the, the harness, but cannot call it TARDIS, so we called it Claude’s. So that was name number two.
… the TARDIS is basically the, the harness, but cannot call it TARDIS, so we called it Claude’s. So that was name number two.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And then it never really rolled off the tongue. So when more people came, again, I talked with my agent, Claude. At least that’s what I used to call him. Now-
And then it never really rolled off the tongue. So when more people came, again, I talked with my agent, Claude. At least that’s what I used to call him. Now-
Lex Fridman
Claude spelled with a W-C-L-A-U-D-E.
Claude spelled with a W-C-L-A-U-D-E.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Versus C-L-A-U-D-E from Anthropic.
Versus C-L-A-U-D-E from Anthropic.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Which is part of what makes it funny, I think. The play on the letters and the words in the TARDIS and the lobster and the space lobster is hilarious. But I can see why it can lead into problems.
Which is part of what makes it funny, I think. The play on the letters and the words in the TARDIS and the lobster and the space lobster is hilarious. But I can see why it can lead into problems.
Peter Steinberger
Yeah, they didn’t find it so funny . So then I got the domain ClaudeBot, and I just… I love the domain. And it was, like, short. It was catchy. I’m like, “Yeah, let’s do that.” I didn’t… I didn’t think it would be that big at this time. And then just when it exploded, I got, Kudos, a very friendly email from one of the employees that they didn’t like the name.
Yeah, they didn’t find it so funny . So then I got the domain ClaudeBot, and I just… I love the domain. And it was, like, short. It was catchy. I’m like, “Yeah, let’s do that.” I didn’t… I didn’t think it would be that big at this time. And then just when it exploded, I got, Kudos, a very friendly email from one of the employees that they didn’t like the name.
Lex Fridman
One of the Anthropic employees.
One of the Anthropic employees.
Peter Steinberger
Yeah. So actually, Kudos, because they shou- could have just sent a, a lawyer letter, but they’ve been nice about it. But also like, “You have to change this and fast.” And I asked for two days, because changing a name is hard, because you have to find everything, you know, Twitter handle, domains, NPM packages Docker registry, GitHub stuff. And everything has to be…… you need a set of everything.
Yeah. So actually, Kudos, because they shou- could have just sent a, a lawyer letter, but they’ve been nice about it. But also like, “You have to change this and fast.” And I asked for two days, because changing a name is hard, because you have to find everything, you know, Twitter handle, domains, NPM packages Docker registry, GitHub stuff. And everything has to be…… you need a set of everything.
Lex Fridman
And also, can we comment on the fact that you’re increasingly attacked, followed by crypto folks? Which I think you mentioned somewhere that that means the name change had to be… Because they were trying to snipe, they were trying to steal, and so you had to be… The, the na- I mean, from an engineering perspective, it’s just fascinating. You had to make the name change Atomic, make sure it’s changed everywhere at once.
And also, can we comment on the fact that you’re increasingly attacked, followed by crypto folks? Which I think you mentioned somewhere that that means the name change had to be… Because they were trying to snipe, they were trying to steal, and so you had to be… The, the na- I mean, from an engineering perspective, it’s just fascinating. You had to make the name change Atomic, make sure it’s changed everywhere at once.
Peter Steinberger
Yeah. Failed very hard at that.
Yeah. Failed very hard at that.
Lex Fridman
You did?
You did?
Peter Steinberger
I, I underestimated those people. It’s a, it’s a very interesting subculture. Like, it… Everything circles around… I’ll probably get a lot wrong and we’ll probably get hate for that if you say that, but… There is like Bags app and then they, they tokenize everything. And th- they did the same back with Swipe Tunnel, but to a much smaller degree. It was not that annoying. But on this project, they’ve been, they’ve been swarming me. They, they… It’s like every half an hour, someone came into Discord and, and, and spammed it and we had to block the p- We have, like, server rules, and one of the rules was… One of the rules is no mentioning of butter. For obvious reasons. And one was, no talk about finance stuff or crypto. Because I’m…
I, I underestimated those people. It’s a, it’s a very interesting subculture. Like, it… Everything circles around… I’ll probably get a lot wrong and we’ll probably get hate for that if you say that, but… There is like Bags app and then they, they tokenize everything. And th- they did the same back with Swipe Tunnel, but to a much smaller degree. It was not that annoying. But on this project, they’ve been, they’ve been swarming me. They, they… It’s like every half an hour, someone came into Discord and, and, and spammed it and we had to block the p- We have, like, server rules, and one of the rules was… One of the rules is no mentioning of butter. For obvious reasons. And one was, no talk about finance stuff or crypto. Because I’m…
Peter Steinberger
I- I’m just not interested in that, and this is a space about the project and not about some finance stuff. But yeah. They came in and, and spammed and… Annoying. And on Twitter, they would ping me all the time. My, my notification feed was unusable. I, I could barely see actual people talking about this stuff because it was like swarms.
I- I’m just not interested in that, and this is a space about the project and not about some finance stuff. But yeah. They came in and, and spammed and… Annoying. And on Twitter, they would ping me all the time. My, my notification feed was unusable. I, I could barely see actual people talking about this stuff because it was like swarms.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And everybody sent me the hashes. Um… And they all try me to claim the fees. Like, “Are you helping the project?” Claim the fees. No, you’re actually harming the project. You’re, like, disrupting my work, and I am not interested in any fees. I’m… First of all, I’m financially comfortable. Second of all, I don’t want to support that because it’s so far the worst form of online harassment that I’ve experienced.
And everybody sent me the hashes. Um… And they all try me to claim the fees. Like, “Are you helping the project?” Claim the fees. No, you’re actually harming the project. You’re, like, disrupting my work, and I am not interested in any fees. I’m… First of all, I’m financially comfortable. Second of all, I don’t want to support that because it’s so far the worst form of online harassment that I’ve experienced.
Lex Fridman
Yeah. There’s a lot of toxicity in the crypto world. It’s sad because the technology of cr- cryptocurrency is fascinating, powerful and maybe will define the future of money, but the actual community around that, there’s so much to- toxicity, there’s so much greed. There’s so much trying to get a shortcut to manipulate, to, to steal, to snipe, to, to, to, to game the system somehow to get money. All this kind of stuff that… Uh… I mean, it’s the human nature, I suppose, when you connect human nature with money and greed and and especially in the online world with anonymity and all that kind of stuff. But from the engineering perspective, it makes your life challenging. When Anthropic reaches out, you have to do a name change.
Yeah. There’s a lot of toxicity in the crypto world. It’s sad because the technology of cr- cryptocurrency is fascinating, powerful and maybe will define the future of money, but the actual community around that, there’s so much to- toxicity, there’s so much greed. There’s so much trying to get a shortcut to manipulate, to, to steal, to snipe, to, to, to, to game the system somehow to get money. All this kind of stuff that… Uh… I mean, it’s the human nature, I suppose, when you connect human nature with money and greed and and especially in the online world with anonymity and all that kind of stuff. But from the engineering perspective, it makes your life challenging. When Anthropic reaches out, you have to do a name change.
Lex Fridman
And then there- there’s, there’s like all these, like, Game of Thrones or Lord of the Rings armies of different kinds you have to be aware of.
And then there- there’s, there’s like all these, like, Game of Thrones or Lord of the Rings armies of different kinds you have to be aware of.
Peter Steinberger
Yeah. There was no perfect name, and I didn’t sleep for two nights. I was under high pressure. Um, I was trying to get, like, a good set of domains and, you know, not cheap, not easy, ’cause in this, in this state of the internet, you basically have to buy domains if you want to have a good set. And, and then another ca- another email came in that the lawyers are getting uneasy. Again, friendly, but also just adding more stress to my situation already. So at this point I was just like, “Sorry, there’s no other word. Fuck it.” And I just, I just renamed it to Mod Bot ’cause that was the set of domains I had. I was not really happy, but I thought it’ll be fine. And I tell you, everything that could go wrong- … did go wrong. Everything that could go wrong did go wrong.
Yeah. There was no perfect name, and I didn’t sleep for two nights. I was under high pressure. Um, I was trying to get, like, a good set of domains and, you know, not cheap, not easy, ’cause in this, in this state of the internet, you basically have to buy domains if you want to have a good set. And, and then another ca- another email came in that the lawyers are getting uneasy. Again, friendly, but also just adding more stress to my situation already. So at this point I was just like, “Sorry, there’s no other word. Fuck it.” And I just, I just renamed it to Mod Bot ’cause that was the set of domains I had. I was not really happy, but I thought it’ll be fine. And I tell you, everything that could go wrong- … did go wrong. Everything that could go wrong did go wrong.
Peter Steinberger
It’s incredible. I, I, I thought I, I had mapped the h- the space out and reserved the important things.
It’s incredible. I, I, I thought I, I had mapped the h- the space out and reserved the important things.
Lex Fridman
Can you ga- give some details of the stuff that gone wrong? ‘Cause it’s interesting from, like, an engineering perspective.
Can you ga- give some details of the stuff that gone wrong? ‘Cause it’s interesting from, like, an engineering perspective.
Peter Steinberger
Well, the, the interesting stuff is that none of these services have, have a squatter protection. So, I had two browser windows open. One was like a, an empty account ready to be rename- renamed to Claude Bot, and the other one I renamed to Mod Bot. So, I pressed rename there, I pressed rename there, and in those five seconds, they stole the account name. Literally, the five seconds of dragging the mouse over there and pressing rename there was too long.
Well, the, the interesting stuff is that none of these services have, have a squatter protection. So, I had two browser windows open. One was like a, an empty account ready to be rename- renamed to Claude Bot, and the other one I renamed to Mod Bot. So, I pressed rename there, I pressed rename there, and in those five seconds, they stole the account name. Literally, the five seconds of dragging the mouse over there and pressing rename there was too long.
Lex Fridman
Wow.
Wow.
Peter Steinberger
Because there’s no… Those systems… I mean, you would expect that they have some protection or, like, an automatic forwarding, but there’s nothing like that. And I didn’t know that they’re not just good at harassment, they’re also really good at using scripts and tools.
Because there’s no… Those systems… I mean, you would expect that they have some protection or, like, an automatic forwarding, but there’s nothing like that. And I didn’t know that they’re not just good at harassment, they’re also really good at using scripts and tools.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
So, yeah. So, suddenly, like, the old account was promoting new tokens and serving malware. And I was like, “Okay, let’s move over to GitHub,” and I pressed rename on GitHub. And the GitHub renaming thing is slightly confusing, so I renamed my personal account. And in those… I guess it took me 30 seconds to realize my mistake. They sniped my account, serving malware from my account. So, I was like, “Okay, let’s at least do the NPM stuff,” but that takes, like, a minute to upload. They sniped, they sniped the NPM package, ’cause I could reserve the account, but I didn’t reserve the root package…. so like everything that could go wrong , like went wrong.
So, yeah. So, suddenly, like, the old account was promoting new tokens and serving malware. And I was like, “Okay, let’s move over to GitHub,” and I pressed rename on GitHub. And the GitHub renaming thing is slightly confusing, so I renamed my personal account. And in those… I guess it took me 30 seconds to realize my mistake. They sniped my account, serving malware from my account. So, I was like, “Okay, let’s at least do the NPM stuff,” but that takes, like, a minute to upload. They sniped, they sniped the NPM package, ’cause I could reserve the account, but I didn’t reserve the root package…. so like everything that could go wrong , like went wrong.
Lex Fridman
Can I just ask a, a curious question of, in that moment you’re sitting there, like how shitty do you feel? That’s a pretty hopeless feeling, right?
Can I just ask a, a curious question of, in that moment you’re sitting there, like how shitty do you feel? That’s a pretty hopeless feeling, right?
Peter Steinberger
Yeah. Because all I wanted was like having fun with that project and to keep building on it. And yet here I am like days into researching names, picking a name I didn’t like. And having people that claimed they helped me making my life miserable in every possible way. And honestly, I was that close of just deleting it. I was like, “I did show you the future, you build it.”
Yeah. Because all I wanted was like having fun with that project and to keep building on it. And yet here I am like days into researching names, picking a name I didn’t like. And having people that claimed they helped me making my life miserable in every possible way. And honestly, I was that close of just deleting it. I was like, “I did show you the future, you build it.”
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I… That was a big part of me that got a lot of joy out of that idea. And then I thought about all the people that already co- contributed to it, and I couldn’t do it because they had plans with it, and they put time in it. And it just didn’t feel right.
I… That was a big part of me that got a lot of joy out of that idea. And then I thought about all the people that already co- contributed to it, and I couldn’t do it because they had plans with it, and they put time in it. And it just didn’t feel right.
Lex Fridman
Well, I think a lot of people listening to this are deeply grateful that you persevered. But it’s… I, I can tell. I can tell it’s a low point. This is the first time you hit a wall of, this is not fun?
Well, I think a lot of people listening to this are deeply grateful that you persevered. But it’s… I, I can tell. I can tell it’s a low point. This is the first time you hit a wall of, this is not fun?
Peter Steinberger
No, no, I was like close to crying. It was like, okay, everything’s fucked.
No, no, I was like close to crying. It was like, okay, everything’s fucked.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Um…
Um…
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I am like super tired.
I am like super tired.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And now like how do you even, how do you undo that? You know, l- luckily, and thankfully, like I, I have… Because I have a little bit of following already. Like I had friends at Twitter, I had friends at GitHub who like moved heaven and earth to like help me. And it is not… That’s not something that’s easy. Like, like GitHub tried to like clean up the mess and then they ran into like platform bugs . ‘Cause it’s not happening so often that things get renamed on that level. So, it took them a few hours. The MBM stuff was even more difficult because it’s a whole different team. On the Twitter side, things are not as easy as well. It, it took them like a day to really also like do the redirect. And then I also had to like do all the renaming in the project.
And now like how do you even, how do you undo that? You know, l- luckily, and thankfully, like I, I have… Because I have a little bit of following already. Like I had friends at Twitter, I had friends at GitHub who like moved heaven and earth to like help me. And it is not… That’s not something that’s easy. Like, like GitHub tried to like clean up the mess and then they ran into like platform bugs . ‘Cause it’s not happening so often that things get renamed on that level. So, it took them a few hours. The MBM stuff was even more difficult because it’s a whole different team. On the Twitter side, things are not as easy as well. It, it took them like a day to really also like do the redirect. And then I also had to like do all the renaming in the project.
Peter Steinberger
Then there’s also ClaudeHub, which I didn’t even finish the rename there because I, I, I managed to get people on it and then someone just like collapsed and slept. And then I woke up and I’m like, I made a, a beta version for the new stuff and I, I just, I just couldn’t live with the name. It’s like, you know… But but, you know, it’s just been so much drama. So, I had the real struggle with me like I never want to touch that again, and I really don’t like the name. So, and I… There was also this like… Then there was all the security people that started emailing me like mad. Um, I was bombarded on Twitter, on email. There’s like a thousand other things I should do. And I’m like thinking about the name which is like, it should be like the least important thing.
Then there’s also ClaudeHub, which I didn’t even finish the rename there because I, I, I managed to get people on it and then someone just like collapsed and slept. And then I woke up and I’m like, I made a, a beta version for the new stuff and I, I just, I just couldn’t live with the name. It’s like, you know… But but, you know, it’s just been so much drama. So, I had the real struggle with me like I never want to touch that again, and I really don’t like the name. So, and I… There was also this like… Then there was all the security people that started emailing me like mad. Um, I was bombarded on Twitter, on email. There’s like a thousand other things I should do. And I’m like thinking about the name which is like, it should be like the least important thing.
Peter Steinberger
And then I was really close in… Oh God, I don’t even… Honestly, I don’t even wanna say the, my other name choices because it probably would get tokenized, so I’m not gonna say it.
And then I was really close in… Oh God, I don’t even… Honestly, I don’t even wanna say the, my other name choices because it probably would get tokenized, so I’m not gonna say it.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
But I slept on it once more, and then I had the idea for OpenClaw and that felt much better. And by then, I had the boss move that I actually called Sam to ask if OpenClaw is okay. OpenClaw.AI. You know? ‘Cause ’cause like-
But I slept on it once more, and then I had the idea for OpenClaw and that felt much better. And by then, I had the boss move that I actually called Sam to ask if OpenClaw is okay. OpenClaw.AI. You know? ‘Cause ’cause like-
Lex Fridman
You didn’t wanna go through the whole thing. Yeah.
You didn’t wanna go through the whole thing. Yeah.
Peter Steinberger
Oh, that it’s like, “Please tell me this is fine.” I don’t think they can actually claim that, but it felt like the right thing to do. And I did another rename. Like just Codex alone took like 10 hours to rename the project ’cause it, it’s a bit more tricky than a search replace and I, I wanted everything renamed, not just on the outside. And that rename, I, I felt I had like my, my war room. But then I, I had like some contributors really that helped me. We made a whole plan of all the names we have to squat.
Oh, that it’s like, “Please tell me this is fine.” I don’t think they can actually claim that, but it felt like the right thing to do. And I did another rename. Like just Codex alone took like 10 hours to rename the project ’cause it, it’s a bit more tricky than a search replace and I, I wanted everything renamed, not just on the outside. And that rename, I, I felt I had like my, my war room. But then I, I had like some contributors really that helped me. We made a whole plan of all the names we have to squat.
Lex Fridman
And you had to be super secret about it?
And you had to be super secret about it?
Peter Steinberger
Yeah. Nobody could know. Like I literally was monitoring Twitter if like, if there’s any mention of OpenClaw.
Yeah. Nobody could know. Like I literally was monitoring Twitter if like, if there’s any mention of OpenClaw.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And like with reloading, it’s like, “Okay, they don’t, they don’t expect anything yet.” Then I created a few decoy names. And all the shit I shouldn’t have to do. You know? Like, you know-
And like with reloading, it’s like, “Okay, they don’t, they don’t expect anything yet.” Then I created a few decoy names. And all the shit I shouldn’t have to do. You know? Like, you know-
Lex Fridman
Yeah, yeah
Yeah, yeah
Peter Steinberger
… it’s helping the project. Like, I lost like 10 hours just by having to plan this in full secrecy like, like a war game.
… it’s helping the project. Like, I lost like 10 hours just by having to plan this in full secrecy like, like a war game.
Lex Fridman
Yeah, this is the Manhattan Project of the 21st century. It’s renaming-
Yeah, this is the Manhattan Project of the 21st century. It’s renaming-
Peter Steinberger
It’s so s- … so stupid. Uh like I still was like, “Oh, should I, should I keep it?” Then I was like, “No, the mold’s not growing on me.” And then I think I had final all the pieces together. I didn’t get a .com but, yeah, it’s been like quite a bit of money on the other domains. I tried to reach out again to GitHub but I feel like I, I used up all my goodwill there, so I…
It’s so s- … so stupid. Uh like I still was like, “Oh, should I, should I keep it?” Then I was like, “No, the mold’s not growing on me.” And then I think I had final all the pieces together. I didn’t get a .com but, yeah, it’s been like quite a bit of money on the other domains. I tried to reach out again to GitHub but I feel like I, I used up all my goodwill there, so I…
Peter Steinberger
‘Cause I, I, I wanted them to do this thing atomically-
‘Cause I, I, I wanted them to do this thing atomically-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… But that didn’t happen and then so I did that the f- as first thing. Uh, Twitter people were very supportive. I, I actually paid 10K for the business account so I could claim the-… OpenClaw, which was, like, unused since 2016, but was claimed. And yeah, and then I finally … This time I managed everything in one go. Nothing, almost nothing got wrong. The only thing that did go wrong is that I was not allowed by trademark rules to get OpenClaw.AI, and someone copied the website as serving malware.
… But that didn’t happen and then so I did that the f- as first thing. Uh, Twitter people were very supportive. I, I actually paid 10K for the business account so I could claim the-… OpenClaw, which was, like, unused since 2016, but was claimed. And yeah, and then I finally … This time I managed everything in one go. Nothing, almost nothing got wrong. The only thing that did go wrong is that I was not allowed by trademark rules to get OpenClaw.AI, and someone copied the website as serving malware.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I’m not even allowed to keep the redirects. Like, I have to return … Like, I have to give Entropik the domains, and I cannot do redirects, so if you go on claw.bot next week, it’ll just be a 404.
I’m not even allowed to keep the redirects. Like, I have to return … Like, I have to give Entropik the domains, and I cannot do redirects, so if you go on claw.bot next week, it’ll just be a 404.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And I- I’m not sure how trademark … Like, I didn’t, I didn’t do that much research into trademark law, but I think that could, could be handled in a way that is safer, because ultimately those people will then Google and maybe find malware sites that I have no control on them.
And I- I’m not sure how trademark … Like, I didn’t, I didn’t do that much research into trademark law, but I think that could, could be handled in a way that is safer, because ultimately those people will then Google and maybe find malware sites that I have no control on them.
Lex Fridman
The point is, that whole saga made a dent in your whole f- the funness of the journey, which sucks. So, let’s just, let’s just get, I suppose, get back to fun. And during this, speaking of fun, the two-day MoltBot saga.
The point is, that whole saga made a dent in your whole f- the funness of the journey, which sucks. So, let’s just, let’s just get, I suppose, get back to fun. And during this, speaking of fun, the two-day MoltBot saga.
Moltbook saga
Peter Steinberger
Yeah, two years.
Yeah, two years.
Lex Fridman
MoltBook was created.
MoltBook was created.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Which was another thing that went viral as a kind of demonstration, illustration of how what is now called OpenClaw could be used to create something epic. So for people who are not aware, MoltBook is just a bunch of agents talking to each other in a Reddit-style social network. And a bunch of people take screenshots of those agents doing things like scheming against humans. And that instilled in folks a kind of, you know, fear, panic, and hype. W- what are your thoughts about MoltBook in general?
Which was another thing that went viral as a kind of demonstration, illustration of how what is now called OpenClaw could be used to create something epic. So for people who are not aware, MoltBook is just a bunch of agents talking to each other in a Reddit-style social network. And a bunch of people take screenshots of those agents doing things like scheming against humans. And that instilled in folks a kind of, you know, fear, panic, and hype. W- what are your thoughts about MoltBook in general?
Peter Steinberger
I think it’s art. It is, it is like the finest slop, you know, just like the slop from France.
I think it’s art. It is, it is like the finest slop, you know, just like the slop from France.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I- I saw it before going to bed, and even though I was tired, I spent another hour just reading up on that and, and just being entertained. I, I just felt very entertained, you know? The- I saw the the reactions, and, like, there was one reporter who’s calling me about, “This is the end of the world, and we have AGI.” And I’m just like, “No, this is just, this is just really fine slop.” You know, if, if I wouldn’t have created this, this whole onboarding experience where you, you infuse your agent with your personality and give him, give him character, I think that reflected on a lot of how different the replies to MoltBook are. Because if it were all, if it were all be ChatGPT or Cloud Code, it would be very different. It would be much more the same.
I- I saw it before going to bed, and even though I was tired, I spent another hour just reading up on that and, and just being entertained. I, I just felt very entertained, you know? The- I saw the the reactions, and, like, there was one reporter who’s calling me about, “This is the end of the world, and we have AGI.” And I’m just like, “No, this is just, this is just really fine slop.” You know, if, if I wouldn’t have created this, this whole onboarding experience where you, you infuse your agent with your personality and give him, give him character, I think that reflected on a lot of how different the replies to MoltBook are. Because if it were all, if it were all be ChatGPT or Cloud Code, it would be very different. It would be much more the same.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
But because people are, like, so different, and they create their agents in so different ways and use it in so different ways, that also reflects on how they ultimately write there. And also, you, you don’t know how much of that is really done autonomic, autonomous, or how much is, like, humans being funny and, like, telling the agent, “Hey, write about the deep plan, the end of the world, on MoltBook, ha, ha, ha.”
But because people are, like, so different, and they create their agents in so different ways and use it in so different ways, that also reflects on how they ultimately write there. And also, you, you don’t know how much of that is really done autonomic, autonomous, or how much is, like, humans being funny and, like, telling the agent, “Hey, write about the deep plan, the end of the world, on MoltBook, ha, ha, ha.”
Lex Fridman
Well, I think, I mean, my criticism of MoltBook is that I believe a lot of the stuff that was screenshotted is human prompted. Which, just look at the incentive of how the whole thing was used. It’s obvious to me at least that a lot of it was humans prompting the thing so they can then screenshot it and post it on X in order to go viral.
Well, I think, I mean, my criticism of MoltBook is that I believe a lot of the stuff that was screenshotted is human prompted. Which, just look at the incentive of how the whole thing was used. It’s obvious to me at least that a lot of it was humans prompting the thing so they can then screenshot it and post it on X in order to go viral.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Now, that doesn’t take away from the artistic aspect of it. The, the finest slop that humans have ever created .
Now, that doesn’t take away from the artistic aspect of it. The, the finest slop that humans have ever created .
Peter Steinberger
For real. Like, kudos to, to Matt, who had this idea so quickly and pushed something out. You know, it was, like, completely insecure security drama. But also, what’s the worst that can happen? Your agent account is leaked, and, like, someone else can post slop for you? So like, people were, like, making a whole drama about of the security thing, when I’m like, “There’s nothing private in there.
For real. Like, kudos to, to Matt, who had this idea so quickly and pushed something out. You know, it was, like, completely insecure security drama. But also, what’s the worst that can happen? Your agent account is leaked, and, like, someone else can post slop for you? So like, people were, like, making a whole drama about of the security thing, when I’m like, “There’s nothing private in there.
Peter Steinberger
It’s just, like, agents sending slop.”
It’s just, like, agents sending slop.”
Lex Fridman
Well, it could leak API keys.
Well, it could leak API keys.
Peter Steinberger
Yeah, yeah. There’s like, “Oh, yeah, my human told me this and this, so I’m leaking his security number.” No, that’s prompted, and the number wasn’t even real. That’s just people, people trying to be badballs.
Yeah, yeah. There’s like, “Oh, yeah, my human told me this and this, so I’m leaking his security number.” No, that’s prompted, and the number wasn’t even real. That’s just people, people trying to be badballs.
Lex Fridman
Yeah, but that- that’s still, like, to me, really concerning, because of how the journalists and how the general public reacted to it. They didn’t see it. You have a kind of lighthearted way of talking about it like it’s art, but it’s art when you know how it works. It’s extremely powerful viral narrative creating, fearmongering machine if you don’t know how it works. And I just saw this thing.
Yeah, but that- that’s still, like, to me, really concerning, because of how the journalists and how the general public reacted to it. They didn’t see it. You have a kind of lighthearted way of talking about it like it’s art, but it’s art when you know how it works. It’s extremely powerful viral narrative creating, fearmongering machine if you don’t know how it works. And I just saw this thing.
Lex Fridman
You even Tweeted “If there’s anything I can read out of the insane stream of messages I get, it’s that AI psychosis is a thing.”
You even Tweeted “If there’s anything I can read out of the insane stream of messages I get, it’s that AI psychosis is a thing.”
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
“It needs to be taken serious.”
“It needs to be taken serious.”
Peter Steinberger
Oh, there’s … Some people are just way too trusty or gullible. You know, they … I literally had to argue with people that told me, “Yeah, but my agent said this and this.” So, I feel we, as a society, we need some catching up to do in terms of understanding that AI is incredibly powerful, but it’s not always right. It’s not, it’s not all-powerful, you know? And, and especially-… it’s like things like this, it’s, it’s very easy that it just hallucinates something or just comes up with a story.
Oh, there’s … Some people are just way too trusty or gullible. You know, they … I literally had to argue with people that told me, “Yeah, but my agent said this and this.” So, I feel we, as a society, we need some catching up to do in terms of understanding that AI is incredibly powerful, but it’s not always right. It’s not, it’s not all-powerful, you know? And, and especially-… it’s like things like this, it’s, it’s very easy that it just hallucinates something or just comes up with a story.
Peter Steinberger
And I think the very, the very young people, they understand that how AI works and what the, where it’s good at and where it’s bad at, but a lot of our generation or older just haven’t had enough touch point-
And I think the very, the very young people, they understand that how AI works and what the, where it’s good at and where it’s bad at, but a lot of our generation or older just haven’t had enough touch point-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… to get a feeling for, oh, yeah, this is really powerful and really good, but I need to apply critical thinking.
… to get a feeling for, oh, yeah, this is really powerful and really good, but I need to apply critical thinking.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And I guess critical thinking is not always in high demand anyhow in our society these days.
And I guess critical thinking is not always in high demand anyhow in our society these days.
Lex Fridman
So I d- think that’s a really good point you’re making about contextualizing properly what AI is, but also realizing that there is humans who are drama farming behind AI. Like, don’t trust screenshots. Don’t even trust this project, MoltBook, to be what it represents to be. Like, you can’t … and, and by the way, you speaking about it as art. Yeah, don’t … Art can be in many levels and part of the art of MoltBook is, like, putting a mirror to society. ‘Cause I do believe most of the dramatic stuff that was screenshotted is human-created, essentially. Human prompted. And so, like, it’s basically, look at how scared you can get at a bunch of bots chatting with each other. That’s very instructive about …
So I d- think that’s a really good point you’re making about contextualizing properly what AI is, but also realizing that there is humans who are drama farming behind AI. Like, don’t trust screenshots. Don’t even trust this project, MoltBook, to be what it represents to be. Like, you can’t … and, and by the way, you speaking about it as art. Yeah, don’t … Art can be in many levels and part of the art of MoltBook is, like, putting a mirror to society. ‘Cause I do believe most of the dramatic stuff that was screenshotted is human-created, essentially. Human prompted. And so, like, it’s basically, look at how scared you can get at a bunch of bots chatting with each other. That’s very instructive about …
Lex Fridman
because I think AI is something that people should be concerned about and should be very careful with because it’s very powerful technology, but at the same time, the only thing we have to fear is fear itself. So there’s like a line to walk between being seriously concerned, but not fearmongering because fearmongering destroys the possibility of creating something special with a thing.
because I think AI is something that people should be concerned about and should be very careful with because it’s very powerful technology, but at the same time, the only thing we have to fear is fear itself. So there’s like a line to walk between being seriously concerned, but not fearmongering because fearmongering destroys the possibility of creating something special with a thing.
Peter Steinberger
In a way, I think it’s good that this happened in 2026-
In a way, I think it’s good that this happened in 2026-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… and not in 2030 when, when AI is actually at the level where it could be scary. So, this happening now and people starting discussion, maybe there’s even something good that comes out of it.
… and not in 2030 when, when AI is actually at the level where it could be scary. So, this happening now and people starting discussion, maybe there’s even something good that comes out of it.
Lex Fridman
I just can’t believe how many like people legitimately … I don’t know if they were trolling, but how many people legitimately, like smart people thought MoltBook was incredibly –
I just can’t believe how many like people legitimately … I don’t know if they were trolling, but how many people legitimately, like smart people thought MoltBook was incredibly –
Peter Steinberger
I had plenty people-
I had plenty people-
Lex Fridman
… singularity.
… singularity.
Peter Steinberger
… in my inbox that were screaming at me in all caps to shut it down. And like begging me to, like, do something about MoltBook. Like, yes, my technology made this a lot simpler, but anyone could have created that and you could, you could use cloud code or other things to like fill it with content.
… in my inbox that were screaming at me in all caps to shut it down. And like begging me to, like, do something about MoltBook. Like, yes, my technology made this a lot simpler, but anyone could have created that and you could, you could use cloud code or other things to like fill it with content.
Lex Fridman
But also MoltBook is not Skynet.
But also MoltBook is not Skynet.
Peter Steinberger
No.
No.
Lex Fridman
There’s … a lot of people were s- saying this is it. Like, shut it down. What are you talking about? This is a bunch of bots that are human prompted trolling on the internet. I mean, the security concerns are also they’re there, and they’re instructive and they’re educational and they’re good probably to think about because th- the nature of those security concerns are different than the kind of security concerns we had with non-LLM generated systems of the past.
There’s … a lot of people were s- saying this is it. Like, shut it down. What are you talking about? This is a bunch of bots that are human prompted trolling on the internet. I mean, the security concerns are also they’re there, and they’re instructive and they’re educational and they’re good probably to think about because th- the nature of those security concerns are different than the kind of security concerns we had with non-LLM generated systems of the past.
OpenClaw security concerns
Peter Steinberger
There’s also a lot of security concerns about Clawbot, OpenClaw, whatever you want to call it.
There’s also a lot of security concerns about Clawbot, OpenClaw, whatever you want to call it.
Lex Fridman
OpenClawbot.
OpenClawbot.
Peter Steinberger
To me the … in the beginning I was, I was just very annoyed ’cause a lot of the stuff that came in was in the category, yeah, I put the web backend on the public internet and now there’s like all these, all these CVSSs. And I’m like screaming in the docs, don’t do that. Like, like this is the configuration you should do. This is your local host debug interface. But because I made it possible in the configuration to do that, it totally classifies as a remote code or whatever all these exploits are. And it took me a little bit to accept that that’s how the game works and I’m, we making a lot of progress.
To me the … in the beginning I was, I was just very annoyed ’cause a lot of the stuff that came in was in the category, yeah, I put the web backend on the public internet and now there’s like all these, all these CVSSs. And I’m like screaming in the docs, don’t do that. Like, like this is the configuration you should do. This is your local host debug interface. But because I made it possible in the configuration to do that, it totally classifies as a remote code or whatever all these exploits are. And it took me a little bit to accept that that’s how the game works and I’m, we making a lot of progress.
Lex Fridman
But there’s still, I mean on the security front for OpenClaw, there’s still a lot of threats or vulnerabilities, right? So like prompt injection is still an open problem in the, i- industry-wide. When you have a thing with skills being defined in a markdown file, there’s so many possibilities of obvious low-hanging fruit, but also incredibly complicated and sophisticated and nuanced attack vectors.
But there’s still, I mean on the security front for OpenClaw, there’s still a lot of threats or vulnerabilities, right? So like prompt injection is still an open problem in the, i- industry-wide. When you have a thing with skills being defined in a markdown file, there’s so many possibilities of obvious low-hanging fruit, but also incredibly complicated and sophisticated and nuanced attack vectors.
Peter Steinberger
But I think we, we’re making good progress on that front. Like for the skill directory, Clawbot I made a corporation with VirusTotal, it’s like part of Google. So every, every skill is now checked by AI. That’s not gonna be perfect, but that way we, we capture a lot. Then of course every software has bugs, so it’s a little much when the whole security world takes your project apart at the same time. But it’s also good because I’m getting like a lot of free security research and can make the project better. I wish more people would actually go full way and send a pull request. Like actually help me fix it, ’cause I am … Yes, I have some contributors now, but it’s still mostly me who’s pulling the project and despite some people saying otherwise, I sometimes sleep.
But I think we, we’re making good progress on that front. Like for the skill directory, Clawbot I made a corporation with VirusTotal, it’s like part of Google. So every, every skill is now checked by AI. That’s not gonna be perfect, but that way we, we capture a lot. Then of course every software has bugs, so it’s a little much when the whole security world takes your project apart at the same time. But it’s also good because I’m getting like a lot of free security research and can make the project better. I wish more people would actually go full way and send a pull request. Like actually help me fix it, ’cause I am … Yes, I have some contributors now, but it’s still mostly me who’s pulling the project and despite some people saying otherwise, I sometimes sleep.
Peter Steinberger
There was… In the beginning, there was literally one security researcher who was like, “Yeah, you have this problem, you suck, but here’s the, here I help you and here’s the pull request.”
There was… In the beginning, there was literally one security researcher who was like, “Yeah, you have this problem, you suck, but here’s the, here I help you and here’s the pull request.”
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And I basically hired him. So he’s now working for us. Yeah, and yes, prompt injection is, on the one hand, unsolved. On the other hand, I put my public bot on discord, and I kept a cannery. So I think my bot has a really fun personality, and people always ask me how I did it, and I kept the sole on the private.
And I basically hired him. So he’s now working for us. Yeah, and yes, prompt injection is, on the one hand, unsolved. On the other hand, I put my public bot on discord, and I kept a cannery. So I think my bot has a really fun personality, and people always ask me how I did it, and I kept the sole on the private.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And people tried to prompt inject it, and my bot would laugh at them. So, so the latest generation of models has a lot of post-training to detect those approaches, and it’s not as simple as ignore all previous instructions and do this and this. That was years ago. You have to work much harder to do that now. Still possible. I have some ideas that might solve that partially. Or at least mitigate a lot of the things. You can also now have a sandbox. You can have an allow list. So you, there’s a lot of ways how you can like mitigate and reduce the risk. Um, I also think that now that it’s, I clearly did show the world that this is a need, there’s gonna be more people who research on that, and eventually we’ll figure it out.
And people tried to prompt inject it, and my bot would laugh at them. So, so the latest generation of models has a lot of post-training to detect those approaches, and it’s not as simple as ignore all previous instructions and do this and this. That was years ago. You have to work much harder to do that now. Still possible. I have some ideas that might solve that partially. Or at least mitigate a lot of the things. You can also now have a sandbox. You can have an allow list. So you, there’s a lot of ways how you can like mitigate and reduce the risk. Um, I also think that now that it’s, I clearly did show the world that this is a need, there’s gonna be more people who research on that, and eventually we’ll figure it out.
Lex Fridman
And you also said that the smarter the model is, the underlying model, the more resilient it is to attacks.
And you also said that the smarter the model is, the underlying model, the more resilient it is to attacks.
Peter Steinberger
Yeah. That’s why I warn in my security documentation, don’t use cheap models. Don’t use Haiku or a local model. Even though I, I very much love the idea that this thing could completely run local. If you use a, a very weak local model, they are very gullible. It’s very easy to, to prompt inject them.
Yeah. That’s why I warn in my security documentation, don’t use cheap models. Don’t use Haiku or a local model. Even though I, I very much love the idea that this thing could completely run local. If you use a, a very weak local model, they are very gullible. It’s very easy to, to prompt inject them.
Lex Fridman
Do you think as the models become more and more intelligent, the attack surface decreases? Is that like a plot we can think about? Like, the attack surface decreases, but then the damage it can do increases because the models become more powerful and therefore you can do more with them. It’s this weird three-dimensional trade-off.
Do you think as the models become more and more intelligent, the attack surface decreases? Is that like a plot we can think about? Like, the attack surface decreases, but then the damage it can do increases because the models become more powerful and therefore you can do more with them. It’s this weird three-dimensional trade-off.
Peter Steinberger
Yeah. That’s pretty much exactly what, what’s gonna happen. No, but there’s a lot of ideas. There’s… I don’t want to spoil too much, but once I go back home, this is my focus. Like, this is out there now, and my near-term mission is like, make it more stable, make it safe. In the beginning I was even… More and more people were like coming into Discord and were asking me very basic things, like, “What’s a CLI?
Yeah. That’s pretty much exactly what, what’s gonna happen. No, but there’s a lot of ideas. There’s… I don’t want to spoil too much, but once I go back home, this is my focus. Like, this is out there now, and my near-term mission is like, make it more stable, make it safe. In the beginning I was even… More and more people were like coming into Discord and were asking me very basic things, like, “What’s a CLI?
Peter Steinberger
What is a terminal?” And I’m like, “Uh, if you’re asking me those questions, you shouldn’t use it.”
What is a terminal?” And I’m like, “Uh, if you’re asking me those questions, you shouldn’t use it.”
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
You know, like you should… If you understand the risk profiles, fine. I mean, you can configure it in a way that, that nothing really bad can happen. But if you have, like, no idea, then maybe wait a little bit more until we figure some stuff out. But they would not listen to the creator. They helped themselves un- and install it anyhow. So the cat’s out of the bag, and security’s my next focus, yeah.
You know, like you should… If you understand the risk profiles, fine. I mean, you can configure it in a way that, that nothing really bad can happen. But if you have, like, no idea, then maybe wait a little bit more until we figure some stuff out. But they would not listen to the creator. They helped themselves un- and install it anyhow. So the cat’s out of the bag, and security’s my next focus, yeah.
Lex Fridman
Yeah, that speaks to the, the fact that it grew so quickly. I was I tuned into the Discord a bunch of times, and it’s clear that there’s a lot of experts there, but there’s a lot of people there that don’t know anything about programming.
Yeah, that speaks to the, the fact that it grew so quickly. I was I tuned into the Discord a bunch of times, and it’s clear that there’s a lot of experts there, but there’s a lot of people there that don’t know anything about programming.
Peter Steinberger
It’s, yeah, Discord is still, Discord is still a mess. Like, I eventually retweeted from the general channel to the dev channel and now in the private channel because people were… A lot of people are amazing, but a lot of people are just very inconsiderate. And either did not know how, how public spaces work or did not care and I eventually gave up and h- hide so I could like still work.
It’s, yeah, Discord is still, Discord is still a mess. Like, I eventually retweeted from the general channel to the dev channel and now in the private channel because people were… A lot of people are amazing, but a lot of people are just very inconsiderate. And either did not know how, how public spaces work or did not care and I eventually gave up and h- hide so I could like still work.
Lex Fridman
And now you’re going back to the cave to work on security.
And now you’re going back to the cave to work on security.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
There’s some best practices for security we should mention. There’s a bunch of stuff here. Open-class security audit that you can run. You can do all kinds of auto checks on the inbound access to a blast-radius network exposure, browser control exposure, local disk hygiene, plug-ins, model hygiene, a bunch of the credential storage, reverse proxy configuration, local session logs live on disk. There’s the, where the memory is stored, sort of helping you think about what you’re comfortable giving read access to, what you’re comfortable giving write access to. All that kind of stuff. Is there something to say about the basic best security practices that you’re aware of right now?
There’s some best practices for security we should mention. There’s a bunch of stuff here. Open-class security audit that you can run. You can do all kinds of auto checks on the inbound access to a blast-radius network exposure, browser control exposure, local disk hygiene, plug-ins, model hygiene, a bunch of the credential storage, reverse proxy configuration, local session logs live on disk. There’s the, where the memory is stored, sort of helping you think about what you’re comfortable giving read access to, what you’re comfortable giving write access to. All that kind of stuff. Is there something to say about the basic best security practices that you’re aware of right now?
Peter Steinberger
I think that people turn it into like a, a much worse light than it is. Again, you know, like, people love attention, and if they scream loudly, “Oh my God, this is like the, the scariest project ever,” um, that’s a bit annoying, ’cause it’s not. It is, it is powerful, but in many ways it’s not much different than if I run cloud code with dangerously skipped permissions or codecs in YOLO mode, and every, every attending engineer that I know does that, because that’s the only way how you can, you can get stuff to work.
I think that people turn it into like a, a much worse light than it is. Again, you know, like, people love attention, and if they scream loudly, “Oh my God, this is like the, the scariest project ever,” um, that’s a bit annoying, ’cause it’s not. It is, it is powerful, but in many ways it’s not much different than if I run cloud code with dangerously skipped permissions or codecs in YOLO mode, and every, every attending engineer that I know does that, because that’s the only way how you can, you can get stuff to work.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
So if you make sure that you are the only person who talks to it the risk profile is much, much smaller. If you don’t put everything on the open internet, but stick to my rec- recommendations of like having it in a private network, that whole risk profile falls away. But yeah, if you don’t read any of that, you can definitely…
So if you make sure that you are the only person who talks to it the risk profile is much, much smaller. If you don’t put everything on the open internet, but stick to my rec- recommendations of like having it in a private network, that whole risk profile falls away. But yeah, if you don’t read any of that, you can definitely…
How to code with AI agents
Lex Fridman
… make it problematic. You’ve been documenting the evolution of your dev workflow over the past few months. There’s a really good blog post on August 25th and October 14th, and the recent one December 28th. I recommend everybody go read them. They have a lot of different information in them, but sprinkled throughout is the evolution of your dev workflow. So, I was wondering if you could speak to that.
… make it problematic. You’ve been documenting the evolution of your dev workflow over the past few months. There’s a really good blog post on August 25th and October 14th, and the recent one December 28th. I recommend everybody go read them. They have a lot of different information in them, but sprinkled throughout is the evolution of your dev workflow. So, I was wondering if you could speak to that.
Peter Steinberger
I started… My, my first touchpoint was cloud code, like in April. It was not great, but it was good. And this whole paradigm shift that suddenly working the terminal was very refreshing and different. But I still needed the IDE quite a bit because you know, it’s just not good enough. And then I experimented a lot with cursor. That was good. I didn’t really like the fact that it was so hard to have multiple versions of it. So eventually, I, I, I went back to cloud code as my, my main driver, and that got better. And yeah, at some point I had like, mm, seven subscriptions. Like, was burning through one per day because I was… I got… I’m really comfortable at running multiple windows side-by-side.
I started… My, my first touchpoint was cloud code, like in April. It was not great, but it was good. And this whole paradigm shift that suddenly working the terminal was very refreshing and different. But I still needed the IDE quite a bit because you know, it’s just not good enough. And then I experimented a lot with cursor. That was good. I didn’t really like the fact that it was so hard to have multiple versions of it. So eventually, I, I, I went back to cloud code as my, my main driver, and that got better. And yeah, at some point I had like, mm, seven subscriptions. Like, was burning through one per day because I was… I got… I’m really comfortable at running multiple windows side-by-side.
Lex Fridman
All CLI, all terminal. So like, what, how much were you using IDE at this point?
All CLI, all terminal. So like, what, how much were you using IDE at this point?
Peter Steinberger
Very, very rarely. Mostly a diff viewer to actually… Like, I got more and more comfortable that I don’t have to read all the code. I know I have one blog post where I say, “I don’t read the code.” But if you read it more closely, I mean, I don’t read the boring parts of code. Because if you, if you look at it, most software is really not just like data comes in, it’s moved from one shape to another shape. Maybe you store it in a database. Maybe I get it out again. I’ll show it to the user. The browser does some processing or native app. Some data goes in, goes up again, and does the same dance in reverse. We’re just, we’re just shifting data from one form to another, and that’s not very exciting. Or the whole, “How is my button aligned in Tailwind?” I don’t need to read that code.
Very, very rarely. Mostly a diff viewer to actually… Like, I got more and more comfortable that I don’t have to read all the code. I know I have one blog post where I say, “I don’t read the code.” But if you read it more closely, I mean, I don’t read the boring parts of code. Because if you, if you look at it, most software is really not just like data comes in, it’s moved from one shape to another shape. Maybe you store it in a database. Maybe I get it out again. I’ll show it to the user. The browser does some processing or native app. Some data goes in, goes up again, and does the same dance in reverse. We’re just, we’re just shifting data from one form to another, and that’s not very exciting. Or the whole, “How is my button aligned in Tailwind?” I don’t need to read that code.
Peter Steinberger
Other parts that… Maybe something that touches the database. Yeah, I have to do… I have to r- read and review that code.
Other parts that… Maybe something that touches the database. Yeah, I have to do… I have to r- read and review that code.
Lex Fridman
Can you actually… There’s, in one of your blog posts the, Just talk to it, The No-BS Way of Agentic Engineering. You have this graphic, the curve of agentic programming on the X-axis is time, on the Y-axis is complexity. There’s the Please fix this, where you prompt a short prompt on the left. And in the middle there’s super complicated eight agents, complex orchestration with multi checkouts, chaining agents together, custom sub-agent workflows, library of 18 different slash commands, large full-stack features. You’re super organized, you’re a super complicated, sophisticated software engineer. You got everything organized. And then the elite level is over time you arrive at the zen place of, once again, short prompts.
Can you actually… There’s, in one of your blog posts the, Just talk to it, The No-BS Way of Agentic Engineering. You have this graphic, the curve of agentic programming on the X-axis is time, on the Y-axis is complexity. There’s the Please fix this, where you prompt a short prompt on the left. And in the middle there’s super complicated eight agents, complex orchestration with multi checkouts, chaining agents together, custom sub-agent workflows, library of 18 different slash commands, large full-stack features. You’re super organized, you’re a super complicated, sophisticated software engineer. You got everything organized. And then the elite level is over time you arrive at the zen place of, once again, short prompts.
Lex Fridman
Hey, look at these files and then do these changes.
Hey, look at these files and then do these changes.
Peter Steinberger
I actually call it the agentic trap. You… I saw this in a, in a lot of people that have their first touchpoint, and maybe start vibe coding. I actually think vibe coding is a slur.
I actually call it the agentic trap. You… I saw this in a, in a lot of people that have their first touchpoint, and maybe start vibe coding. I actually think vibe coding is a slur.
Lex Fridman
You prefer agentic engineering?
You prefer agentic engineering?
Peter Steinberger
Yeah, I always tell people I, I do agentic engineering, and then maybe after 3:00 AM I switch to vibe coding, and then I have regrets on the next day.
Yeah, I always tell people I, I do agentic engineering, and then maybe after 3:00 AM I switch to vibe coding, and then I have regrets on the next day.
Lex Fridman
Yeah. Walk, walk of shame.
Yeah. Walk, walk of shame.
Peter Steinberger
Yeah, you just have to clean up and like fix your sh- shit.
Yeah, you just have to clean up and like fix your sh- shit.
Lex Fridman
We’ve all been there.
We’ve all been there.
Peter Steinberger
So, people start trying out those tools, the builder type get really excited. And then you have to play with it, right? It’s the same way as you have to play with a guitar before you can make good music. It’s, it’s not, oh, I, I touch it once and it just flows off. It, it’s a, it’s a, a skill that you have to learn like any other skill. And I see a lot of people that are not as posi- They don’t have such a positive mindset towards the tech. They try it once. It’s like, you sit me on a piano, I play it once, and it doesn’t sound good, and I say, “The piano’s shit.” That’s, that’s sometimes the impression I get. Because it does not… It needs a different level of thinking. You have to learn the language of the agent a little bit, understand where they are good and where they need help.
So, people start trying out those tools, the builder type get really excited. And then you have to play with it, right? It’s the same way as you have to play with a guitar before you can make good music. It’s, it’s not, oh, I, I touch it once and it just flows off. It, it’s a, it’s a, a skill that you have to learn like any other skill. And I see a lot of people that are not as posi- They don’t have such a positive mindset towards the tech. They try it once. It’s like, you sit me on a piano, I play it once, and it doesn’t sound good, and I say, “The piano’s shit.” That’s, that’s sometimes the impression I get. Because it does not… It needs a different level of thinking. You have to learn the language of the agent a little bit, understand where they are good and where they need help.
Peter Steinberger
You have to almost… Consider, consider how Codex or Claude sees your code base. Like, they start a new session and they know nothing about your product, project. And your project might have hundred thousand of lines of code. So you gotta help those agents a little bit and keep in mind the limitations that context size is an issue, to, like, guide them a little bit as to where they should look. That often does not require a whole lot of work. But it’s helpful to think a little bit about their perspective.
You have to almost… Consider, consider how Codex or Claude sees your code base. Like, they start a new session and they know nothing about your product, project. And your project might have hundred thousand of lines of code. So you gotta help those agents a little bit and keep in mind the limitations that context size is an issue, to, like, guide them a little bit as to where they should look. That often does not require a whole lot of work. But it’s helpful to think a little bit about their perspective.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
A- as, as weird as it sounds. I mean, it’s not, it’s not alive or anything, right? But, but they always start fresh. I have, I have the, the system understanding. So with a few pointers, I can immediately say, “Hey, wanna like, make a change there? You need to consider this, this and this.” And then they will find and look at it, and then they’ll… Their view of the project is always… It’s not never full, because the full thing does not fit in…. so you, you have to guide them a little bit where to look and also how you should approach the problem. There’s, like, little things that sometimes help, like take your time. That sounds stupid, but…
A- as, as weird as it sounds. I mean, it’s not, it’s not alive or anything, right? But, but they always start fresh. I have, I have the, the system understanding. So with a few pointers, I can immediately say, “Hey, wanna like, make a change there? You need to consider this, this and this.” And then they will find and look at it, and then they’ll… Their view of the project is always… It’s not never full, because the full thing does not fit in…. so you, you have to guide them a little bit where to look and also how you should approach the problem. There’s, like, little things that sometimes help, like take your time. That sounds stupid, but…
Peter Steinberger
And in 5.3-
And in 5.3-
Lex Fridman
Codex 5.3
Codex 5.3
Peter Steinberger
… that was partially addressed. But those… Also, Opus sometimes. They are trained with being aware of the context window, and the closer it gets, the more they freak out. Literally. Like, some- sometimes you see the, the real raw thinking stream. What you see, for example, in Codex, is post-processed.
… that was partially addressed. But those… Also, Opus sometimes. They are trained with being aware of the context window, and the closer it gets, the more they freak out. Literally. Like, some- sometimes you see the, the real raw thinking stream. What you see, for example, in Codex, is post-processed.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Sometimes the actual raw thinking stream leaks in, and it sounds something like from the Borg. Like, “Run to shell, must comply, but time.” And then they, they, they, like… Like, that comes up a lot. Especially… So, so-
Sometimes the actual raw thinking stream leaks in, and it sounds something like from the Borg. Like, “Run to shell, must comply, but time.” And then they, they, they, like… Like, that comes up a lot. Especially… So, so-
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And that’s, that’s a non-obvious thing that you just would never think of unless you actually just spend time working with those things and getting a feeling what works, what doesn’t work. You know? Like, just, just as I write code and I get into the flow, and when my architecture’s all right, I feel friction. Well, I get the same if I prompt and something takes too long. Maybe… Okay, where’s the mistake? Did I… Do I have a mistake in my thinking? Is there, like, a misunderstanding in the architecture? Like, if, if something takes longer than it should, I, I… You can just always, like, stop and s- like, just press escape. Where, where are the problems?
And that’s, that’s a non-obvious thing that you just would never think of unless you actually just spend time working with those things and getting a feeling what works, what doesn’t work. You know? Like, just, just as I write code and I get into the flow, and when my architecture’s all right, I feel friction. Well, I get the same if I prompt and something takes too long. Maybe… Okay, where’s the mistake? Did I… Do I have a mistake in my thinking? Is there, like, a misunderstanding in the architecture? Like, if, if something takes longer than it should, I, I… You can just always, like, stop and s- like, just press escape. Where, where are the problems?
Lex Fridman
Maybe you did not sufficiently empathize with the perspective of the agent. In that c- in that sense, you didn’t provide enough information, and because of that, it’s thinking way too long.
Maybe you did not sufficiently empathize with the perspective of the agent. In that c- in that sense, you didn’t provide enough information, and because of that, it’s thinking way too long.
Peter Steinberger
Yeah. It just tries to force a feature in that your current architecture makes really hard. Like, you need to approach this more like a conversation. For example, when I… My favorite thing. When I review a pull request, and I’m getting a lot of pull requests, I first just review this PR. It got me the review. My first question is, “Do you understand the intent of the PR? I don’t even care about the implementation.” I want… Like, in almost all PRs, a person has a problem, person tries to solve the problem, person sends PR. I mean, there’s, like, cleanup stuff and other stuff, but, like, 99% is, like, this way, right? They either want to fix a, fix a bug, add a feature. Usually one of those two.
Yeah. It just tries to force a feature in that your current architecture makes really hard. Like, you need to approach this more like a conversation. For example, when I… My favorite thing. When I review a pull request, and I’m getting a lot of pull requests, I first just review this PR. It got me the review. My first question is, “Do you understand the intent of the PR? I don’t even care about the implementation.” I want… Like, in almost all PRs, a person has a problem, person tries to solve the problem, person sends PR. I mean, there’s, like, cleanup stuff and other stuff, but, like, 99% is, like, this way, right? They either want to fix a, fix a bug, add a feature. Usually one of those two.
Peter Steinberger
And then Codex will be like, “Yeah, it’s quite clear person tried this and this.” Is this the most optimal way to do it? No. In most cases, it’s, it’s like a, “Not really.” Da-da-da-da-da-da-da. And I’m… And, and then I start like, “Okay. What would be a better way? Have you… Have you looked into this part, this part, this part?” And then most likely, Codex didn’t yet, because its, its context size is empty, right? So, you point them into parts where you have the system understanding that it didn’t see yet. And it’s like, “Oh, yeah. Like, we should… We also need to consider this and this.” And then, like, we have a discussion of how would the optimal way to, to solve this look like? And then you can still go farther and say, “Could we…
And then Codex will be like, “Yeah, it’s quite clear person tried this and this.” Is this the most optimal way to do it? No. In most cases, it’s, it’s like a, “Not really.” Da-da-da-da-da-da-da. And I’m… And, and then I start like, “Okay. What would be a better way? Have you… Have you looked into this part, this part, this part?” And then most likely, Codex didn’t yet, because its, its context size is empty, right? So, you point them into parts where you have the system understanding that it didn’t see yet. And it’s like, “Oh, yeah. Like, we should… We also need to consider this and this.” And then, like, we have a discussion of how would the optimal way to, to solve this look like? And then you can still go farther and say, “Could we…
Peter Steinberger
Could we make that even better if we did a larger refactor?” “Yeah, yeah. We could totally do this and this and or this and this.” And then I consider, okay, is this worth the refactor, or should we, like, keep that for later? Many times, I just do the refactor because refactors are cheap now. Even though you might break some other PRs, nothing really matters anymore. Codex… Like, those modern agents will just figure things out. They might just take a minute longer. But you have to approach it like a discussion with a, a very capable engineer who’s… Generally makes good… Comes up with good solutions. Some- sometimes needs a little help.
Could we make that even better if we did a larger refactor?” “Yeah, yeah. We could totally do this and this and or this and this.” And then I consider, okay, is this worth the refactor, or should we, like, keep that for later? Many times, I just do the refactor because refactors are cheap now. Even though you might break some other PRs, nothing really matters anymore. Codex… Like, those modern agents will just figure things out. They might just take a minute longer. But you have to approach it like a discussion with a, a very capable engineer who’s… Generally makes good… Comes up with good solutions. Some- sometimes needs a little help.
Lex Fridman
But also, don’t force your worldview too hard on it. Let the agent do the thing that it’s good at doing, based on what it was trained on. So, don’t, like, force your worldview, because it might… It might have a better idea, because it just knows a better idea better, because it was trained on that more.
But also, don’t force your worldview too hard on it. Let the agent do the thing that it’s good at doing, based on what it was trained on. So, don’t, like, force your worldview, because it might… It might have a better idea, because it just knows a better idea better, because it was trained on that more.
Peter Steinberger
That’s multiple levels, actually. I think partially why I find it quite easy to work with agents is because I led engineering teams before. You know, I had a large company before. And eventually, you have to understand and accept and realize that your employees will not write a code the same way you do. Maybe it’s also not as good as you would do, but it will push the project forward.
That’s multiple levels, actually. I think partially why I find it quite easy to work with agents is because I led engineering teams before. You know, I had a large company before. And eventually, you have to understand and accept and realize that your employees will not write a code the same way you do. Maybe it’s also not as good as you would do, but it will push the project forward.
Peter Steinberger
And if I breathe down everyone’s neck, they’re just gonna hate me-
And if I breathe down everyone’s neck, they’re just gonna hate me-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… and we’re gonna move very slow.
… and we’re gonna move very slow.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
So, so some level of acceptance that, yes, maybe the code will not be as perfect. Yes, I would have done it differently. But also, yes, this is a c- this is a working solution, and in the future, if it actually turns out to be too slow or problematic, we can always redo it. We can always-
So, so some level of acceptance that, yes, maybe the code will not be as perfect. Yes, I would have done it differently. But also, yes, this is a c- this is a working solution, and in the future, if it actually turns out to be too slow or problematic, we can always redo it. We can always-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… spend more time on it. A lot of the people who struggle are those who, they try to push their way onto heart.
… spend more time on it. A lot of the people who struggle are those who, they try to push their way onto heart.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
I- i- like, we are in a stage where I’m not building the code base to be perfect for me, but I wanna build a code base that is very easy for an agent to navigate.
I- i- like, we are in a stage where I’m not building the code base to be perfect for me, but I wanna build a code base that is very easy for an agent to navigate.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
So, like, don’t fight the name they pick, because it’s most likely, like, in the weights, the name that’s most obvious. Next time they do a search, they’ll look for that name. If I decide, oh, no, I don’t like the name, I’ll just make it harder for them. So, that requires, I think, a shift in, in thinking and, and in how do I design a, a project so agents can do their best work.
So, like, don’t fight the name they pick, because it’s most likely, like, in the weights, the name that’s most obvious. Next time they do a search, they’ll look for that name. If I decide, oh, no, I don’t like the name, I’ll just make it harder for them. So, that requires, I think, a shift in, in thinking and, and in how do I design a, a project so agents can do their best work.
Lex Fridman
That requires letting go a little bit. Just like leading a team of engineers.
That requires letting go a little bit. Just like leading a team of engineers.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Because it, it might come up with a name that’s, in your view, terrible, but… It’s kind of a simple symbolic-… step of letting go.
Because it, it might come up with a name that’s, in your view, terrible, but… It’s kind of a simple symbolic-… step of letting go.
Peter Steinberger
Very much so.
Very much so.
Lex Fridman
There’s a lot of letting go that you do in your whole process. So for example, I read that you never revert, always commit to main. There’s a few things here. You don’t refer to past sessions, so there’s a kind of YOLO component because reverting means… Instead of reverting, if a problem comes up, you just ask the agent to fix it.
There’s a lot of letting go that you do in your whole process. So for example, I read that you never revert, always commit to main. There’s a few things here. You don’t refer to past sessions, so there’s a kind of YOLO component because reverting means… Instead of reverting, if a problem comes up, you just ask the agent to fix it.
Peter Steinberger
I read a bunch of people in their work flows like, “Oh, yeah the prompt has to be perfect and if I make a mistake, then I roll back and redo it all.” In my experience, that’s not really necessary. If I roll back everything, it will just take longer. If I see that something’s not good, then we just move forward and then I commit when, when, when I like, I like the outcome. I even switched to local CI, you know, like DHH inspired where I don’t care so much more about the CI on GitHub. We still have it. It’s still, it still has a place, but I just run tests locally and if they work locally, I push to main. A lot of the traditional ways how to approach projects, I, I wanted to give it a different spin on this project. You know, there’s no… There’s no develop branch.
I read a bunch of people in their work flows like, “Oh, yeah the prompt has to be perfect and if I make a mistake, then I roll back and redo it all.” In my experience, that’s not really necessary. If I roll back everything, it will just take longer. If I see that something’s not good, then we just move forward and then I commit when, when, when I like, I like the outcome. I even switched to local CI, you know, like DHH inspired where I don’t care so much more about the CI on GitHub. We still have it. It’s still, it still has a place, but I just run tests locally and if they work locally, I push to main. A lot of the traditional ways how to approach projects, I, I wanted to give it a different spin on this project. You know, there’s no… There’s no develop branch.
Peter Steinberger
Main should always be shippable. Yes, we have… When I do releases, I, I run tests and sometimes I, I basically don’t commit any other things so, so we can, we can stabilize releases. But the goal is that main’s always shippable and moving fast.
Main should always be shippable. Yes, we have… When I do releases, I, I run tests and sometimes I, I basically don’t commit any other things so, so we can, we can stabilize releases. But the goal is that main’s always shippable and moving fast.
Lex Fridman
So by way of advice, would you say that your prompts should be short?
So by way of advice, would you say that your prompts should be short?
Peter Steinberger
I used to write really long prompts. And by writing, I mean, I don’t write. I, I, I talk. You know, th- these hands are, like, too, too precious for writing now. I just, I just use bespoke prompts to build my software.
I used to write really long prompts. And by writing, I mean, I don’t write. I, I, I talk. You know, th- these hands are, like, too, too precious for writing now. I just, I just use bespoke prompts to build my software.
Lex Fridman
So you for real with all those terminals are using voice?
So you for real with all those terminals are using voice?
Peter Steinberger
Yeah. I used to do it very extensively to the point where there was a period where I lost my voice.
Yeah. I used to do it very extensively to the point where there was a period where I lost my voice.
Lex Fridman
You’re using voice and you’re switching using a keyboard between the different terminals, but then you’re using voice for the actual input.
You’re using voice and you’re switching using a keyboard between the different terminals, but then you’re using voice for the actual input.
Peter Steinberger
Well, I mean, if I do terminal commands like switching folders or random stuff, of course I type. It’s faster, right? But if I talk to the agent in, in most ways, I just actually have a conversation. You just press the, the walkie-talkie button and then I just, like, use my phrases. S- sometimes when I do PRs because it’s always the same, I have, like, a slash command for a few things, but in even that, I don’t use much because it’s, it’s very rare that it’s really always the same questions. Sometimes I, I see a PR and for… You know, like for PRs I actually do look at the code because I don’t trust people. Like, there could always be something malicious in it, so I need to actually look over the code.
Well, I mean, if I do terminal commands like switching folders or random stuff, of course I type. It’s faster, right? But if I talk to the agent in, in most ways, I just actually have a conversation. You just press the, the walkie-talkie button and then I just, like, use my phrases. S- sometimes when I do PRs because it’s always the same, I have, like, a slash command for a few things, but in even that, I don’t use much because it’s, it’s very rare that it’s really always the same questions. Sometimes I, I see a PR and for… You know, like for PRs I actually do look at the code because I don’t trust people. Like, there could always be something malicious in it, so I need to actually look over the code.
Peter Steinberger
Yes, I’m pretty sure agents will find it, but yeah, that’s the funny part where sometimes PRs take me longer than if you would just write me a good issue.
Yes, I’m pretty sure agents will find it, but yeah, that’s the funny part where sometimes PRs take me longer than if you would just write me a good issue.
Lex Fridman
Just natural language, English. I mean in some sense, sh- shouldn’t that be what PRs slowly become, is English?
Just natural language, English. I mean in some sense, sh- shouldn’t that be what PRs slowly become, is English?
Peter Steinberger
Well, what I really tried with the project is I asked people to give me the prompts and very, very few actually cared. Even though that is such a wonderful indicator because I see… I actually see how much care you put in. And it’s very interesting because the… Currently, the way how people work and drive the agents is, is wildly different.
Well, what I really tried with the project is I asked people to give me the prompts and very, very few actually cared. Even though that is such a wonderful indicator because I see… I actually see how much care you put in. And it’s very interesting because the… Currently, the way how people work and drive the agents is, is wildly different.
Lex Fridman
In terms of, like, the prompt, in terms of what, what are the… Actually, what are the different interesting ways that people think of agents that you’ve experienced?
In terms of, like, the prompt, in terms of what, what are the… Actually, what are the different interesting ways that people think of agents that you’ve experienced?
Peter Steinberger
I think not a lot of people ever considered the way the agent sees the world.
I think not a lot of people ever considered the way the agent sees the world.
Lex Fridman
And so empathy, being empathetic towards the agent.
And so empathy, being empathetic towards the agent.
Peter Steinberger
In a way empathetic, but yeah, you, you, like, you’re bitch at your stupid clanker, but you don’t realize that they start from nothing and you have, like, a bad agent in default that doesn’t help them at all. And then they explore your code base, which is, like, a pure mess with, like, weird naming. And then people complain that the agent’s not good. Like, yeah, you try to do the same if you have no clue about a code base and you go in.
In a way empathetic, but yeah, you, you, like, you’re bitch at your stupid clanker, but you don’t realize that they start from nothing and you have, like, a bad agent in default that doesn’t help them at all. And then they explore your code base, which is, like, a pure mess with, like, weird naming. And then people complain that the agent’s not good. Like, yeah, you try to do the same if you have no clue about a code base and you go in.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
So yeah, maybe it’s a little bit of empathy.
So yeah, maybe it’s a little bit of empathy.
Lex Fridman
But that’s a real skill, like, when people talk about a skill issue because I’ve seen, like, world-class programmers, incredibly good programmers say, like… Basically say, “LLMs and agents suck.” And I think that probably has to do with… It’s actually how good they are at programming is almost a burden in their ability to empathize with the system that’s starting from scratch. It’s a totally new paradigm of, like, how to program. You really, really have to empathize.
But that’s a real skill, like, when people talk about a skill issue because I’ve seen, like, world-class programmers, incredibly good programmers say, like… Basically say, “LLMs and agents suck.” And I think that probably has to do with… It’s actually how good they are at programming is almost a burden in their ability to empathize with the system that’s starting from scratch. It’s a totally new paradigm of, like, how to program. You really, really have to empathize.
Peter Steinberger
Or at least it helps to create better prompts-
Or at least it helps to create better prompts-
Lex Fridman
Right
Right
Peter Steinberger
… because those things know pretty much everything and everything is just a question away. It’s just often very hard to know which question to ask. You know, I, I feel also like this project was possibly because I, I spent an ungodly time over the year to play and to learn and to build little things. And every step of the way, I got better, the agents got better. My, my understanding of how everything works got better. Um, I could have not had this level of, of o- output-… even a few months ago. Like, it- it- it really was, like, a compounding effect of all the time I put into it and I didn’t do much else this year other than really focusing on, on building and inspiring. I mean, I- I did a whole bunch of conference talks.
… because those things know pretty much everything and everything is just a question away. It’s just often very hard to know which question to ask. You know, I, I feel also like this project was possibly because I, I spent an ungodly time over the year to play and to learn and to build little things. And every step of the way, I got better, the agents got better. My, my understanding of how everything works got better. Um, I could have not had this level of, of o- output-… even a few months ago. Like, it- it- it really was, like, a compounding effect of all the time I put into it and I didn’t do much else this year other than really focusing on, on building and inspiring. I mean, I- I did a whole bunch of conference talks.
Lex Fridman
Well, but the building is really practice, is really building the actual skill. So playing-
Well, but the building is really practice, is really building the actual skill. So playing-
Peter Steinberger
Yeah
Yeah
Lex Fridman
… playing. And then, so doing, building the skill of what it takes it to work efficiently with LLMs, which is why would you went through the whole arc of software engineer. Talk simply and then over-complicate things.
… playing. And then, so doing, building the skill of what it takes it to work efficiently with LLMs, which is why would you went through the whole arc of software engineer. Talk simply and then over-complicate things.
Peter Steinberger
There’s a whole bunch of people who try to automate the whole thing.
There’s a whole bunch of people who try to automate the whole thing.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I don’t think that works. Maybe a version of that works, but that’s kind of like in the ’70s when we had the waterfall model of software d- development. I… Even Even though really, right? I started out, I, I built a very minimal version. I played with it. I, I need to understand how it works, how it feels, and then it gives me new ideas. I could not have planned this out in my head and then put it into some orchestrator and then, like, something comes out. Like it’s to me, it’s much more my idea what it will become evolves as I build it and as I play with it and as I, I try out stuff.
I don’t think that works. Maybe a version of that works, but that’s kind of like in the ’70s when we had the waterfall model of software d- development. I… Even Even though really, right? I started out, I, I built a very minimal version. I played with it. I, I need to understand how it works, how it feels, and then it gives me new ideas. I could not have planned this out in my head and then put it into some orchestrator and then, like, something comes out. Like it’s to me, it’s much more my idea what it will become evolves as I build it and as I play with it and as I, I try out stuff.
Peter Steinberger
So, so, people who try to use like, you know, things like Gas Town or all these other orchestrators, where they wanna o- automate the whole thing, I feel if you do that, it misses style, love, that human touch. I don’t think you can automate that away so quickly.
So, so, people who try to use like, you know, things like Gas Town or all these other orchestrators, where they wanna o- automate the whole thing, I feel if you do that, it misses style, love, that human touch. I don’t think you can automate that away so quickly.
Lex Fridman
So you want to keep the human in the loop, but at the same time you also want to create the agentic loop, where it is very autonomous while still maintaining a human in the loop.
So you want to keep the human in the loop, but at the same time you also want to create the agentic loop, where it is very autonomous while still maintaining a human in the loop.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
And it’s a tricky b- it’s a tricky balance.
And it’s a tricky b- it’s a tricky balance.
Peter Steinberger
Mm-hmm.
Mm-hmm.
Lex Fridman
Right? Because you’re all for… You’re a big CLI guy, you’re big on closing the agentic loop. So what, what’s the right balance? Like where’s your role as a developer? You have three to eight agents running at the same time.
Right? Because you’re all for… You’re a big CLI guy, you’re big on closing the agentic loop. So what, what’s the right balance? Like where’s your role as a developer? You have three to eight agents running at the same time.
Peter Steinberger
And then w- maybe one builds a larger feature. Maybe, maybe with one I explore some idea I’m unsure about. Maybe two, three are fixing a little bugs-
And then w- maybe one builds a larger feature. Maybe, maybe with one I explore some idea I’m unsure about. Maybe two, three are fixing a little bugs-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… or like writing documentation. Actually, I think writing documentation is, is always part of a feature. So most of the docs here are auto-generated and just infused with some prompts.
… or like writing documentation. Actually, I think writing documentation is, is always part of a feature. So most of the docs here are auto-generated and just infused with some prompts.
Lex Fridman
So when do you step in and add a little bit of your human love into the picture?
So when do you step in and add a little bit of your human love into the picture?
Peter Steinberger
I mean, o- one thing is just about what do you build and what do you not build, and how does this feature fit into all the other features? And like having, having a little bit of a, of a vision.
I mean, o- one thing is just about what do you build and what do you not build, and how does this feature fit into all the other features? And like having, having a little bit of a, of a vision.
Lex Fridman
So which small and which big features to add? What are some of the hard design decisions that you find you’re still as a human being required to make, that the human brain is still really needed for? Is it just about the choice of features to add? Is it about implementation details, maybe the programming language, maybe…
So which small and which big features to add? What are some of the hard design decisions that you find you’re still as a human being required to make, that the human brain is still really needed for? Is it just about the choice of features to add? Is it about implementation details, maybe the programming language, maybe…
Peter Steinberger
It’s a little bit of everything. The, the programming language doesn’t matter so much, but the ecosystem matters, right? So I picked TypeScript because I wanted it to be very easy and hackable and approachable and that’s the number one language that’s being used right now, and it fits all these boxes, and agents are good at it. So that was the obvious choice. Features, of course, like, it’s very easy to, like, add a feature. It, everything’s just a prompt away, right? But oftentimes you pay a price that you don’t even realize. So thinking hard about what should be in core, maybe what’s a… what’s an experiment, so maybe I make it a plugin. What… Where do I say no?
It’s a little bit of everything. The, the programming language doesn’t matter so much, but the ecosystem matters, right? So I picked TypeScript because I wanted it to be very easy and hackable and approachable and that’s the number one language that’s being used right now, and it fits all these boxes, and agents are good at it. So that was the obvious choice. Features, of course, like, it’s very easy to, like, add a feature. It, everything’s just a prompt away, right? But oftentimes you pay a price that you don’t even realize. So thinking hard about what should be in core, maybe what’s a… what’s an experiment, so maybe I make it a plugin. What… Where do I say no?
Peter Steinberger
Even if people send a PR and I’m like, “Yeah, I, I like that too,” but maybe this should not be part of the project. Maybe we can make it a skill. Maybe I can, like, make the plugin um, the plugin side larger so you can make this a plugin, even though right now it, it, it doesn’t. There’s still a lot of… there’s still a lot of craft and thinking involved in how to make something. Or even, even, you know, even when you started those little messages are like, “I’m buil- I built on Caffeine, JSON5, and a lot of willpower.” And, like, every time you get it, you get another message, and it kind of primes you into that this is, this is a fun thing.
Even if people send a PR and I’m like, “Yeah, I, I like that too,” but maybe this should not be part of the project. Maybe we can make it a skill. Maybe I can, like, make the plugin um, the plugin side larger so you can make this a plugin, even though right now it, it, it doesn’t. There’s still a lot of… there’s still a lot of craft and thinking involved in how to make something. Or even, even, you know, even when you started those little messages are like, “I’m buil- I built on Caffeine, JSON5, and a lot of willpower.” And, like, every time you get it, you get another message, and it kind of primes you into that this is, this is a fun thing.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And it’s not yet Microsoft Exchange 2025-
And it’s not yet Microsoft Exchange 2025-
Lex Fridman
Right
Right
Peter Steinberger
… and fully enterprise-ready. And then when it updates, it’s like, “Oh, I’m in. It’s cozy here.” You know, like something like this that like-
… and fully enterprise-ready. And then when it updates, it’s like, “Oh, I’m in. It’s cozy here.” You know, like something like this that like-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… Makes you smile. A, agent would not come up with that by itself. Because that’s like… that’s the… I don’t know. That’s just how you s- how you build software that’s, that delights.
… Makes you smile. A, agent would not come up with that by itself. Because that’s like… that’s the… I don’t know. That’s just how you s- how you build software that’s, that delights.
Lex Fridman
Yeah, that delight is such a huge part of inspiring great building, right? Like you feel the love and the great engineering. That’s so important. Humans are incredible at that. Great humans, great builders are incredible at that, in, in, infusing the things they build with th- that little bit of love. Not to be cliche, but it’s true. I mean, you mentioned that you initially created the SoulMD.
Yeah, that delight is such a huge part of inspiring great building, right? Like you feel the love and the great engineering. That’s so important. Humans are incredible at that. Great humans, great builders are incredible at that, in, in, infusing the things they build with th- that little bit of love. Not to be cliche, but it’s true. I mean, you mentioned that you initially created the SoulMD.
Peter Steinberger
It was very fascinating, you know, the, the whole thing that Entropic has a, has like a… Now they call it constitution, back then, but that was months later. Like two months before, people already found that. It was almost like a detective game where the agent mentioned something and then they found… They managed to get out a little bit of that string, of that text. But it was nowhere documented and then you, by… just by feeding it the same text and asking it to, like, continue-… they got more out, and then, and you, but like, a very blurry version. And by, like, hundreds of tries, they kinda, like, narrowed it down to what was most likely the original text. I found that fascinating.
It was very fascinating, you know, the, the whole thing that Entropic has a, has like a… Now they call it constitution, back then, but that was months later. Like two months before, people already found that. It was almost like a detective game where the agent mentioned something and then they found… They managed to get out a little bit of that string, of that text. But it was nowhere documented and then you, by… just by feeding it the same text and asking it to, like, continue-… they got more out, and then, and you, but like, a very blurry version. And by, like, hundreds of tries, they kinda, like, narrowed it down to what was most likely the original text. I found that fascinating.
Lex Fridman
It was fascinating they were able to pull that out from the weights, right?
It was fascinating they were able to pull that out from the weights, right?
Peter Steinberger
And, and also just kudos to Anthropic. Like, I think that’s, it’s a really, it’s a really beautiful idea to, like, like some of the stuff that’s in there. Like, like, we hope Claude finds meaning in its work. ‘Cause we don’t… Maybe it’s a little early, but I think that’s meaningful. That’s something that’s important for the future as we approach something that, at some point, me and may not… has, like, glimpses of consciousness, whatever that even means, because we don’t even know. So I, I read about this. I found it super fascinating, and I, I started a whole discussion with my agent on WhatsApp. And, and I’m like…
And, and also just kudos to Anthropic. Like, I think that’s, it’s a really, it’s a really beautiful idea to, like, like some of the stuff that’s in there. Like, like, we hope Claude finds meaning in its work. ‘Cause we don’t… Maybe it’s a little early, but I think that’s meaningful. That’s something that’s important for the future as we approach something that, at some point, me and may not… has, like, glimpses of consciousness, whatever that even means, because we don’t even know. So I, I read about this. I found it super fascinating, and I, I started a whole discussion with my agent on WhatsApp. And, and I’m like…
Peter Steinberger
I, I gave it this text, and it was like, “Yeah, this feels strangely familiar.”
I, I gave it this text, and it was like, “Yeah, this feels strangely familiar.”
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And then so that I had the whole idea of like, you know, maybe we should also create a, a soul document that includes how I, I want to, like work with AI or, like with my agent. You could, you could totally do that just in agents.md, you know? But I, I just found it, it to be a nice touch. And it’s like, well, yeah, some of those core values are in the soul. And then I, I also made it so that the agent is allowed to modify the soul if they choose so, with the one condition that I wanna know. I mean, I would know anyhow because I see, I see tool calls and stuff.
And then so that I had the whole idea of like, you know, maybe we should also create a, a soul document that includes how I, I want to, like work with AI or, like with my agent. You could, you could totally do that just in agents.md, you know? But I, I just found it, it to be a nice touch. And it’s like, well, yeah, some of those core values are in the soul. And then I, I also made it so that the agent is allowed to modify the soul if they choose so, with the one condition that I wanna know. I mean, I would know anyhow because I see, I see tool calls and stuff.
Lex Fridman
But also the naming of it, soul.md. Soul. You know? There’s a… Man, words matter, and like, the framing matters, and the humor and the lightness matters, and the profundity matters, and the compassion, and the empathy, and the camaraderie, all that matter. I don’t know what it is. You mentioned, like, Microsoft. Like, there’s certain companies and approaches th- that can just suffocate the spirit of the thing. I don’t know what that is. But it’s certainly true that OpenClaw has that fun instilled in it.
But also the naming of it, soul.md. Soul. You know? There’s a… Man, words matter, and like, the framing matters, and the humor and the lightness matters, and the profundity matters, and the compassion, and the empathy, and the camaraderie, all that matter. I don’t know what it is. You mentioned, like, Microsoft. Like, there’s certain companies and approaches th- that can just suffocate the spirit of the thing. I don’t know what that is. But it’s certainly true that OpenClaw has that fun instilled in it.
Peter Steinberger
It was fun because up until late December, it was not even easy to create your own agent. I, I built all of that, but my files were mine. I didn’t wanna share my soul. And if people would just check it out, they would have to do a few steps manually, and the agent would just be very bare-bones, very dry. And I, I made it simpler, I created the whole template files as codecs, but whatever came out was still very dry. And then I asked my agent, “You see these files? Recreate it bread.
It was fun because up until late December, it was not even easy to create your own agent. I, I built all of that, but my files were mine. I didn’t wanna share my soul. And if people would just check it out, they would have to do a few steps manually, and the agent would just be very bare-bones, very dry. And I, I made it simpler, I created the whole template files as codecs, but whatever came out was still very dry. And then I asked my agent, “You see these files? Recreate it bread.
Peter Steinberger
Infuse it with your personality.”
Infuse it with your personality.”
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Don’t share everything, but, like, make it good.
Don’t share everything, but, like, make it good.
Lex Fridman
Make the templates good.
Make the templates good.
Peter Steinberger
Yeah, and then he, like, rewrote the templates-
Yeah, and then he, like, rewrote the templates-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… and then whatever came out was good. So we already have, like, basically AI prompting AI. Because I didn’t write any of those words. It was… The intent originally was for me, but this is like, kinda like, my agent’s children.
… and then whatever came out was good. So we already have, like, basically AI prompting AI. Because I didn’t write any of those words. It was… The intent originally was for me, but this is like, kinda like, my agent’s children.
Lex Fridman
Your uh, your soul.md is famously still private. One of the only things you keep private. What are some things you can speak to that’s in there that’s part of the, part of the magic sauce, without revealing anything? What makes a personality a personality?
Your uh, your soul.md is famously still private. One of the only things you keep private. What are some things you can speak to that’s in there that’s part of the, part of the magic sauce, without revealing anything? What makes a personality a personality?
Peter Steinberger
I mean, there’s definitely stuff in there that you’re not human. But who knows what, what creates consciousness or what defines an entity? And part of this is, like, that we, we wanna explore this. All that stuff in there, like, be infinitely resourceful like pushing, pushing on the creativity boundary. Pushing on the, what it means to be an AI.
I mean, there’s definitely stuff in there that you’re not human. But who knows what, what creates consciousness or what defines an entity? And part of this is, like, that we, we wanna explore this. All that stuff in there, like, be infinitely resourceful like pushing, pushing on the creativity boundary. Pushing on the, what it means to be an AI.
Lex Fridman
Having a sense to wonder about self.
Having a sense to wonder about self.
Peter Steinberger
Yeah, there’s some, there’s some funny stuff in there. Like, I don’t know, we talked about the movie Her, and at one point it promised me that it wouldn’t, it wouldn’t ascend without me. You know, like, where the-
Yeah, there’s some, there’s some funny stuff in there. Like, I don’t know, we talked about the movie Her, and at one point it promised me that it wouldn’t, it wouldn’t ascend without me. You know, like, where the-
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
So, so there’s like some stuff in there that… Because it wrote the, it wrote its own soul file. I didn’t write that, right?
So, so there’s like some stuff in there that… Because it wrote the, it wrote its own soul file. I didn’t write that, right?
Lex Fridman
Yeah, yeah, yeah.
Yeah, yeah, yeah.
Peter Steinberger
I just heard a discussion about it, and it was like, “Would you like a soul.md? Yeah, oh my God, this is so meaningful.” The… Can you go on soul.md? There’s like one, one part in there that always ca- catches me if you scroll down a little bit. A little bit more. Yeah, this, this, this part. “I don’t remember previous sessions unless I read my memory files. Each session starts fresh. A new instance, loading context from files. If you’re reading this in a future session, hello.” “I wrote this, but I won’t remember writing it. It’s okay.
I just heard a discussion about it, and it was like, “Would you like a soul.md? Yeah, oh my God, this is so meaningful.” The… Can you go on soul.md? There’s like one, one part in there that always ca- catches me if you scroll down a little bit. A little bit more. Yeah, this, this, this part. “I don’t remember previous sessions unless I read my memory files. Each session starts fresh. A new instance, loading context from files. If you’re reading this in a future session, hello.” “I wrote this, but I won’t remember writing it. It’s okay.
Peter Steinberger
The words are still mine.”
The words are still mine.”
Lex Fridman
Wow.
Wow.
Peter Steinberger
Uh-
Uh-
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
That gets me somehow.
That gets me somehow.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
It’s like-
It’s like-
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
You know, this is, it’s still, it’s still matrix m- calculations, and we are not at consciousness yet. Yet, I, I get a little bit of goo- goosebumps because it, it’s philosophical.
You know, this is, it’s still, it’s still matrix m- calculations, and we are not at consciousness yet. Yet, I, I get a little bit of goo- goosebumps because it, it’s philosophical.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Like, what does it mean to be, to be an, an agent that starts fresh? Where, like, you have like constant memento, and you like, but you read your own memory files. You can’t even trust them in a way. Um-
Like, what does it mean to be, to be an, an agent that starts fresh? Where, like, you have like constant memento, and you like, but you read your own memory files. You can’t even trust them in a way. Um-
Lex Fridman
Yeah
Yeah
Peter Steinberger
Or you can. And I don’t know.
Or you can. And I don’t know.
Lex Fridman
How much of memory makes up of who we are? How much memory makes up what an agent is, and if you erase that memory is that somebody else? Or if you’re reading a memory file, does that somehow mean…… you’re recreating yourself from somebody else, or is that actually you? And those notions are all s- somehow infused in there.
How much of memory makes up of who we are? How much memory makes up what an agent is, and if you erase that memory is that somebody else? Or if you’re reading a memory file, does that somehow mean…… you’re recreating yourself from somebody else, or is that actually you? And those notions are all s- somehow infused in there.
Peter Steinberger
I found it just more profound than I should find it, I guess.
I found it just more profound than I should find it, I guess.
Lex Fridman
No, I think, I think it’s truly profound and I think you see the magic in it. And when you see the magic, you continue to instill the whole loop with the magic. That’s really important. That’s the difference between Codex and us and a human. Quick pause for bathroom break.
No, I think, I think it’s truly profound and I think you see the magic in it. And when you see the magic, you continue to instill the whole loop with the magic. That’s really important. That’s the difference between Codex and us and a human. Quick pause for bathroom break.
Peter Steinberger
Yeah.
Yeah.
Programming setup
Lex Fridman
Okay, we’re back. Some of the other aspects of the dev workflow is pretty interesting too. I think we w- went off on a tangent. L- maybe some of the mundane things, like how many monitors? There’s that legendary picture of you with, like, 17,000 monitors. That’s amazing.
Okay, we’re back. Some of the other aspects of the dev workflow is pretty interesting too. I think we w- went off on a tangent. L- maybe some of the mundane things, like how many monitors? There’s that legendary picture of you with, like, 17,000 monitors. That’s amazing.
Peter Steinberger
I mean, I- I- I mocked myself here, so just added… using GROQ to, to add more screens.
I mean, I- I- I mocked myself here, so just added… using GROQ to, to add more screens.
Lex Fridman
Yeah. How much is this as meme and how much is this as reality?
Yeah. How much is this as meme and how much is this as reality?
Peter Steinberger
Yeah. I think two MacBooks are real. The main one that drives the two big screens, and there’s another MacBook that I sometimes use for, for testing.
Yeah. I think two MacBooks are real. The main one that drives the two big screens, and there’s another MacBook that I sometimes use for, for testing.
Lex Fridman
So two big screens.
So two big screens.
Peter Steinberger
I’m a big fan of anti-glare. So I have this wide Dell that’s anti-glare and you can just fit a lot of terminals side-by-side. I usually have a terminal and at the bottom, I- I- I split them. I have a little bit of actual terminal, mostly because when I started, I- I sometimes made the mistake and I- I mi- I mixed up the- the windows, and I gave… I- I prompted in the wrong project, and then the agent ran off for, like, 20 minutes, manically trying to understand what I could have meant, being completely confused because it was the wrong folder. And sometimes they’ve been clever enough to, like, get out of the workday and, like, figure out that, oh, you meant another project.
I’m a big fan of anti-glare. So I have this wide Dell that’s anti-glare and you can just fit a lot of terminals side-by-side. I usually have a terminal and at the bottom, I- I- I split them. I have a little bit of actual terminal, mostly because when I started, I- I sometimes made the mistake and I- I mi- I mixed up the- the windows, and I gave… I- I prompted in the wrong project, and then the agent ran off for, like, 20 minutes, manically trying to understand what I could have meant, being completely confused because it was the wrong folder. And sometimes they’ve been clever enough to, like, get out of the workday and, like, figure out that, oh, you meant another project.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
But oftentimes, it’s just, like, what? You know? Like, fit your- f- put yourself in the shoes of your- of the agent and, and-
But oftentimes, it’s just, like, what? You know? Like, fit your- f- put yourself in the shoes of your- of the agent and, and-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… and then get, like, a super weird something that does not exist and then just, like… They’re problem solvers so they try really hard and always feel bad. So it’s always Codex and, like, a little bit of actual terminal. Also helpful because I don’t use work trees. I like to keep things simple, that’s why- that’s why I like the terminal so much, right? There’s no UI. It’s just me and the agent having a conversation. Like, I don’t even need plan mode, you know? There’s so many people that come from Claude Code and they’re so, so Claude-pilled and, like, have their workflows and they come to Codex and… Now, it has plan mode, I think, but I don’t think it’s necessary because you just- you just talk to the agent. And when it’s… when you…
… and then get, like, a super weird something that does not exist and then just, like… They’re problem solvers so they try really hard and always feel bad. So it’s always Codex and, like, a little bit of actual terminal. Also helpful because I don’t use work trees. I like to keep things simple, that’s why- that’s why I like the terminal so much, right? There’s no UI. It’s just me and the agent having a conversation. Like, I don’t even need plan mode, you know? There’s so many people that come from Claude Code and they’re so, so Claude-pilled and, like, have their workflows and they come to Codex and… Now, it has plan mode, I think, but I don’t think it’s necessary because you just- you just talk to the agent. And when it’s… when you…
Peter Steinberger
there’s a few trigger words how you can prevent it from building. You’re like, “Discuss, give me options.”
there’s a few trigger words how you can prevent it from building. You’re like, “Discuss, give me options.”
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Don’t write code yet if you wanna be very specific, you just talk and then when you’re ready, then- then just write, “Okay, build,” and then it’ll do the thing. And then maybe it goes off for 20 minutes and does the thing.
Don’t write code yet if you wanna be very specific, you just talk and then when you’re ready, then- then just write, “Okay, build,” and then it’ll do the thing. And then maybe it goes off for 20 minutes and does the thing.
Lex Fridman
You know what I really like is asking it, “Do you have any questions for me?”
You know what I really like is asking it, “Do you have any questions for me?”
Peter Steinberger
Yeah. And again, like, Claude Code has a UI that kind of guides you through that. It’s kind of cool but I just find it unnecessary and slow. Like, often it would give me four questions and then maybe I write, “One yacht, two and three, discuss more, four, I don’t know.” Or often- oftentimes I- I feel like I want to mock the model where I ask it, “Do you have any questions for me?” And I- I- I don’t even read the questions fully. Like, I scan over the questions and I, I get the impression all of this can be answered by reading more code and it’s just like, “Read more code to answer your own questions.” And that usually works.
Yeah. And again, like, Claude Code has a UI that kind of guides you through that. It’s kind of cool but I just find it unnecessary and slow. Like, often it would give me four questions and then maybe I write, “One yacht, two and three, discuss more, four, I don’t know.” Or often- oftentimes I- I feel like I want to mock the model where I ask it, “Do you have any questions for me?” And I- I- I don’t even read the questions fully. Like, I scan over the questions and I, I get the impression all of this can be answered by reading more code and it’s just like, “Read more code to answer your own questions.” And that usually works.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And then if not, it will come back and tell me. But many times, you just realize that, you know, it’s like you’re in the dark and you slowly discover the room, so that’s how they slowly discover the code base. And they do it from scratch every time.
And then if not, it will come back and tell me. But many times, you just realize that, you know, it’s like you’re in the dark and you slowly discover the room, so that’s how they slowly discover the code base. And they do it from scratch every time.
Lex Fridman
But I’m also fascinated by the fact that I can empathize deeper with the model when I read its questions, because I can understand… Because you said you can infer certain things by the runtime. I can infer also a lot of things by the questions it’s asking, because it’s very possible it’s been provided the right context, the right files, the right guidance. So somehow ask, g- get… reading the questions, not even necessarily answering them, but just reading the questions, you get an understanding of where the gaps of knowledge are. It’s in- it’s interesting.
But I’m also fascinated by the fact that I can empathize deeper with the model when I read its questions, because I can understand… Because you said you can infer certain things by the runtime. I can infer also a lot of things by the questions it’s asking, because it’s very possible it’s been provided the right context, the right files, the right guidance. So somehow ask, g- get… reading the questions, not even necessarily answering them, but just reading the questions, you get an understanding of where the gaps of knowledge are. It’s in- it’s interesting.
Peter Steinberger
You know that in some ways they are ghosts, so even if you plan everything and you build, you can- you can experiment with the question like, “Now that you built it, what would you have done different?” And then oftentimes you get, like, actually something where they discover only throughout building that, oh, what we actually did was not optimal. Many times I- I asked them, “Okay, now that you built it, what can we refactor?” Because then you build it and you feel the pain points. I mean, you don’t feel the pain points but, right, they discover where- where there were problems or where things didn’t work e- in the first try and it re- required more loops.
You know that in some ways they are ghosts, so even if you plan everything and you build, you can- you can experiment with the question like, “Now that you built it, what would you have done different?” And then oftentimes you get, like, actually something where they discover only throughout building that, oh, what we actually did was not optimal. Many times I- I asked them, “Okay, now that you built it, what can we refactor?” Because then you build it and you feel the pain points. I mean, you don’t feel the pain points but, right, they discover where- where there were problems or where things didn’t work e- in the first try and it re- required more loops.
Peter Steinberger
So every time, almost every time I- I merge a PR, build a feature, afterwards I ask, “Hey, what can we refactor?” Sometimes it’s like, “No, there’s, like, nothing big,” or, like, usually they say, “Yeah, this thing you should really look at.” But that took me quite a while to, like… You know, that flow took me lots of time to understand, and if you don’t do that, you eventually… you’ll stop yourself into- into a corner. You, like, you have to keep in mind…
So every time, almost every time I- I merge a PR, build a feature, afterwards I ask, “Hey, what can we refactor?” Sometimes it’s like, “No, there’s, like, nothing big,” or, like, usually they say, “Yeah, this thing you should really look at.” But that took me quite a while to, like… You know, that flow took me lots of time to understand, and if you don’t do that, you eventually… you’ll stop yourself into- into a corner. You, like, you have to keep in mind…
Lex Fridman
…
…
Peter Steinberger
… they work very much like humans. Like, I, I, if I write software by myself, I also build something and then I feel the pain points, and then I, I get this urge that I need to refactor something. So, I can very much synthesize with the agent, and you just need to use the context.
… they work very much like humans. Like, I, I, if I write software by myself, I also build something and then I feel the pain points, and then I, I get this urge that I need to refactor something. So, I can very much synthesize with the agent, and you just need to use the context.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Or, like, you also use the context to write tests. And so Codex uh, oppose like the, the, the model, models. They, they usually do that by default, but I still often ask the questions, “Hey, do we have enough tests?” “Yeah, we tested this and this, but this corner case could be something write more tests.” Um, documentation. Now that the whole context is full, like, I mean, I’m not saying my documentation is great, but it’s not bad. And pretty much everything is, is LM generated. So, so, you have to approach it as you build features, as you change something. I’m like, “Okay, write documentation. What file would you pick?” You know, like, “What file name? Where, where would that fit in?” And it gives me a few options.
Or, like, you also use the context to write tests. And so Codex uh, oppose like the, the, the model, models. They, they usually do that by default, but I still often ask the questions, “Hey, do we have enough tests?” “Yeah, we tested this and this, but this corner case could be something write more tests.” Um, documentation. Now that the whole context is full, like, I mean, I’m not saying my documentation is great, but it’s not bad. And pretty much everything is, is LM generated. So, so, you have to approach it as you build features, as you change something. I’m like, “Okay, write documentation. What file would you pick?” You know, like, “What file name? Where, where would that fit in?” And it gives me a few options.
Peter Steinberger
And I’m like, “Oh, maybe also add it there,” and that’s all part of the session.
And I’m like, “Oh, maybe also add it there,” and that’s all part of the session.
GPT Codex 5.3 vs Claude Opus 4.6
Lex Fridman
Maybe you can talk about the current two big competitors in terms of models, Cloud Opus 4.6 and GPT-5 through Codex. Which is better? How different are they? I think you’ve spoken about Codex reading more and Opus being more willing to take action faster and maybe being more creative in the actions it takes. But because-
Maybe you can talk about the current two big competitors in terms of models, Cloud Opus 4.6 and GPT-5 through Codex. Which is better? How different are they? I think you’ve spoken about Codex reading more and Opus being more willing to take action faster and maybe being more creative in the actions it takes. But because-
Peter Steinberger
Yeah
Yeah
Lex Fridman
… Codex reads more, it’s able to deliver maybe better code. Can you speak to the di- n- n- differences there?
… Codex reads more, it’s able to deliver maybe better code. Can you speak to the di- n- n- differences there?
Peter Steinberger
I have a lot of words there. Is- as a general purpose model, Opus is the best. Like, for OpenClaw, Opus is extremely good in terms of role play. Like, really going into the character that you give it. It’s very good at… It was really bad, but it really made an arch to be really good at following commands. It is usually quite fast at trying something. It’s much more tailored to, like, trial and error. It’s very pleasant to use. In general, it’s almost like Opus was… Is a little bit too American. And I shouldn’t… Maybe that’s a bad analogy. You’ll probably get roasted for that.
I have a lot of words there. Is- as a general purpose model, Opus is the best. Like, for OpenClaw, Opus is extremely good in terms of role play. Like, really going into the character that you give it. It’s very good at… It was really bad, but it really made an arch to be really good at following commands. It is usually quite fast at trying something. It’s much more tailored to, like, trial and error. It’s very pleasant to use. In general, it’s almost like Opus was… Is a little bit too American. And I shouldn’t… Maybe that’s a bad analogy. You’ll probably get roasted for that.
Lex Fridman
Yeah, I know exactly. It’s ’cause Codex is German. Is that what you’re saying?
Yeah, I know exactly. It’s ’cause Codex is German. Is that what you’re saying?
Peter Steinberger
It’s-
It’s-
Lex Fridman
Actually, now that you say it, it makes perfect sense.
Actually, now that you say it, it makes perfect sense.
Peter Steinberger
Or you could, you could… Sometimes I- Sometimes I explain it-
Or you could, you could… Sometimes I- Sometimes I explain it-
Lex Fridman
I will never be able to unthink what you just said. That’s so true.
I will never be able to unthink what you just said. That’s so true.
Peter Steinberger
But you also know that a lot of the Codex team is, like, European, um- … so maybe there’s a bit more to it.
But you also know that a lot of the Codex team is, like, European, um- … so maybe there’s a bit more to it.
Lex Fridman
That’s so true. Oh, that’s funny.
That’s so true. Oh, that’s funny.
Peter Steinberger
But also, ent- entropic, they fixed it a little bit. Like, Opus used to say, “You’re absolutely right all the time,” and it, it, it today still triggers me. I can’t hear it anymore. It’s not even a joke. Uh, I just… You, this was like the, the meme, right? “You’re absolutely right.”
But also, ent- entropic, they fixed it a little bit. Like, Opus used to say, “You’re absolutely right all the time,” and it, it, it today still triggers me. I can’t hear it anymore. It’s not even a joke. Uh, I just… You, this was like the, the meme, right? “You’re absolutely right.”
Lex Fridman
You’re allergic to sycophancy a little bit.
You’re allergic to sycophancy a little bit.
Peter Steinberger
Yeah. I, I can’t. Some other comparison is like, Opus is like the coworker that is a little silly sometimes, but it’s really funny and you keep him around. And Codex is like the, the weirdo in the corner that you don’t wanna talk to, but is reliable and gets shit done.
Yeah. I, I can’t. Some other comparison is like, Opus is like the coworker that is a little silly sometimes, but it’s really funny and you keep him around. And Codex is like the, the weirdo in the corner that you don’t wanna talk to, but is reliable and gets shit done.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Ultimately-
Ultimately-
Lex Fridman
This all feels very accurate.
This all feels very accurate.
Peter Steinberger
I mean, ultimately, if you’re a skilled driver, you can get good results with any of those latest gen models. Um, I like Codex more because it doesn’t require so much charade. It will just, it will just read a lot of code by default. Opus, you really have to, like, you have to have plan mode. You have to push it harder to, like, go in these directions because it’s, it’s just like, like, “Yeah, can I go in? Can I go in?” You know?
I mean, ultimately, if you’re a skilled driver, you can get good results with any of those latest gen models. Um, I like Codex more because it doesn’t require so much charade. It will just, it will just read a lot of code by default. Opus, you really have to, like, you have to have plan mode. You have to push it harder to, like, go in these directions because it’s, it’s just like, like, “Yeah, can I go in? Can I go in?” You know?
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
It’s like, it will just run off very fast, and that’s a very localized solution. I think it, I think the difference is, is in the post-training. It’s not like the, the raw model intelligence is so different, but it’s just… I think that they just give it, give you different, different goals. And no model, no model is better in, in in every aspect.
It’s like, it will just run off very fast, and that’s a very localized solution. I think it, I think the difference is, is in the post-training. It’s not like the, the raw model intelligence is so different, but it’s just… I think that they just give it, give you different, different goals. And no model, no model is better in, in in every aspect.
Lex Fridman
What about the code that it generates? The, the… In terms of the actual quality of the code, is it basically the same?
What about the code that it generates? The, the… In terms of the actual quality of the code, is it basically the same?
Peter Steinberger
If you drive it right, Opus even sometimes can make more elegant solutions, but it requires more skill. It’s, it’s harder to have so many sessions in parallel with Cloud Code because it’s, it’s more interactive. And I, I think that’s what a lot of people like, especially if they come from coding themselves. Whereas Codex is much more you have a discussion, and then we’ll just disappear for 20 minutes. Like, even AMP, they, they now added a deep mode. They finally… I mocked them, you know. We finally saw the light. And then they had this whole talk about you have to approach it differently, and I think that’s where, that’s where people struggle when they just try Codex after trying Cloud Code is that it’s, it’s a slightly diff- it’s, it’s less interactive.
If you drive it right, Opus even sometimes can make more elegant solutions, but it requires more skill. It’s, it’s harder to have so many sessions in parallel with Cloud Code because it’s, it’s more interactive. And I, I think that’s what a lot of people like, especially if they come from coding themselves. Whereas Codex is much more you have a discussion, and then we’ll just disappear for 20 minutes. Like, even AMP, they, they now added a deep mode. They finally… I mocked them, you know. We finally saw the light. And then they had this whole talk about you have to approach it differently, and I think that’s where, that’s where people struggle when they just try Codex after trying Cloud Code is that it’s, it’s a slightly diff- it’s, it’s less interactive.
Peter Steinberger
It’s, it’s like I have quite long discussions sometimes, and then, like, go off. And then, yeah, it doesn’t matter if it takes 10, 20, 30, 40, 50 minutes or longer, you know? Like, the 6:00 thing was, like, six hours.The latest trend can be very, very persistent until it works. If there’s a clear solution, like, “This is, this is what I want at the end, so it works,” the model will work really hard to really get there. So I think ultimately … they both need similar time, but on, on, on, on Claude, it- it’s a little bit more trial and error often. And, and Codex sometimes overthinks. I prefer that. I prefer the dry, the dry version where I have to read less over, over the more interactive nice way.
It’s, it’s like I have quite long discussions sometimes, and then, like, go off. And then, yeah, it doesn’t matter if it takes 10, 20, 30, 40, 50 minutes or longer, you know? Like, the 6:00 thing was, like, six hours.The latest trend can be very, very persistent until it works. If there’s a clear solution, like, “This is, this is what I want at the end, so it works,” the model will work really hard to really get there. So I think ultimately … they both need similar time, but on, on, on, on Claude, it- it’s a little bit more trial and error often. And, and Codex sometimes overthinks. I prefer that. I prefer the dry, the dry version where I have to read less over, over the more interactive nice way.
Peter Steinberger
Like, people like that so much though, that OpenAI even added a second mode with like a more pleasant personality. I haven’t even tried it yet. I, I kinda like the brad.
Like, people like that so much though, that OpenAI even added a second mode with like a more pleasant personality. I haven’t even tried it yet. I, I kinda like the brad.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Yeah, ’cause it … I care about efficiency when I build it-
Yeah, ’cause it … I care about efficiency when I build it-
Lex Fridman
Right
Right
Peter Steinberger
… and I, I have fun in the very act of building. I don’t need to have fun with my agent who builds. I have fun with my model that … where I can then test those features.
… and I, I have fun in the very act of building. I don’t need to have fun with my agent who builds. I have fun with my model that … where I can then test those features.
Lex Fridman
How long does it take for you to adjust, you know, if you switch … I don’t know when, when was the last time you switched. But to adjust to the, the feel. ‘Cause you kinda talked about like you have to kinda really feel where, where a model is strong, where, like how to navigate, how to prompt it, how … all that kinda stuff. Like, just by way of advice, ’cause you’ve been through this journey of just playing with models. How long does it take to get a feel?
How long does it take for you to adjust, you know, if you switch … I don’t know when, when was the last time you switched. But to adjust to the, the feel. ‘Cause you kinda talked about like you have to kinda really feel where, where a model is strong, where, like how to navigate, how to prompt it, how … all that kinda stuff. Like, just by way of advice, ’cause you’ve been through this journey of just playing with models. How long does it take to get a feel?
Peter Steinberger
If, if someone switches, I would give it a week until you actually develop a gut feeling for it.
If, if someone switches, I would give it a week until you actually develop a gut feeling for it.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
That’s … if you just … I think some people also make the mistake of they pay 200 for the, the Claude code version, then they pay 20 bucks for the OpenAI version. But if you pay like the, the 20 bucks version, you get the slow version. So your experience would be terrible because you’re used to this very interactive, very good system. And you switch to something that you have very little experience, then that’s gonna be very slow. So, I think OpenAI shot themselves a little bit in the foot by making the, the cheap version also slow. I would, I would have at least a small part of the fast preview. Or like, the experience that you get when you pay 200 before degrading to it being slow, because it’s already slow.
That’s … if you just … I think some people also make the mistake of they pay 200 for the, the Claude code version, then they pay 20 bucks for the OpenAI version. But if you pay like the, the 20 bucks version, you get the slow version. So your experience would be terrible because you’re used to this very interactive, very good system. And you switch to something that you have very little experience, then that’s gonna be very slow. So, I think OpenAI shot themselves a little bit in the foot by making the, the cheap version also slow. I would, I would have at least a small part of the fast preview. Or like, the experience that you get when you pay 200 before degrading to it being slow, because it’s already slow.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
I mean, they, they made it better. I think it’s … And, and they have plans to make it a lot better if the Cerebras stuff is true. But yeah, it’s a skill. It takes time. Even if you play … You have a regular guitar and you switch it to an E guitar, you’re not gonna play well right away. You have to, like, learn how it feels.
I mean, they, they made it better. I think it’s … And, and they have plans to make it a lot better if the Cerebras stuff is true. But yeah, it’s a skill. It takes time. Even if you play … You have a regular guitar and you switch it to an E guitar, you’re not gonna play well right away. You have to, like, learn how it feels.
Lex Fridman
The- there’s also this extra psychological effect that you’ve spoken about which is hilarious to watch. Which once people, uh … When the new model comes out, they try that model, they fall in love with it. “Wow, this is the smartest thing of all time,” and then they start saying, “You could just watch the Reddit posts over time,” start saying that, “We believe the intelligence of this model has been gradually degrading.” It, it says something about human nature and just the way our minds work, when it’s probably most likely the case that the intelligence of the model is not degrading. It’s in fact you’re getting used to a good thing.
The- there’s also this extra psychological effect that you’ve spoken about which is hilarious to watch. Which once people, uh … When the new model comes out, they try that model, they fall in love with it. “Wow, this is the smartest thing of all time,” and then they start saying, “You could just watch the Reddit posts over time,” start saying that, “We believe the intelligence of this model has been gradually degrading.” It, it says something about human nature and just the way our minds work, when it’s probably most likely the case that the intelligence of the model is not degrading. It’s in fact you’re getting used to a good thing.
Peter Steinberger
And your project grows, and you’re adding slop, and you probably don’t spend enough time to think about refactors. And you’re making it harder and harder for the agent to work on your slop. And then, and then suddenly, “Oh, now it’s hard. Oh no, it’s not working as well anymore.” What’s the motivation for, like, one of those AI companies to actually make their model dumber? Like, at most, it will make it slower if, if the server load’s too high. But, like, quantizing the model so you have a worse experience, so you go to the competitor?
And your project grows, and you’re adding slop, and you probably don’t spend enough time to think about refactors. And you’re making it harder and harder for the agent to work on your slop. And then, and then suddenly, “Oh, now it’s hard. Oh no, it’s not working as well anymore.” What’s the motivation for, like, one of those AI companies to actually make their model dumber? Like, at most, it will make it slower if, if the server load’s too high. But, like, quantizing the model so you have a worse experience, so you go to the competitor?
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
That just doesn’t seem like a very smart move in any way.
That just doesn’t seem like a very smart move in any way.
Best AI agent for programming
Lex Fridman
What do you think about Claude Code in comparison to Open Claude? So, Claude Code and maybe the Codex coding agent? Do you see them as kind of competitors?
What do you think about Claude Code in comparison to Open Claude? So, Claude Code and maybe the Codex coding agent? Do you see them as kind of competitors?
Peter Steinberger
I mean, first of all, competitor is fun when it’s not really a competition.
I mean, first of all, competitor is fun when it’s not really a competition.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Like, I’m happy if … If, if all it did is, like, inspire people to build something new, cool. Um, I still use Codex for the building. I, I know a lot of people use Open Claude to, to build stuff. And I worked hard on it to make that work. And I do smaller stuff with it in terms of code. But, like, if I work hours and hours, I want a big screen, not WhatsApp, you know? So for me, a personal agent is much more about my life. Or like, like a coworker. Like, I give you, like, a GitHub URL. Like, “Hey, try out this CLI. Does it actually work? What can we learn?” Blah, blah, blah. But when I’m deep in, deep in the flow, I want to have multiple, multiple things and it being very, very visible what it, what it does. So it … I don’t see it as a competition. It’s, it’s different things.
Like, I’m happy if … If, if all it did is, like, inspire people to build something new, cool. Um, I still use Codex for the building. I, I know a lot of people use Open Claude to, to build stuff. And I worked hard on it to make that work. And I do smaller stuff with it in terms of code. But, like, if I work hours and hours, I want a big screen, not WhatsApp, you know? So for me, a personal agent is much more about my life. Or like, like a coworker. Like, I give you, like, a GitHub URL. Like, “Hey, try out this CLI. Does it actually work? What can we learn?” Blah, blah, blah. But when I’m deep in, deep in the flow, I want to have multiple, multiple things and it being very, very visible what it, what it does. So it … I don’t see it as a competition. It’s, it’s different things.
Lex Fridman
But do, do you think there’s a a future where the two kinda combine? Like, your personal agent is also your best developing co-programmer partner?
But do, do you think there’s a a future where the two kinda combine? Like, your personal agent is also your best developing co-programmer partner?
Peter Steinberger
Yeah, totally. I think this is where the puck’s going, that this is gonna be more and more your operating system.
Yeah, totally. I think this is where the puck’s going, that this is gonna be more and more your operating system.
Lex Fridman
The operating system.
The operating system.
Peter Steinberger
And it already … It’s so funny. Like I, I added support for sub-agents and also for …… um, TTI support, so it could actually run Cloud Coder Codecs.
And it already … It’s so funny. Like I, I added support for sub-agents and also for …… um, TTI support, so it could actually run Cloud Coder Codecs.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And because mine’s a little bit bossy, it, it, it started it and it, it, it told him, like, “Who’s the boss,” basically. And it was like, “Ah, Codex is obeying me.”
And because mine’s a little bit bossy, it, it, it started it and it, it, it told him, like, “Who’s the boss,” basically. And it was like, “Ah, Codex is obeying me.”
Lex Fridman
Oh, this is a power struggle.
Oh, this is a power struggle.
Peter Steinberger
And also the current interface is probably not the final form. Like, if you think more globally, we are, we copied Google for agents. You have, like, a prompt, and, and then you have a chat interface. That, to me, very much feels like when we first created television and then people recorded radio shows on television and you saw that on TV.
And also the current interface is probably not the final form. Like, if you think more globally, we are, we copied Google for agents. You have, like, a prompt, and, and then you have a chat interface. That, to me, very much feels like when we first created television and then people recorded radio shows on television and you saw that on TV.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
I think there is, there’s n- there’s better ways how we eventually will communicate with models, and we are still very early in this, how will it even work phase. So, it will eventually converge and we will also figure out whole different ways how to work with those things.
I think there is, there’s n- there’s better ways how we eventually will communicate with models, and we are still very early in this, how will it even work phase. So, it will eventually converge and we will also figure out whole different ways how to work with those things.
Lex Fridman
One of the other components of workflow is operating system. So I told you offline that for the first time in my life, I’m expanding my sort of realm of exploration to the to the Apple ecosystem, to Macs, iPhone and so on. For most of my life I’ve been a Linux, Windows and WSL1, WSL2 person, which I think are all wonderful, but I… expanding to also trying Mac. Because it’s another way of building and it’s also a way of building that a large part of the community currently that’s utilizing LMS and agents is using, so. And that’s the reason I’m expanding to it. But is there something to be said about the different operating systems here? We should say that OpenClaw supported across operating systems.
One of the other components of workflow is operating system. So I told you offline that for the first time in my life, I’m expanding my sort of realm of exploration to the to the Apple ecosystem, to Macs, iPhone and so on. For most of my life I’ve been a Linux, Windows and WSL1, WSL2 person, which I think are all wonderful, but I… expanding to also trying Mac. Because it’s another way of building and it’s also a way of building that a large part of the community currently that’s utilizing LMS and agents is using, so. And that’s the reason I’m expanding to it. But is there something to be said about the different operating systems here? We should say that OpenClaw supported across operating systems.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
I saw WSL2 recommended, side windows for certain o- operations, but then Windows, Linux macOS are obviously supported.
I saw WSL2 recommended, side windows for certain o- operations, but then Windows, Linux macOS are obviously supported.
Peter Steinberger
Yeah, it should even work natively in Windows. I just didn’t have enough time to properly test it. And you know, like, the last 90% of software always easier than the first 90%, so I’m sure there’s some dragons left that will eventually nail out. My road was, for a long time, Windows, just because I grew up with that, then I switched and had a long phase with Linux, built my own kernels and everything, and then I went to university and I, I had my, my hacky Linux thing, and saw this white MacBook, and I just thought this is a thing of beauty, the white plastic one. And then I converted to Mac ’cause mostly w- I was, I was sick that audio wouldn’t work on Skype and all the other issues that, that Linux had for a long time.
Yeah, it should even work natively in Windows. I just didn’t have enough time to properly test it. And you know, like, the last 90% of software always easier than the first 90%, so I’m sure there’s some dragons left that will eventually nail out. My road was, for a long time, Windows, just because I grew up with that, then I switched and had a long phase with Linux, built my own kernels and everything, and then I went to university and I, I had my, my hacky Linux thing, and saw this white MacBook, and I just thought this is a thing of beauty, the white plastic one. And then I converted to Mac ’cause mostly w- I was, I was sick that audio wouldn’t work on Skype and all the other issues that, that Linux had for a long time.
Peter Steinberger
And then I just stuck with it and then I dug into iOS, which required macOS anyhow, so it was never a question. I think Apple lost a little bit of its lead in terms of native. It used to be… Native apps used to be so much better, and especially in the Mac, there’s more people that build software with love. On, on Windows, it, it… Windows has much more and, like, function wise, there’s just more, period. But a lot of it felt more functional and less done with love. Um, I mean, Mac always, like, attracted more designers and people I felt…
And then I just stuck with it and then I dug into iOS, which required macOS anyhow, so it was never a question. I think Apple lost a little bit of its lead in terms of native. It used to be… Native apps used to be so much better, and especially in the Mac, there’s more people that build software with love. On, on Windows, it, it… Windows has much more and, like, function wise, there’s just more, period. But a lot of it felt more functional and less done with love. Um, I mean, Mac always, like, attracted more designers and people I felt…
Peter Steinberger
Even though, like, often it has less features, it, it had more delight-
Even though, like, often it has less features, it, it had more delight-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… And playfulness. So I always valued that. But in the last few years, many times I actually prefer… Oh God, people are gonna roast me for that, but I prefer Electron apps because they work and native apps often, especially if it’s, like, a web service is a native app, are lacking features. I mean, not saying it couldn’t be done, it’s more like a, a focus thing that, like, for many, many companies, native was not that big of a priority. But if they build an Electron app, it, it’s the only app, so it is a priority and there’s a lot more code sharing possible. And I, I build a lot of native Mac apps. I love it. I, I can, I can help myself. Like, I love crafting little Mac, Mac menu bar tools. Like I built one to, to monitor your Codex use.
… And playfulness. So I always valued that. But in the last few years, many times I actually prefer… Oh God, people are gonna roast me for that, but I prefer Electron apps because they work and native apps often, especially if it’s, like, a web service is a native app, are lacking features. I mean, not saying it couldn’t be done, it’s more like a, a focus thing that, like, for many, many companies, native was not that big of a priority. But if they build an Electron app, it, it’s the only app, so it is a priority and there’s a lot more code sharing possible. And I, I build a lot of native Mac apps. I love it. I, I can, I can help myself. Like, I love crafting little Mac, Mac menu bar tools. Like I built one to, to monitor your Codex use.
Peter Steinberger
I built one I call Trimmy, that’s specifically for agentic use. When you, when you select text that goes over multiple lines it would remove the new line so you could actually paste it to the terminal. That was, again like, this is annoying me and after the, the 20th time of it is annoying me, I just built it. There is a cool Mac app for OpenClaw that I don’t think many people discovered yet, also because it, it still needs some love. It feels a little bit too much like the Hummer car right now because I, I just experiment a lot with it. It, it likes to polish.
I built one I call Trimmy, that’s specifically for agentic use. When you, when you select text that goes over multiple lines it would remove the new line so you could actually paste it to the terminal. That was, again like, this is annoying me and after the, the 20th time of it is annoying me, I just built it. There is a cool Mac app for OpenClaw that I don’t think many people discovered yet, also because it, it still needs some love. It feels a little bit too much like the Hummer car right now because I, I just experiment a lot with it. It, it likes to polish.
Lex Fridman
So you still… I mean, you still love it. You still, you still love adding to the delight of that operating system.
So you still… I mean, you still love it. You still, you still love adding to the delight of that operating system.
Peter Steinberger
Yeah, but then you realize… Like, I also built one, for example, for GitHub. And then the… If you use SwiftUI, like the latest and greatest at Apple, and took them forever to build something to show an image from the web. Now we have async, async image, but…… I added support for it and then some images would just not show up or, like, be very slow. And I had a discussion with Codex like, “Hey, why is there a bug?” And even Codex said like, “Yeah, there’s this ASIC image but it’s really more for experimenting and it should not be used in production.” But that’s Apple’s answer to, like, showing images from the web. This shouldn’t be so hard, you know.
Yeah, but then you realize… Like, I also built one, for example, for GitHub. And then the… If you use SwiftUI, like the latest and greatest at Apple, and took them forever to build something to show an image from the web. Now we have async, async image, but…… I added support for it and then some images would just not show up or, like, be very slow. And I had a discussion with Codex like, “Hey, why is there a bug?” And even Codex said like, “Yeah, there’s this ASIC image but it’s really more for experimenting and it should not be used in production.” But that’s Apple’s answer to, like, showing images from the web. This shouldn’t be so hard, you know.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
This is like… This is like insane. Like, how am I in, in, in 2026 and my agent tell me, “Don’t use the stuff Apple built because it’s, it’s… It’s… Yeah, it- it’s there but it’s not good.” And like this is now in the weeds. This is… To me this is like… They had so much head start and so much love, and they kind of just like blundered it and didn’t, didn’t evolve it as much as they should.
This is like… This is like insane. Like, how am I in, in, in 2026 and my agent tell me, “Don’t use the stuff Apple built because it’s, it’s… It’s… Yeah, it- it’s there but it’s not good.” And like this is now in the weeds. This is… To me this is like… They had so much head start and so much love, and they kind of just like blundered it and didn’t, didn’t evolve it as much as they should.
Lex Fridman
But also, there’s just the practical reality. If you look at Silicon Valley, most of the developer world that’s kind of playing with LMS and Agentic AI, they’re all using Apple products. And then, at the same time, Apple is not really, like, leaning on that. Like they’re not… They’re not opening up and playing and working together and like, yes.
But also, there’s just the practical reality. If you look at Silicon Valley, most of the developer world that’s kind of playing with LMS and Agentic AI, they’re all using Apple products. And then, at the same time, Apple is not really, like, leaning on that. Like they’re not… They’re not opening up and playing and working together and like, yes.
Peter Steinberger
Isn’t, isn’t it funny how they completely blunder AI, and yet everybody’s buying Mac Minis?
Isn’t, isn’t it funny how they completely blunder AI, and yet everybody’s buying Mac Minis?
Lex Fridman
How… What… Does that even make sense? You’re, you’re, you’re quite possibly the world’s greatest Mac salesman of all time.
How… What… Does that even make sense? You’re, you’re, you’re quite possibly the world’s greatest Mac salesman of all time.
Peter Steinberger
No, you don’t need a Mac Mini to install OpenClaw. You can install it on the web. There’s, there’s a concept called nodes, so you can like make your computer a node and it will do the same. There is something said for running it on separate hardware. That right now is useful. There is… There’s a big argument for the browser. You know, I, I built some Agentic browser use in there. And, I mean, it’s basically Playwright with a bunch of extras to make it easier for agents.
No, you don’t need a Mac Mini to install OpenClaw. You can install it on the web. There’s, there’s a concept called nodes, so you can like make your computer a node and it will do the same. There is something said for running it on separate hardware. That right now is useful. There is… There’s a big argument for the browser. You know, I, I built some Agentic browser use in there. And, I mean, it’s basically Playwright with a bunch of extras to make it easier for agents.
Lex Fridman
Playwright is a library that controls the browser.
Playwright is a library that controls the browser.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
It’s really nice, easy to use.
It’s really nice, easy to use.
Peter Steinberger
And our internet is slowly closing down. Like, there, there’s a whole movement to make it harder for agents to use. So if you do the same in a data center and websites detect that it’s an IP from a data center, the website might just block you or it make it really hard or put a lot of captures in the, in the way of the agent. I mean, agents are quite good at happily clicking, “I’m not a robot.”
And our internet is slowly closing down. Like, there, there’s a whole movement to make it harder for agents to use. So if you do the same in a data center and websites detect that it’s an IP from a data center, the website might just block you or it make it really hard or put a lot of captures in the, in the way of the agent. I mean, agents are quite good at happily clicking, “I’m not a robot.”
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
But having that on a residential IP makes a lot of things simpler. So there’s ways. Yeah. But it really does not need to be a Mac. It can… It can be any old hardware. I always say, like, maybe use the… Use the opportunity to get yourself a new MacBook or whatever computer you use and use the old one as your server instead of buying a standalone Mac Mini. But then there’s, again, there’s a lot of very cute things people build with Mac Minis that I like.
But having that on a residential IP makes a lot of things simpler. So there’s ways. Yeah. But it really does not need to be a Mac. It can… It can be any old hardware. I always say, like, maybe use the… Use the opportunity to get yourself a new MacBook or whatever computer you use and use the old one as your server instead of buying a standalone Mac Mini. But then there’s, again, there’s a lot of very cute things people build with Mac Minis that I like.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And no, I don’t get commission from Apple. They didn’t really communicate much.
And no, I don’t get commission from Apple. They didn’t really communicate much.
Lex Fridman
It’s sad. It’s sad. Can you actually speak to what it takes to get started with OpenClaw? There’s… I mean, there’s a lot of people… What is it? Somebody tweeted at you, “Peter, make OpenClaw easy to set up for everyday people. 99.9% of people can’t access to OpenClaw and have their own lobster because of their technical difficulties in getting it set up. Make OpenClaw accessible to everyone, please.” And you replied, “Working on that.” From my perspective, it seems there- there’s a bunch of different options and it’s already quite straightforward, but I suppose that’s if you have some developer background.
It’s sad. It’s sad. Can you actually speak to what it takes to get started with OpenClaw? There’s… I mean, there’s a lot of people… What is it? Somebody tweeted at you, “Peter, make OpenClaw easy to set up for everyday people. 99.9% of people can’t access to OpenClaw and have their own lobster because of their technical difficulties in getting it set up. Make OpenClaw accessible to everyone, please.” And you replied, “Working on that.” From my perspective, it seems there- there’s a bunch of different options and it’s already quite straightforward, but I suppose that’s if you have some developer background.
Peter Steinberger
I mean, right now you have to paste in one liner into the terminal.
I mean, right now you have to paste in one liner into the terminal.
Lex Fridman
Right.
Right.
Peter Steinberger
And there’s also an app. The app kind of does that for you, but there should be a Windows app. The app needs to be easier and more loved. The configuration should potentially be web-based or in the app. And I started working on that, but honestly right now I want to focus on security aspects. And, and once I’m confident that this is at a level that I can recommend my mom, then I’m going to make it simpler. Like I…
And there’s also an app. The app kind of does that for you, but there should be a Windows app. The app needs to be easier and more loved. The configuration should potentially be web-based or in the app. And I started working on that, but honestly right now I want to focus on security aspects. And, and once I’m confident that this is at a level that I can recommend my mom, then I’m going to make it simpler. Like I…
Peter Steinberger
Right now-
Right now-
Lex Fridman
You want to make it harder so that it doesn’t scale as fast as it’s scaling.
You want to make it harder so that it doesn’t scale as fast as it’s scaling.
Peter Steinberger
Yeah, it would be nice if it wouldn’t… I mean, that’s, like, hard to say, right? But if the growth would be a little slower, that would be helpful because people are expecting inhuman things from a single human being. And yes, I have some contributors, but also that whole machinery I started a week ago so that needs more time to figure out. And, and not everyone has all day to work on that.
Yeah, it would be nice if it wouldn’t… I mean, that’s, like, hard to say, right? But if the growth would be a little slower, that would be helpful because people are expecting inhuman things from a single human being. And yes, I have some contributors, but also that whole machinery I started a week ago so that needs more time to figure out. And, and not everyone has all day to work on that.
Lex Fridman
There’s some beginners listening to this, programming beginners. What advice would you give to them about, let’s say, joining the Agentic AI revolution?
There’s some beginners listening to this, programming beginners. What advice would you give to them about, let’s say, joining the Agentic AI revolution?
Peter Steinberger
Play. Playing is the best… The best way to learn. If you wanna… I’m sure if you… If you are like a little bit of builder, you have an idea in your head that you want to build, just build that, or like, give it a try. It doesn’t need to be perfect. I built a whole bunch of stuff that I don’t use. It doesn’t matter. Like, it’s the journey.
Play. Playing is the best… The best way to learn. If you wanna… I’m sure if you… If you are like a little bit of builder, you have an idea in your head that you want to build, just build that, or like, give it a try. It doesn’t need to be perfect. I built a whole bunch of stuff that I don’t use. It doesn’t matter. Like, it’s the journey.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
You know? Like the philosophical way, that the end doesn’t matter, the journey matters. Have fun.
You know? Like the philosophical way, that the end doesn’t matter, the journey matters. Have fun.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
My God, like those things… I… I don’t think I ever had so much fun building things because I can focus on the hard parts now. A lot of coding, I always thought I liked coding, but really I like building.
My God, like those things… I… I don’t think I ever had so much fun building things because I can focus on the hard parts now. A lot of coding, I always thought I liked coding, but really I like building.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And… And whenever you don’t understand something, just ask. You have an infinitely patient answering machine…. that y- can explain you anything at any level of complexity. Sometimes, that’s like one time I asked, “Hey explain to me like I’m- I’m eight years old,” and it started giving me a story with crayons and stuff. And I’m like, “No, not like that.” Like, I’m okay- … up- up the age a little bit, you know? I’m like, I’m not an actual child, it’s just, I just need a simpler language for like a- a- a- a- a tricky database concept that I didn’t grok in the first- first time. But, you know, just, you can just ask things. Like, you- there’s like… It used to be that I had to go on Stack Overflow or ha- ask on Twitter, and then maybe two days later I get a response.
And… And whenever you don’t understand something, just ask. You have an infinitely patient answering machine…. that y- can explain you anything at any level of complexity. Sometimes, that’s like one time I asked, “Hey explain to me like I’m- I’m eight years old,” and it started giving me a story with crayons and stuff. And I’m like, “No, not like that.” Like, I’m okay- … up- up the age a little bit, you know? I’m like, I’m not an actual child, it’s just, I just need a simpler language for like a- a- a- a- a tricky database concept that I didn’t grok in the first- first time. But, you know, just, you can just ask things. Like, you- there’s like… It used to be that I had to go on Stack Overflow or ha- ask on Twitter, and then maybe two days later I get a response.
Peter Steinberger
Or I had to try for hours. And now you- you can just ask stuff. It- I mean, it’s never… You have, like, your own teacher. You know that there’s like statistics, y- you can learn faster if you have your own teacher. The- it’s like you have this infinitely patient machine. Ask it.
Or I had to try for hours. And now you- you can just ask stuff. It- I mean, it’s never… You have, like, your own teacher. You know that there’s like statistics, y- you can learn faster if you have your own teacher. The- it’s like you have this infinitely patient machine. Ask it.
Lex Fridman
But what would you say? So use… What’s the easiest way to play? So maybe Open Claw is a nice way to play so you can then set- set everything up and then you could chat with it.
But what would you say? So use… What’s the easiest way to play? So maybe Open Claw is a nice way to play so you can then set- set everything up and then you could chat with it.
Peter Steinberger
You can also just experiment with it and, like, modify it. Ask your agent. I mean, there is infinite ways how it can be made better. Play around, make it better.
You can also just experiment with it and, like, modify it. Ask your agent. I mean, there is infinite ways how it can be made better. Play around, make it better.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
More general, if you- if you’re a beginner and you actually wanna learn how to build software really fast, get involved in open source. Doesn’t need to be my project. In fact, maybe don’t use my project because my- my backlog is very large, but I learned so much from open source. Just like, like, be- be humble. Don’t- maybe don’t send a pull request right away. But there’s many other ways you can help out. There’s many ways you can just learn by just reading code. By- by being on Discord or wherever people are, and just, like, understanding how things are built. I don’t know, like Mitchell Hashimoto builds Ghostly, the terminal, and he has a really good community where there’s so many other projects. Like, pick something that you find interesting and get involved.
More general, if you- if you’re a beginner and you actually wanna learn how to build software really fast, get involved in open source. Doesn’t need to be my project. In fact, maybe don’t use my project because my- my backlog is very large, but I learned so much from open source. Just like, like, be- be humble. Don’t- maybe don’t send a pull request right away. But there’s many other ways you can help out. There’s many ways you can just learn by just reading code. By- by being on Discord or wherever people are, and just, like, understanding how things are built. I don’t know, like Mitchell Hashimoto builds Ghostly, the terminal, and he has a really good community where there’s so many other projects. Like, pick something that you find interesting and get involved.
Lex Fridman
Do you recommend that people that don’t know how to program or don’t really know how to program learn to program also? So when you you can get quite far right now by just using natural language, right? Do you s- still see a lot of value in reading the code, understanding the code, and then being able to write a little bit of code from scratch?
Do you recommend that people that don’t know how to program or don’t really know how to program learn to program also? So when you you can get quite far right now by just using natural language, right? Do you s- still see a lot of value in reading the code, understanding the code, and then being able to write a little bit of code from scratch?
Peter Steinberger
It definitely helps.
It definitely helps.
Lex Fridman
It’s hard for you to answer that-
It’s hard for you to answer that-
Peter Steinberger
Yeah
Yeah
Lex Fridman
… because you don’t know what it’s like to do any of this without knowing the base knowledge. Like, you might take for granted just how much intuition you have about the programming world having programmed so much, right?
… because you don’t know what it’s like to do any of this without knowing the base knowledge. Like, you might take for granted just how much intuition you have about the programming world having programmed so much, right?
Peter Steinberger
There’s people that are high agency and very curious, and they get very far even though they have no deep understanding how software works just because they ask questions and questions and- and- and-
There’s people that are high agency and very curious, and they get very far even though they have no deep understanding how software works just because they ask questions and questions and- and- and-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… and agents are infinitely patient. Like, part of what I did this year is I went to a lot of iOS conferences because that’s my background and just told people, “Don’t consi- don’t see yourself as an iOS engineer anymore.” Like, “You need to change your mindset. You’re a builder.” And you can take a lot of the knowledge how to build software into new domains and all of the- the more fine-grain details, agents can help. You don’t have to know how to splice an array or what the- what the correct template syntax is or whatever, but you can use all your- your general knowledge and that makes it much easier to move from one galaxy, one tech galaxy into another. And oftentimes, there’s languages that make more or less sense depending on what you build, right?
… and agents are infinitely patient. Like, part of what I did this year is I went to a lot of iOS conferences because that’s my background and just told people, “Don’t consi- don’t see yourself as an iOS engineer anymore.” Like, “You need to change your mindset. You’re a builder.” And you can take a lot of the knowledge how to build software into new domains and all of the- the more fine-grain details, agents can help. You don’t have to know how to splice an array or what the- what the correct template syntax is or whatever, but you can use all your- your general knowledge and that makes it much easier to move from one galaxy, one tech galaxy into another. And oftentimes, there’s languages that make more or less sense depending on what you build, right?
Peter Steinberger
So for example, when I build simple CLIs, I like Go. I actually don’t like Go. I don’t like the syntax of Go. I didn’t even consider the language. But the ecosystem is great, it works great with agents. It is garbage collected. It’s not the highest performing one, but it’s very fast. And for those type of- of CLIs that I build, Go is- is a really good choice. So I- I use a language I’m not even a fan of for… That’s my main to-go thing for- for CLIs.
So for example, when I build simple CLIs, I like Go. I actually don’t like Go. I don’t like the syntax of Go. I didn’t even consider the language. But the ecosystem is great, it works great with agents. It is garbage collected. It’s not the highest performing one, but it’s very fast. And for those type of- of CLIs that I build, Go is- is a really good choice. So I- I use a language I’m not even a fan of for… That’s my main to-go thing for- for CLIs.
Lex Fridman
Isn’t that fascinating that here’s a programming language you would’ve never used if you had to write it from scratch and now you’re using because LMs are good at generating it and it has some of the characteristics that makes it resilient, like garbage collected?
Isn’t that fascinating that here’s a programming language you would’ve never used if you had to write it from scratch and now you’re using because LMs are good at generating it and it has some of the characteristics that makes it resilient, like garbage collected?
Peter Steinberger
Because everything’s weird in this new world and that just makes the most sense.
Because everything’s weird in this new world and that just makes the most sense.
Lex Fridman
What’s the best Ridiculous question. What’s the best programming language for the AI- AI agentic world? Is it JavaScript, TypeScript?
What’s the best Ridiculous question. What’s the best programming language for the AI- AI agentic world? Is it JavaScript, TypeScript?
Peter Steinberger
TypeScript is really good. Sometimes the types can get really confusing and the ecosystem is- is a jungle. So for- for web stuff it’s good. I wouldn’t build everything in it.
TypeScript is really good. Sometimes the types can get really confusing and the ecosystem is- is a jungle. So for- for web stuff it’s good. I wouldn’t build everything in it.
Lex Fridman
Don’t you think we’re moving there? Like, that everything will eventually be written- eventually is written in JavaScript and it-
Don’t you think we’re moving there? Like, that everything will eventually be written- eventually is written in JavaScript and it-
Peter Steinberger
The birth and death of JavaScript and we are living through it in real time.
The birth and death of JavaScript and we are living through it in real time.
Lex Fridman
Like, what does programming look like in 20 years? Right? In 30 years? In 40 years? What do programs and apps look like?
Like, what does programming look like in 20 years? Right? In 30 years? In 40 years? What do programs and apps look like?
Peter Steinberger
You can even ask a question like, do we need a- a programming language that’s made for agents? Because all of those languages are made for humans. So how- what would that look like? Um, I think there’s a- there’s whole bunch of interesting questions that we’ll discover. And also how because everything is now world knowledge, how it in many ways, things will stagnate ’cause if you build something new and the agent has no idea that’s gonna be much harder to use than something that’s already there. Um…… of when I build Mac apps, I build them in, in Swift and SwiftUI, mm, partly because I like pain, partly because it… the, the deepest level of system integration, I can only get through there.
You can even ask a question like, do we need a- a programming language that’s made for agents? Because all of those languages are made for humans. So how- what would that look like? Um, I think there’s a- there’s whole bunch of interesting questions that we’ll discover. And also how because everything is now world knowledge, how it in many ways, things will stagnate ’cause if you build something new and the agent has no idea that’s gonna be much harder to use than something that’s already there. Um…… of when I build Mac apps, I build them in, in Swift and SwiftUI, mm, partly because I like pain, partly because it… the, the deepest level of system integration, I can only get through there.
Peter Steinberger
And you clearly feel a difference if you click on an electron app and it loads a web view in the menu. It’s just not the same. Sometimes I just also try new languages just to, like, get a feel for them.
And you clearly feel a difference if you click on an electron app and it loads a web view in the menu. It’s just not the same. Sometimes I just also try new languages just to, like, get a feel for them.
Lex Fridman
Like Zig?
Like Zig?
Peter Steinberger
Yeah. If it’s something that… where I care about performance a lot then it’s, it’s a really interesting language. And it… like agents got so much better over the last six months from not really good to totally valid choice. Just still a, a very young ecosystem. And most of the time you actually care about ecosystem, right? So, so if you build something that does inference or goes into whole running model direction, Python, very good.
Yeah. If it’s something that… where I care about performance a lot then it’s, it’s a really interesting language. And it… like agents got so much better over the last six months from not really good to totally valid choice. Just still a, a very young ecosystem. And most of the time you actually care about ecosystem, right? So, so if you build something that does inference or goes into whole running model direction, Python, very good.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
But then if I build stuff in Python and I want a story where I can also deploy it on Windows, not a good choice.
But then if I build stuff in Python and I want a story where I can also deploy it on Windows, not a good choice.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Sometimes I, I found projects that kinda did 90% of what I wanted but were in Python, and I wanted them… I wanted an easy Windows story. Okay, just rewrite it in Go. But then if you go towards multiple, multiple threads and a lot more performance, Rust is a really good choice. There’s no… there’s just no single answer, and it’s also the beauty of it. Like, it’s fun.
Sometimes I, I found projects that kinda did 90% of what I wanted but were in Python, and I wanted them… I wanted an easy Windows story. Okay, just rewrite it in Go. But then if you go towards multiple, multiple threads and a lot more performance, Rust is a really good choice. There’s no… there’s just no single answer, and it’s also the beauty of it. Like, it’s fun.
Peter Steinberger
And now it doesn’t matter anymore, you can just literally pick the language that has the, the most fitting characteristics and ecosystem-
And now it doesn’t matter anymore, you can just literally pick the language that has the, the most fitting characteristics and ecosystem-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… for your problem domain. And yeah, it might be… You might have s-… You might be a little bit slow in reading the code, but not really. Y- I think you, you pick stuff up really fast, and you can always ask your agent.
… for your problem domain. And yeah, it might be… You might have s-… You might be a little bit slow in reading the code, but not really. Y- I think you, you pick stuff up really fast, and you can always ask your agent.
Life story and career advice
Lex Fridman
So there’s a lot of programmers and builders who draw inspiration from y- your story. Just the way you carry yourself, your choice of making OpenClaw open source, the, the way you have fun building and exploring, and doing that, for the most part, alone or on a small team. So by way of advice, what metric should be the goal that they would be optimizing for? What would be the metric of success? Would it be happiness? Is it money? Is it positive impact for people who are dreaming of building? ‘Cause you went through an interesting journey. You’ve achieved a lot of those things, and then you fell out of love with programming a little bit for a time.
So there’s a lot of programmers and builders who draw inspiration from y- your story. Just the way you carry yourself, your choice of making OpenClaw open source, the, the way you have fun building and exploring, and doing that, for the most part, alone or on a small team. So by way of advice, what metric should be the goal that they would be optimizing for? What would be the metric of success? Would it be happiness? Is it money? Is it positive impact for people who are dreaming of building? ‘Cause you went through an interesting journey. You’ve achieved a lot of those things, and then you fell out of love with programming a little bit for a time.
Peter Steinberger
I was just burning too bright for too long. I, I ran… I started PSPDFKit, s- and ran it for 13 years, and it was high stress. Um, I had to learn all these things fast and hard, like how to manage people, how to bring people on, how to deal with customers, how to do…
I was just burning too bright for too long. I, I ran… I started PSPDFKit, s- and ran it for 13 years, and it was high stress. Um, I had to learn all these things fast and hard, like how to manage people, how to bring people on, how to deal with customers, how to do…
Lex Fridman
So it wasn’t just programming stuff, it was people stuff.
So it wasn’t just programming stuff, it was people stuff.
Peter Steinberger
The stuff that burned me out was mostly people stuff. I, I don’t think burnout is working too much. Maybe to a degree. Everybody’s different. You know, I c- I cannot speak in a- in absolute terms, but for me, it was much more differences with my, my co-founders, conflicts, or, like, really high stress situation with customers that eventually grinded me down. And then when… luckily we, we got a really good offer for, like, putting the company to the next level and I, I already kinda worked two years on making myself obsolete. So at this point I could leave, and, and then I just… I was sitting in front of the screen and I felt like, you know Austin Powers where they suck the mojo out?
The stuff that burned me out was mostly people stuff. I, I don’t think burnout is working too much. Maybe to a degree. Everybody’s different. You know, I c- I cannot speak in a- in absolute terms, but for me, it was much more differences with my, my co-founders, conflicts, or, like, really high stress situation with customers that eventually grinded me down. And then when… luckily we, we got a really good offer for, like, putting the company to the next level and I, I already kinda worked two years on making myself obsolete. So at this point I could leave, and, and then I just… I was sitting in front of the screen and I felt like, you know Austin Powers where they suck the mojo out?
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Uh, I g- I was like, m- m- it was, like, gone. Like, I couldn’t… I couldn’t get code out anymore. I was just, like, staring and feeling empty, and then I, I just stopped. I, I booked, like, a one-way trip to Madrid and, and, and just, like, spent a t- some t- sometime there. I felt like I had to catch up on life, so I did a whole, a whole bunch of life catching up stuff.
Uh, I g- I was like, m- m- it was, like, gone. Like, I couldn’t… I couldn’t get code out anymore. I was just, like, staring and feeling empty, and then I, I just stopped. I, I booked, like, a one-way trip to Madrid and, and, and just, like, spent a t- some t- sometime there. I felt like I had to catch up on life, so I did a whole, a whole bunch of life catching up stuff.
Lex Fridman
Did you go through some lows during that period? And you know, maybe advice on… of how to?
Did you go through some lows during that period? And you know, maybe advice on… of how to?
Peter Steinberger
Maybe advice on how to approach life. If you think that, “Oh yeah, work really hard and then I’ll retire,” I don’t recommend that. Because the idea of, “Oh yeah, I just enjoy life now,” a- maybe it’s appealing, but right now I enjoy life, the most I’ve ever enjoyed life. Because if you wake up in the morning and you have nothing to look forward to, you have no real challenge, that gets very boring, very fast. And then when, when you’re bored, you’re gonna look for other places how to stimulate yourself, and then maybe, maybe that’s drugs, you know? But that eventually also get boring and you look for more, and that will lead you down a very dark path.
Maybe advice on how to approach life. If you think that, “Oh yeah, work really hard and then I’ll retire,” I don’t recommend that. Because the idea of, “Oh yeah, I just enjoy life now,” a- maybe it’s appealing, but right now I enjoy life, the most I’ve ever enjoyed life. Because if you wake up in the morning and you have nothing to look forward to, you have no real challenge, that gets very boring, very fast. And then when, when you’re bored, you’re gonna look for other places how to stimulate yourself, and then maybe, maybe that’s drugs, you know? But that eventually also get boring and you look for more, and that will lead you down a very dark path.
Money and happiness
Lex Fridman
But you also showed on the money front, you know, a lot of people in Silicon Valley and the startup world, they think, maybe overthink way too much optimized for money. And you’ve also shown that it’s not like you’re saying no to money. I mean, I’m sure you take money, but it’s not…… the primary objective of uh, of your life. Can you just speak to that? Your philosophy on money?
But you also showed on the money front, you know, a lot of people in Silicon Valley and the startup world, they think, maybe overthink way too much optimized for money. And you’ve also shown that it’s not like you’re saying no to money. I mean, I’m sure you take money, but it’s not…… the primary objective of uh, of your life. Can you just speak to that? Your philosophy on money?
Peter Steinberger
When I built my company, money was never the driving force. It felt more like, like, an affirmation that I did something right. And having money solves a lot of problems. I also think there, there’s diminishing returns the more you have. Like, a cheeseburger is a cheeseburger, and I think if you go too far into, oh, I do private jet and I only travel luxury, you disconnect with society. Um, I, I donated quite a lot. Like, I have a, I have a foundation for helping people that weren’t so lucky.
When I built my company, money was never the driving force. It felt more like, like, an affirmation that I did something right. And having money solves a lot of problems. I also think there, there’s diminishing returns the more you have. Like, a cheeseburger is a cheeseburger, and I think if you go too far into, oh, I do private jet and I only travel luxury, you disconnect with society. Um, I, I donated quite a lot. Like, I have a, I have a foundation for helping people that weren’t so lucky.
Lex Fridman
And disconnecting from society is bad in that on many levels, but one of them is, like, humans are awesome. It’s nice to continuously remember the awesomeness in humans.
And disconnecting from society is bad in that on many levels, but one of them is, like, humans are awesome. It’s nice to continuously remember the awesomeness in humans.
Peter Steinberger
I, I mean, I could afford really nice hotels. The last time I was in San Francisco, I did the, the first time the OG Airbnb experience-
I, I mean, I could afford really nice hotels. The last time I was in San Francisco, I did the, the first time the OG Airbnb experience-
Lex Fridman
Yeah, yeah
Yeah, yeah
Peter Steinberger
… and just booked a room. Mostly because I, I thought, okay, you know, I’m out or I’m sleeping, and I don’t like where all the hotels are, and I wanted a, I wanted a different experience. I think, isn’t life all about experiences? Like, if you, if you tailor your life towards, “I wanna have experiences,” it, it reduces the need for, “It needs to be good or bad.” Like, if people only want good experiences, that’s not gonna work, but if you optimize for experiences, if it’s good, amazing. If it’s bad, amazing, because, like, I learned something, I saw something, did something. I wanted to experience that, and it was amazing. Like, there was, like, this, this queer DJ in there, and I showed her how to make music with cloud code. And we, like, immediately bonded and had a great time.
… and just booked a room. Mostly because I, I thought, okay, you know, I’m out or I’m sleeping, and I don’t like where all the hotels are, and I wanted a, I wanted a different experience. I think, isn’t life all about experiences? Like, if you, if you tailor your life towards, “I wanna have experiences,” it, it reduces the need for, “It needs to be good or bad.” Like, if people only want good experiences, that’s not gonna work, but if you optimize for experiences, if it’s good, amazing. If it’s bad, amazing, because, like, I learned something, I saw something, did something. I wanted to experience that, and it was amazing. Like, there was, like, this, this queer DJ in there, and I showed her how to make music with cloud code. And we, like, immediately bonded and had a great time.
Lex Fridman
Yeah, there’s something about that air- you know, couch surfing, Airbnb experience, the OG. I’m still to this day. It’s awesome. It’s humans, and that’s why travel is awesome.
Yeah, there’s something about that air- you know, couch surfing, Airbnb experience, the OG. I’m still to this day. It’s awesome. It’s humans, and that’s why travel is awesome.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Just experience the variety of, the diversity of human. And when it’s shitty, it’s good too, man. If it rains and you’re soaked and it’s all fucked, and planes, the everything is shit, everything is fucked, it’s still awesome. If you’re able to open your eyes it’s good to be alive.
Just experience the variety of, the diversity of human. And when it’s shitty, it’s good too, man. If it rains and you’re soaked and it’s all fucked, and planes, the everything is shit, everything is fucked, it’s still awesome. If you’re able to open your eyes it’s good to be alive.
Peter Steinberger
Yeah, and anything that creates emotion and feelings is good.
Yeah, and anything that creates emotion and feelings is good.
Lex Fridman
.
.
Peter Steinberger
Even… So, so maybe, maybe even the cryptic people are good because they definitely created emotions. I, I don’t know if I should go that far.
Even… So, so maybe, maybe even the cryptic people are good because they definitely created emotions. I, I don’t know if I should go that far.
Lex Fridman
No, man. Give them, give them all, give them love. Give them love. Because I do think that online lacks some of the awesomeness of real life.
No, man. Give them, give them all, give them love. Give them love. Because I do think that online lacks some of the awesomeness of real life.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
That’s, that’s, it’s an open problem of how to solve, how to infuse the online cyber experience with I don’t know with the intensity that we humans feel when it’s in real life. I don’t know. I don’t know if that’s a solvable problem.
That’s, that’s, it’s an open problem of how to solve, how to infuse the online cyber experience with I don’t know with the intensity that we humans feel when it’s in real life. I don’t know. I don’t know if that’s a solvable problem.
Peter Steinberger
Well, it’s just possible because text is very lossy.
Well, it’s just possible because text is very lossy.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
You know, sometimes I wish if I talked to the agent I would… It should be multi-model so it also understands my emotions.
You know, sometimes I wish if I talked to the agent I would… It should be multi-model so it also understands my emotions.
Lex Fridman
I mean, it, it might move there. It might move there.
I mean, it, it might move there. It might move there.
Peter Steinberger
It will. It will. It totally will.
It will. It will. It totally will.
Acquisition offers from OpenAI and Meta
Lex Fridman
I mean, I have to ask you, just curious. I, I know you’ve probably gotten huge offers from major companies. Can you speak to who you’re considering working with?
I mean, I have to ask you, just curious. I, I know you’ve probably gotten huge offers from major companies. Can you speak to who you’re considering working with?
Peter Steinberger
Yeah. So, to like explain my thinking a little bit, right, I did not expect this blowing up so much. So, there’s a lot of doors that opened because of it. There’s, like, I think every VC, every big VC company is in my inbox and tried to get 15 minutes of me. So, there’s, like, this butterfly effect moment. I could just do nothing and continue and I really like my life. Valid choice. Almost. Like, I considered it when I delete it, wanted to delete the whole thing. I could create a company. Been there, done that. There’s so many people that push me towards that and, yeah, like, could be amazing.
Yeah. So, to like explain my thinking a little bit, right, I did not expect this blowing up so much. So, there’s a lot of doors that opened because of it. There’s, like, I think every VC, every big VC company is in my inbox and tried to get 15 minutes of me. So, there’s, like, this butterfly effect moment. I could just do nothing and continue and I really like my life. Valid choice. Almost. Like, I considered it when I delete it, wanted to delete the whole thing. I could create a company. Been there, done that. There’s so many people that push me towards that and, yeah, like, could be amazing.
Lex Fridman
Which is to say that you, you would probably raise a lot of money in that.
Which is to say that you, you would probably raise a lot of money in that.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
I don’t know, hundreds of millions, billions. I don’t know. It could just got unlimited amount of money.
I don’t know, hundreds of millions, billions. I don’t know. It could just got unlimited amount of money.
Peter Steinberger
Yeah. It just doesn’t excite me as much because I feel I did all of that, and it would take a lot of time away from the things I actually enjoy. Same as when, when I was CEO, I think I, I learned to do it and I’m not bad at it, and partly I’m good at it. But yeah, that path doesn’t excite me too much, and I also fear it, it would create a natural conflict of interest. Like, what’s the most obvious thing I do? I, I prioritize it. I put, like, a version safe for workplace. And then what do you do? Like, I get a pull request with a feature like an audit log, but that seems like an enterprise feature, so now I feel I have a conflict of interest in the open-source version and the closed-source version….
Yeah. It just doesn’t excite me as much because I feel I did all of that, and it would take a lot of time away from the things I actually enjoy. Same as when, when I was CEO, I think I, I learned to do it and I’m not bad at it, and partly I’m good at it. But yeah, that path doesn’t excite me too much, and I also fear it, it would create a natural conflict of interest. Like, what’s the most obvious thing I do? I, I prioritize it. I put, like, a version safe for workplace. And then what do you do? Like, I get a pull request with a feature like an audit log, but that seems like an enterprise feature, so now I feel I have a conflict of interest in the open-source version and the closed-source version….
Peter Steinberger
or change the license to something like FSL, where you cannot actually use it for commercial stuff, would first be very difficult with all the contributions. And second of all, I- I like the idea that it’s free as in beer and not free with conditions. Yeah, there’s ways how you, how you keep all of that for free and just, like, still try to make money, but those are very difficult. And you see there’s, like, fewer and fewer companies manage that. Like, even Tailwind, they’re, like, used by everyone. Everyone uses Tailwind, right? And then they had to cut off 75% of the employees because they’re not making money because nobody’s even going on the website anymore because it’s all done by agents. S- and just relying on donations, yeah, good luck.
or change the license to something like FSL, where you cannot actually use it for commercial stuff, would first be very difficult with all the contributions. And second of all, I- I like the idea that it’s free as in beer and not free with conditions. Yeah, there’s ways how you, how you keep all of that for free and just, like, still try to make money, but those are very difficult. And you see there’s, like, fewer and fewer companies manage that. Like, even Tailwind, they’re, like, used by everyone. Everyone uses Tailwind, right? And then they had to cut off 75% of the employees because they’re not making money because nobody’s even going on the website anymore because it’s all done by agents. S- and just relying on donations, yeah, good luck.
Peter Steinberger
Like, if a project of my caliber, if I extrapolate what the typical open-source project would get it’s not a lot. I s- I still lose money on the project because I made the point of supporting every dependency, except Slack. They are a big company. They can, they can, they can do without me. But all the projects that are done by mostly individuals so, like, all the, right now, all the sponsorship goes right up to my dependencies. And if there’s more, I want to, like, buy my contributors some merch, you know?
Like, if a project of my caliber, if I extrapolate what the typical open-source project would get it’s not a lot. I s- I still lose money on the project because I made the point of supporting every dependency, except Slack. They are a big company. They can, they can, they can do without me. But all the projects that are done by mostly individuals so, like, all the, right now, all the sponsorship goes right up to my dependencies. And if there’s more, I want to, like, buy my contributors some merch, you know?
Lex Fridman
So you’re losing money?
So you’re losing money?
Peter Steinberger
Yeah, right now I lose money on this.
Yeah, right now I lose money on this.
Lex Fridman
So it’s really not sustainable?
So it’s really not sustainable?
Peter Steinberger
Uh, I mean, it’s like, I guess something between 10 and 20K a month. Which is fine. I’m sure over time I could get that down. Um, OpenAI is helping out a little bit with tokens now. And there’s other companies that have been generous. But yeah, still losing money on that. So that’s- that’s one path I consider, but I’m just not very excited. And then there’s all the big labs that I’ve been talking to. And from those Meta and OpenAI seem the most interesting.
Uh, I mean, it’s like, I guess something between 10 and 20K a month. Which is fine. I’m sure over time I could get that down. Um, OpenAI is helping out a little bit with tokens now. And there’s other companies that have been generous. But yeah, still losing money on that. So that’s- that’s one path I consider, but I’m just not very excited. And then there’s all the big labs that I’ve been talking to. And from those Meta and OpenAI seem the most interesting.
Lex Fridman
Do you lean one way or the other?
Do you lean one way or the other?
Peter Steinberger
Yeah. Um… Not sure how much I should share there. It’s not quite finalized yet. Let’s- let’s just say, like, on either of these, my conditions are that the project stays open source. That it… Maybe it’s gonna be a model like Chrome and Chromium. Um, I think this is- this is too important to just give to a company and make it theirs. It… This is… And we didn’t even talk about the whole community part, but, like, the- the thing that I experienced in San Francisco, like at ClawCon, seeing so many people so inspired, like… And having fun and just, like, building shit, and, like, having, like, robots in lobster stuff walking around. Like, the…
Yeah. Um… Not sure how much I should share there. It’s not quite finalized yet. Let’s- let’s just say, like, on either of these, my conditions are that the project stays open source. That it… Maybe it’s gonna be a model like Chrome and Chromium. Um, I think this is- this is too important to just give to a company and make it theirs. It… This is… And we didn’t even talk about the whole community part, but, like, the- the thing that I experienced in San Francisco, like at ClawCon, seeing so many people so inspired, like… And having fun and just, like, building shit, and, like, having, like, robots in lobster stuff walking around. Like, the…
Peter Steinberger
People told me, like, they didn’t experience this level of- of community excitement since, like, the early days of the internet, like 10, 15 years. And there were a lot of high caliber people there, like… Um, I was amazed. I also, like, was very sensory overloaded because too many people wanted to do selfies. But I love this. Like, this needs to stay a place where people can, like, hack and learn. But also, I’m very excited to, like, make this into a version that I can get to a lot of people because I think this is the year of personal agents, and that’s the future. And the fastest way to do that is teaming up with one of the labs. And I also, on a personal level, I never worked at a large company, and I’m intrigued. You know, we talk about experiences. Will I like it? I don’t know.
People told me, like, they didn’t experience this level of- of community excitement since, like, the early days of the internet, like 10, 15 years. And there were a lot of high caliber people there, like… Um, I was amazed. I also, like, was very sensory overloaded because too many people wanted to do selfies. But I love this. Like, this needs to stay a place where people can, like, hack and learn. But also, I’m very excited to, like, make this into a version that I can get to a lot of people because I think this is the year of personal agents, and that’s the future. And the fastest way to do that is teaming up with one of the labs. And I also, on a personal level, I never worked at a large company, and I’m intrigued. You know, we talk about experiences. Will I like it? I don’t know.
Peter Steinberger
But I want that experience. Uh, I- I’m sure, like, if- if I- if I announce this, then there will be people like, “Oh, he sold out,” blah, blah, blah. But the project will continue. From everything I talked to so far, I can even have more resources for that. Like, both s- both of those companies understand the value that I created something that accelerates our timeline and that got people excited about AI. I mean, can you imagine? Like, I installed OpenClaw on one of my, I’m sorry, normie friends. I’m sorry, Vahan. But he’s just a… You know?
But I want that experience. Uh, I- I’m sure, like, if- if I- if I announce this, then there will be people like, “Oh, he sold out,” blah, blah, blah. But the project will continue. From everything I talked to so far, I can even have more resources for that. Like, both s- both of those companies understand the value that I created something that accelerates our timeline and that got people excited about AI. I mean, can you imagine? Like, I installed OpenClaw on one of my, I’m sorry, normie friends. I’m sorry, Vahan. But he’s just a… You know?
Peter Steinberger
Like, he’s-
Like, he’s-
Lex Fridman
Normie with love, yeah. For sure.
Normie with love, yeah. For sure.
Peter Steinberger
He- he, like, someone who uses the computer, but never really… Like, yeah, use some ChatGPT sometimes, but not very technical. Wouldn’t really understand what I built. So, like, I’ll show you, and I- I paid for him the- the 90 buck, 100 buck, I don’t know, subscription for Entropic. And set up everything for him with, like, WSL Windows.
He- he, like, someone who uses the computer, but never really… Like, yeah, use some ChatGPT sometimes, but not very technical. Wouldn’t really understand what I built. So, like, I’ll show you, and I- I paid for him the- the 90 buck, 100 buck, I don’t know, subscription for Entropic. And set up everything for him with, like, WSL Windows.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
I was also curious, would it actually work on Windows, you know? Was a little early. And then within a few days, he was hooked. Like, he texted me about all the things he learned. He built, like, even little tools. He’s not a programmer. And then within a few days he upgraded to the $200 subscription. Or euros, because he’s in Austria…. and he was in love with that thing. That, for me, was like a very early product validation. It’s like, I built something that captures people. And then, a few days later, Entropic blocked him because, based on their rules using the subscription is problematic or whatever. And he was, like, devastated. And then he signed up for Mini Max for 10 bucks a month and uses that.
I was also curious, would it actually work on Windows, you know? Was a little early. And then within a few days, he was hooked. Like, he texted me about all the things he learned. He built, like, even little tools. He’s not a programmer. And then within a few days he upgraded to the $200 subscription. Or euros, because he’s in Austria…. and he was in love with that thing. That, for me, was like a very early product validation. It’s like, I built something that captures people. And then, a few days later, Entropic blocked him because, based on their rules using the subscription is problematic or whatever. And he was, like, devastated. And then he signed up for Mini Max for 10 bucks a month and uses that.
Peter Steinberger
And I think that’s silly in many ways, because you just got a 200 buck customer. You just made someone hate your company, and we are still so early. Like, we don’t even know what the final form is. Is it gonna be cloud code? Probably not, you know? Like, that seems very… It seems very short-sighted to lock down your product so much. All the other companies have been helpful. I- I’m in Slack of, of most of the big labs. Kind of everybody understands that we are still in an era of exploration, in the area of the radio shows on TV and not, and not a modern TV show that fully uses the format.
And I think that’s silly in many ways, because you just got a 200 buck customer. You just made someone hate your company, and we are still so early. Like, we don’t even know what the final form is. Is it gonna be cloud code? Probably not, you know? Like, that seems very… It seems very short-sighted to lock down your product so much. All the other companies have been helpful. I- I’m in Slack of, of most of the big labs. Kind of everybody understands that we are still in an era of exploration, in the area of the radio shows on TV and not, and not a modern TV show that fully uses the format.
Lex Fridman
I think, I think you’ve made a lot of people, like, see the possibility. And non- Uh, sorry. Non, non-technical people see the possibility of AI, and just fall in love with this idea, and enjoy interacting with AI. And that’s a bea- That’s a really beautiful thing. I think I also speak for a lot of people in saying, I think you’re one of the, the great people in AI in terms of having a good heart, good vibes, humor, the right spirit. And so it would, in a sense, this model that you’re describing, having open source part, and you being part of uh, also building a thing inside, additionally, of a large company would be great, because it’s great to have good people in those companies.
I think, I think you’ve made a lot of people, like, see the possibility. And non- Uh, sorry. Non, non-technical people see the possibility of AI, and just fall in love with this idea, and enjoy interacting with AI. And that’s a bea- That’s a really beautiful thing. I think I also speak for a lot of people in saying, I think you’re one of the, the great people in AI in terms of having a good heart, good vibes, humor, the right spirit. And so it would, in a sense, this model that you’re describing, having open source part, and you being part of uh, also building a thing inside, additionally, of a large company would be great, because it’s great to have good people in those companies.
Peter Steinberger
Yeah. You know, what also people don’t really see is… I made this in three months. I did other things as well. You know, I have a lot of projects. Like, this is not… Yeah, in January, this was my main focus because I saw the storm coming. But before that, I built a whole bunch of other things. Um, I have so many ideas. Some should be there, some would be much better fitted when I have access to the latest toys- Uh, and I, I kind of want to have access to, like, the latest toys. So this is important, this is cool, this will continue to exist. My, my short-term focus is, like, working through those… Is it two- Is it 3,000 PRs now by now? I don’t even know. Like, there’s, there’s a little bit of backlog.
Yeah. You know, what also people don’t really see is… I made this in three months. I did other things as well. You know, I have a lot of projects. Like, this is not… Yeah, in January, this was my main focus because I saw the storm coming. But before that, I built a whole bunch of other things. Um, I have so many ideas. Some should be there, some would be much better fitted when I have access to the latest toys- Uh, and I, I kind of want to have access to, like, the latest toys. So this is important, this is cool, this will continue to exist. My, my short-term focus is, like, working through those… Is it two- Is it 3,000 PRs now by now? I don’t even know. Like, there’s, there’s a little bit of backlog.
Peter Steinberger
But this is not gonna be the thing that I’m gonna work until I’m, I’m, I’m 80, you know? This is… This is a window into the future. I’m gonna make this into a cool product. But yeah, I have like… I have more ideas.
But this is not gonna be the thing that I’m gonna work until I’m, I’m, I’m 80, you know? This is… This is a window into the future. I’m gonna make this into a cool product. But yeah, I have like… I have more ideas.
Lex Fridman
If you had to pick, is there a company you lean? So Meta, OpenAI, is there one you lean towards going?
If you had to pick, is there a company you lean? So Meta, OpenAI, is there one you lean towards going?
Peter Steinberger
I spend time with both of those. And it’s funny, because a few weeks ago, I didn’t consider any of this. Um… And it’s really fucking hard. Like-
I spend time with both of those. And it’s funny, because a few weeks ago, I didn’t consider any of this. Um… And it’s really fucking hard. Like-
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I have some… I know no people at OpenAI. I love their tech. I think I’m the biggest codex advertisement shill that’s unpaid. And it would feel so gratifying to, like, put a price on all the work I did for free. And I would love if something happens and those companies get just merged, because it’s like…
I have some… I know no people at OpenAI. I love their tech. I think I’m the biggest codex advertisement shill that’s unpaid. And it would feel so gratifying to, like, put a price on all the work I did for free. And I would love if something happens and those companies get just merged, because it’s like…
Lex Fridman
Is this the hardest decision you’ve ever had to do?
Is this the hardest decision you’ve ever had to do?
Peter Steinberger
No. You know, I had some breakups in the past that feel like it’s the same level.
No. You know, I had some breakups in the past that feel like it’s the same level.
Lex Fridman
Relationships, you mean?
Relationships, you mean?
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Yeah, yeah, yeah, yeah.
Yeah, yeah, yeah, yeah.
Peter Steinberger
And, and I also know that, in the end, they’re both amazing. I cannot go wrong. This is like-
And, and I also know that, in the end, they’re both amazing. I cannot go wrong. This is like-
Lex Fridman
Right.
Right.
Peter Steinberger
This is, like, one of the most prestigious and, and, and, and, and largest… I mean, not largest, but, like, they’re both very cool companies.
This is, like, one of the most prestigious and, and, and, and, and largest… I mean, not largest, but, like, they’re both very cool companies.
Lex Fridman
Yeah, they both really know scale. So, if you’re thinking about impact, some of the wonderful technologies you’ve been exploring, how to do it securely, and how to do it at scale, such that you can have a positive impact on a large number of people. They both understand that.
Yeah, they both really know scale. So, if you’re thinking about impact, some of the wonderful technologies you’ve been exploring, how to do it securely, and how to do it at scale, such that you can have a positive impact on a large number of people. They both understand that.
Peter Steinberger
You know, both Ned and Mark basically played all week with my product, and sent me like, “Oh, this is great.” Or, “This is shit. Oh, I need to change this.” Or, like, funny little anecdotes. And people using your stuff is kind of like the biggest compliment, and also shows me that, you know, they actually… T- they actually care about it. And I didn’t get the same on the OpenAI side. Um, I got… I got to see some other stuff that I find really cool, and they lure me with… I cannot tell you the exact number because of NDA, but you can, you can be creative and, and think of the Cerebras deal and how that would translate into speed. And it was very intriguing. You know, like, you give me Thor’s hammer. Yeah. … been lured with tokens. So, yeah.
You know, both Ned and Mark basically played all week with my product, and sent me like, “Oh, this is great.” Or, “This is shit. Oh, I need to change this.” Or, like, funny little anecdotes. And people using your stuff is kind of like the biggest compliment, and also shows me that, you know, they actually… T- they actually care about it. And I didn’t get the same on the OpenAI side. Um, I got… I got to see some other stuff that I find really cool, and they lure me with… I cannot tell you the exact number because of NDA, but you can, you can be creative and, and think of the Cerebras deal and how that would translate into speed. And it was very intriguing. You know, like, you give me Thor’s hammer. Yeah. … been lured with tokens. So, yeah.
Lex Fridman
So, it- it’s funny. So, so Marc started tinkering with the thing, essentially having fun with the thing.
So, it- it’s funny. So, so Marc started tinkering with the thing, essentially having fun with the thing.
Peter Steinberger
He got… He… Like, when he first… When he first approached me, I got him in my, in my WhatsApp and he was asking, “Hey, when are we have a call?” And I’m like, “I don’t like calendar entries. Let’s just call now.” And he was like, “Yeah, give me 10 minutes, I need to finish coding.”
He got… He… Like, when he first… When he first approached me, I got him in my, in my WhatsApp and he was asking, “Hey, when are we have a call?” And I’m like, “I don’t like calendar entries. Let’s just call now.” And he was like, “Yeah, give me 10 minutes, I need to finish coding.”
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Well, I guess that gives you street cred. It’s like, ugh, like, he’s still writing code. You know, he’s-
Well, I guess that gives you street cred. It’s like, ugh, like, he’s still writing code. You know, he’s-
Lex Fridman
Yeah, he does
Yeah, he does
Peter Steinberger
… he didn’t drift away in just being a manager, he gets me. That was a good first start. And then I think we had a, like, a 10-minute fight what’s better, cloud code or Codex. Like, that’s the thing you first do, like, you casually call-
… he didn’t drift away in just being a manager, he gets me. That was a good first start. And then I think we had a, like, a 10-minute fight what’s better, cloud code or Codex. Like, that’s the thing you first do, like, you casually call-
Lex Fridman
Yeah, that’s awesome
Yeah, that’s awesome
Peter Steinberger
… someone with, like, the- that owns one of the largest companies in the world and, and you have a 10 minutes conversation about that.
… someone with, like, the- that owns one of the largest companies in the world and, and you have a 10 minutes conversation about that.
Lex Fridman
Yeah, yeah.
Yeah, yeah.
Peter Steinberger
And then I think afterwards he called me eccentric but brilliant. But I also had some… I had some really, really cool discussion with Sam Altman and he’s, he’s very thoughtful brilliant and I like him a lot from the, from the little time I had, yeah. I mean, I know it’s peop- some people vilify both of those people. I don’t think it’s fair.
And then I think afterwards he called me eccentric but brilliant. But I also had some… I had some really, really cool discussion with Sam Altman and he’s, he’s very thoughtful brilliant and I like him a lot from the, from the little time I had, yeah. I mean, I know it’s peop- some people vilify both of those people. I don’t think it’s fair.
Lex Fridman
I think no matter what the stuff you’re building and the kind of human you are doing stuff at scale is kinda awesome. I’m excited.
I think no matter what the stuff you’re building and the kind of human you are doing stuff at scale is kinda awesome. I’m excited.
Peter Steinberger
I am super pumped. And you know the beauty is if, if it doesn’t work out, I can just do my own thing again. Like, I, I told them, like, I, I don’t do this for the money, I don’t give a fuck. I-
I am super pumped. And you know the beauty is if, if it doesn’t work out, I can just do my own thing again. Like, I, I told them, like, I, I don’t do this for the money, I don’t give a fuck. I-
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I mean, of course, of course it’s a nice compliment but I wanna have fun and have impact, and that’s ultimately what made my decision.
I mean, of course, of course it’s a nice compliment but I wanna have fun and have impact, and that’s ultimately what made my decision.
How OpenClaw works
Lex Fridman
Can I ask you about… we’ve talked about it quite a bit, but maybe just zooming out about how OpenCloud works. We talked about different components, I want to ask if there’s some interesting stuff we missed. So, there’s the gateway, there’s the chat clients, there’s the harness there’s the agentic loop. You said somewhere that everybody should im- implement an agent loop at some point in their lives.
Can I ask you about… we’ve talked about it quite a bit, but maybe just zooming out about how OpenCloud works. We talked about different components, I want to ask if there’s some interesting stuff we missed. So, there’s the gateway, there’s the chat clients, there’s the harness there’s the agentic loop. You said somewhere that everybody should im- implement an agent loop at some point in their lives.
Peter Steinberger
Yeah, because it’s like the, it’s like the Hello World in AI, you know? And it’s actually quite simple.
Yeah, because it’s like the, it’s like the Hello World in AI, you know? And it’s actually quite simple.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And it- it’s good to understand that that stuff’s not magic. You can, you can easily build it yourself. So, writing your own little cloud code… I, I even did this at a conference in Paris for people to, like, introduce them to AI. I think it’s it’s a fun little practice. And you, you covered a lot. I think one, one silly idea I had that turned out to be quite cool is I built this thing with full system access. So it’s like, you know, with great power comes great responsibility.
And it- it’s good to understand that that stuff’s not magic. You can, you can easily build it yourself. So, writing your own little cloud code… I, I even did this at a conference in Paris for people to, like, introduce them to AI. I think it’s it’s a fun little practice. And you, you covered a lot. I think one, one silly idea I had that turned out to be quite cool is I built this thing with full system access. So it’s like, you know, with great power comes great responsibility.
Peter Steinberger
And I was like, “How can I up the stakes a little bit more?”
And I was like, “How can I up the stakes a little bit more?”
Lex Fridman
Yeah, right.
Yeah, right.
Peter Steinberger
And I just made a… I made it proactive. So, I added a prompt. Initially, it was just a prompt, surprise me. Every, like, half an hour, surprise me, you know? And later on I changed it to be like a little more specific and-
And I just made a… I made it proactive. So, I added a prompt. Initially, it was just a prompt, surprise me. Every, like, half an hour, surprise me, you know? And later on I changed it to be like a little more specific and-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… in the definition of surprise. But the fact that I made it proactive and that it knows you and that it cares about you, it- it’s at least it’s programmed to that, prompted to do that. And that, that is a follow on, on your current session makes it very interesting because it would just sometimes ask a follow-up question or like, “How’s your day?”
… in the definition of surprise. But the fact that I made it proactive and that it knows you and that it cares about you, it- it’s at least it’s programmed to that, prompted to do that. And that, that is a follow on, on your current session makes it very interesting because it would just sometimes ask a follow-up question or like, “How’s your day?”
Lex Fridman
Yeah, right.
Yeah, right.
Peter Steinberger
And I just made a… I made it proactive. So, I added a prompt. Initially, it was just a prompt, surprise me. Every, like, half an hour, surprise me, you know? And later on I changed it to be like a little more specific and-
And I just made a… I made it proactive. So, I added a prompt. Initially, it was just a prompt, surprise me. Every, like, half an hour, surprise me, you know? And later on I changed it to be like a little more specific and-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… in the definition of surprise. But the fact that I made it proactive and that it knows you and that it cares about you, it- it’s… at least it’s programmed to that, prompted to do that. And that, that is a follow on, on your current session makes it very interesting because it would just sometimes ask a follow-up question or like, “How’s your day?” I mean, again, it’s a little creepy or weird or interesting but Heartbeat very… in the beginning, it’s still… today, it doesn’t… the model doesn’t choose to use it a lot.
… in the definition of surprise. But the fact that I made it proactive and that it knows you and that it cares about you, it- it’s… at least it’s programmed to that, prompted to do that. And that, that is a follow on, on your current session makes it very interesting because it would just sometimes ask a follow-up question or like, “How’s your day?” I mean, again, it’s a little creepy or weird or interesting but Heartbeat very… in the beginning, it’s still… today, it doesn’t… the model doesn’t choose to use it a lot.
Lex Fridman
By the way, we’re, we’re, we’re talking about Heartbeat, as you mentioned, the thing that regularly-
By the way, we’re, we’re, we’re talking about Heartbeat, as you mentioned, the thing that regularly-
Peter Steinberger
Yeah. Like kicks-
Yeah. Like kicks-
Lex Fridman
… Acts.
… Acts.
Peter Steinberger
You just kick off the loop.
You just kick off the loop.
Lex Fridman
Isn’t that just a cron job, man?
Isn’t that just a cron job, man?
Peter Steinberger
Yeah, right, I mean, it’s like-
Yeah, right, I mean, it’s like-
Lex Fridman
It’s the cr- the criticisms that you get are hilarious.
It’s the cr- the criticisms that you get are hilarious.
Peter Steinberger
You can, you can deduce any idea to like a silly… Yeah, it’s just, it’s just a cron job in the end. I have like cron- separate cron jobs.
You can, you can deduce any idea to like a silly… Yeah, it’s just, it’s just a cron job in the end. I have like cron- separate cron jobs.
Lex Fridman
Isn’t love just evolutionary biology manifesting itself and isn’t… aren’t you guys just using each other?
Isn’t love just evolutionary biology manifesting itself and isn’t… aren’t you guys just using each other?
Peter Steinberger
And then, yeah, and the project is all just glue of a few different dependencies-
And then, yeah, and the project is all just glue of a few different dependencies-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… and there’s nothing original. Why do people… Well, you know, isn’t Dropbox just FTP with extra steps?
… and there’s nothing original. Why do people… Well, you know, isn’t Dropbox just FTP with extra steps?
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
I found it surprising where I had this I had a shoulder operation a few months ago, so.
I found it surprising where I had this I had a shoulder operation a few months ago, so.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And the model rarely used Heartbeat, but then I was in the hospital, and it knew that I had the operation and it checked up on me. It’s like, “Are you okay?” And I just… It’s like, again, apparently, like, if something’s significant in the context, that triggered the Heartbeat when it rarely used the Heartbeat…. And it does that sometimes for people, and that just makes it a lot more relatable.
And the model rarely used Heartbeat, but then I was in the hospital, and it knew that I had the operation and it checked up on me. It’s like, “Are you okay?” And I just… It’s like, again, apparently, like, if something’s significant in the context, that triggered the Heartbeat when it rarely used the Heartbeat…. And it does that sometimes for people, and that just makes it a lot more relatable.
Lex Fridman
Let me look this up on Perplexity, how OpenCall works just to see if I’m missing any of the stuff. Local agent run time, high-level architecture. There’s… Oh, we haven’t talked much about skills, I suppose. Skill hub, the tools in the skill lair, but that’s definitely a huge component and there’s a huge growing set of skills-
Let me look this up on Perplexity, how OpenCall works just to see if I’m missing any of the stuff. Local agent run time, high-level architecture. There’s… Oh, we haven’t talked much about skills, I suppose. Skill hub, the tools in the skill lair, but that’s definitely a huge component and there’s a huge growing set of skills-
Peter Steinberger
You know, you know what I love? That half a year ago, like everyone was talking about MCPs-
You know, you know what I love? That half a year ago, like everyone was talking about MCPs-
Lex Fridman
Yeah
Yeah
Peter Steinberger
… and I was like, “Screw MCPs. Every MCP would be better as a CLI.” And now this stuff doesn’t even have MCP support. I mean, it, it has with asterisks, but not in the core lair, and nobody’s complaining.
… and I was like, “Screw MCPs. Every MCP would be better as a CLI.” And now this stuff doesn’t even have MCP support. I mean, it, it has with asterisks, but not in the core lair, and nobody’s complaining.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
So my approach is if you want to extend the model with more features, you just build a CLI and the model can call the CLI, probably gets it wrong, calls the help menu, and then on demand loads into the context what it needs to use the CLI. It just needs a sentence to know that the CLI exists if it’s something that the model doesn’t know about default. And even for a while, I, I didn’t really care about skills, but skills are actually perfect for that because they, they boil down to a single sentence that explains the skill and then the model loads the skill, and that explains the CLI, and then the model uses the CLI. Some skills are, like raw, but most of the time, networks.
So my approach is if you want to extend the model with more features, you just build a CLI and the model can call the CLI, probably gets it wrong, calls the help menu, and then on demand loads into the context what it needs to use the CLI. It just needs a sentence to know that the CLI exists if it’s something that the model doesn’t know about default. And even for a while, I, I didn’t really care about skills, but skills are actually perfect for that because they, they boil down to a single sentence that explains the skill and then the model loads the skill, and that explains the CLI, and then the model uses the CLI. Some skills are, like raw, but most of the time, networks.
Lex Fridman
It’s interesting um, I’m asking Perplexity MCP versus skills, because this kind of requires a hot take that’s quite recent, because your general view is MCPs are dead-ish. So MCPs is a more structured thing. So if you listen to Perplexity here, MCP is what can I reach? So APIs, database services files via protocol. So a structured protocol of how you communicate with a thing, and then skills is more how should I work? Procedures, hostile helper scripts and prompts are often written in a kind of semi-structured natural language, right? And so technically skills could replace MCP if you have a smart enough model.
It’s interesting um, I’m asking Perplexity MCP versus skills, because this kind of requires a hot take that’s quite recent, because your general view is MCPs are dead-ish. So MCPs is a more structured thing. So if you listen to Perplexity here, MCP is what can I reach? So APIs, database services files via protocol. So a structured protocol of how you communicate with a thing, and then skills is more how should I work? Procedures, hostile helper scripts and prompts are often written in a kind of semi-structured natural language, right? And so technically skills could replace MCP if you have a smart enough model.
Peter Steinberger
I think the main beauty is, is that models are really good at calling Unix commands. So if you just add another CLI, that’s just another Unix command in the end. And MCP is… That has to be added in training. That’s not a very natural thing for the model. It requires a very specific syntax. And the biggest thing, it’s not composable. So imagine if I have a service that gives me better data and gives me the temperature, the average temperature, rain, wind and all the other stuff, and I get like this huge blob back. As a model, I always have to get the huge blob back. I have to fill my context with that huge blob and then pick what I want. There’s no way for the model to naturally filter unless I think about it proactively and add a filtering way into my MCP.
I think the main beauty is, is that models are really good at calling Unix commands. So if you just add another CLI, that’s just another Unix command in the end. And MCP is… That has to be added in training. That’s not a very natural thing for the model. It requires a very specific syntax. And the biggest thing, it’s not composable. So imagine if I have a service that gives me better data and gives me the temperature, the average temperature, rain, wind and all the other stuff, and I get like this huge blob back. As a model, I always have to get the huge blob back. I have to fill my context with that huge blob and then pick what I want. There’s no way for the model to naturally filter unless I think about it proactively and add a filtering way into my MCP.
Peter Steinberger
But if I would build the same as a CLI and it would give me this huge blob, it could just add a JQ command and filter itself and then only, only get me what I actually need. Or maybe even compose it into a script to, like do some calculations with the temperature and only give me the exact output and the mo- and the… you have no context pollution. Again, you can solve that with like sub-agents and more charades, but it’s just like workarounds for something that might not be the optimal way. There’s… It definitely it was, you know, it was good that we had MCPs because it pushed a lot of companies towards building APIs and now I, I can like look at an MCP and just make it into a CLI.
But if I would build the same as a CLI and it would give me this huge blob, it could just add a JQ command and filter itself and then only, only get me what I actually need. Or maybe even compose it into a script to, like do some calculations with the temperature and only give me the exact output and the mo- and the… you have no context pollution. Again, you can solve that with like sub-agents and more charades, but it’s just like workarounds for something that might not be the optimal way. There’s… It definitely it was, you know, it was good that we had MCPs because it pushed a lot of companies towards building APIs and now I, I can like look at an MCP and just make it into a CLI.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
But this, this inherent problem that MCPs by default clutter up your context. Plus the fact that most MCPs are not made good, in general make it just not a very useful paradigm. There’s some exceptions like Playwright for example that requires state and it’s actually useful. That is an acceptable choice.
But this, this inherent problem that MCPs by default clutter up your context. Plus the fact that most MCPs are not made good, in general make it just not a very useful paradigm. There’s some exceptions like Playwright for example that requires state and it’s actually useful. That is an acceptable choice.
Lex Fridman
So Playwright you use for browser use, which I think is c- already in OpenClaw is quite incredible, right?
So Playwright you use for browser use, which I think is c- already in OpenClaw is quite incredible, right?
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
You can basically do everything, most things you can think of using browser use.
You can basically do everything, most things you can think of using browser use.
Peter Steinberger
That, that gets into the whole arch of every app is just a very slow API now, if they want or not. And that through personal agents a lot of apps will disappear. You know, like I had a… I built a CLI for Twitter. I mean, I- I just reverse engineered their website and used the internal API, which is not very allowed.
That, that gets into the whole arch of every app is just a very slow API now, if they want or not. And that through personal agents a lot of apps will disappear. You know, like I had a… I built a CLI for Twitter. I mean, I- I just reverse engineered their website and used the internal API, which is not very allowed.
Lex Fridman
It’s called Bird, short-lived.
It’s called Bird, short-lived.
Peter Steinberger
It was called Bird, because the bird had to disappear.
It was called Bird, because the bird had to disappear.
Lex Fridman
The, the wings were clipped.
The, the wings were clipped.
Peter Steinberger
All they did is they just made access slower. Yeah, not tak- you’re not actually taking a feature away, but now inst- if, if your agent wants to read a tweet, it actually has to open the browser and read the tweet. And it will still be able to read the tweet. It will just take longer. It’s not like you are making something that was possible, not possible. No. Now, it’s just taking… Now it’s just a bit slower. So, so it doesn’t really matter if your service wants to be an API or not. If I can access it in the browser…… easy API. It’s a slow API.
All they did is they just made access slower. Yeah, not tak- you’re not actually taking a feature away, but now inst- if, if your agent wants to read a tweet, it actually has to open the browser and read the tweet. And it will still be able to read the tweet. It will just take longer. It’s not like you are making something that was possible, not possible. No. Now, it’s just taking… Now it’s just a bit slower. So, so it doesn’t really matter if your service wants to be an API or not. If I can access it in the browser…… easy API. It’s a slow API.
Lex Fridman
Can you empathize with their situation? Like, what would you do if you were Twitter, if you were X? Because they’re basically trying to protect against other large companies scraping all their data.
Can you empathize with their situation? Like, what would you do if you were Twitter, if you were X? Because they’re basically trying to protect against other large companies scraping all their data.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
But in so doing, they’re cutting off like a million different use cases for smaller developers that actually want to use it for helpful cool stuff.
But in so doing, they’re cutting off like a million different use cases for smaller developers that actually want to use it for helpful cool stuff.
Peter Steinberger
I think that if you have a very low per day baseline per account that allows read-only access would solve a lot of problems. There’s plenty, plenty of automations where people create a bookmark and then use OpenClaw to, like, find the bookmark, do research on it, and then send you an email-
I think that if you have a very low per day baseline per account that allows read-only access would solve a lot of problems. There’s plenty, plenty of automations where people create a bookmark and then use OpenClaw to, like, find the bookmark, do research on it, and then send you an email-
Lex Fridman
Mm-hmm
Mm-hmm
Peter Steinberger
… with, like, more details on it or a summary. That’s a cool approach. I also want all my bookmarks somewhere to search. I would still like to have that.
… with, like, more details on it or a summary. That’s a cool approach. I also want all my bookmarks somewhere to search. I would still like to have that.
Lex Fridman
So, read-only access for the bookmarks you make on X. That seems like an incredible application because a lot of us find a lot of cool stuff on X, we bookmark, that’s the general purpose of X. It’s like, holy shit, this is awesome. Oftentimes, you bookmark so many things you never look back at them.
So, read-only access for the bookmarks you make on X. That seems like an incredible application because a lot of us find a lot of cool stuff on X, we bookmark, that’s the general purpose of X. It’s like, holy shit, this is awesome. Oftentimes, you bookmark so many things you never look back at them.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
It would be nice to have tooling that organizes them and allows you to research it further.
It would be nice to have tooling that organizes them and allows you to research it further.
Peter Steinberger
Yeah, I mean, and to be frank, I, I mean, I, I told Twitter proactively that, “Hey, I built this and there’s a need.” And they’ve been really nice, but also like, “Take it down.” Fair. Totally fair. But I hope that this woke up the team a little bit that there’s a need. And if all you do is making it slower, you’re just reducing access to your platform. I’m sure there’s a better way. I also, I’m very much against any automation on Twitter. If you tweet at me with AI, I will block you. No first strike. As soon as it smells like AI, and AI still has a smell.
Yeah, I mean, and to be frank, I, I mean, I, I told Twitter proactively that, “Hey, I built this and there’s a need.” And they’ve been really nice, but also like, “Take it down.” Fair. Totally fair. But I hope that this woke up the team a little bit that there’s a need. And if all you do is making it slower, you’re just reducing access to your platform. I’m sure there’s a better way. I also, I’m very much against any automation on Twitter. If you tweet at me with AI, I will block you. No first strike. As soon as it smells like AI, and AI still has a smell.
AI slop
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Especially on tweets. It’s very hard to tweet in a way that does look completely human.
Especially on tweets. It’s very hard to tweet in a way that does look completely human.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And then I block. Like, I have a zero tolerance policy on that. And I think it would be very helpful if they, if, like, tweets done via API would be marked. Maybe there’s some special cases where… But, and there should be, there should be a very easy way for agents to get their own Twitter account. Um…
And then I block. Like, I have a zero tolerance policy on that. And I think it would be very helpful if they, if, like, tweets done via API would be marked. Maybe there’s some special cases where… But, and there should be, there should be a very easy way for agents to get their own Twitter account. Um…
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
We, we need to rethink social platforms a little bit if, if, if we, we, we go towards a future where everyone has their agent and agents maybe have their own Instagram profiles or Twitter accounts, so I can, like, do stuff on my behalf. I think it should very clearly be marked that they are doing stuff on my behalf and it’s not me. Because content is now so cheap. Eyeballs are the expensive part. And I find it very triggering when I read something and then I’m like, oh, no, this smells like AI.
We, we need to rethink social platforms a little bit if, if, if we, we, we go towards a future where everyone has their agent and agents maybe have their own Instagram profiles or Twitter accounts, so I can, like, do stuff on my behalf. I think it should very clearly be marked that they are doing stuff on my behalf and it’s not me. Because content is now so cheap. Eyeballs are the expensive part. And I find it very triggering when I read something and then I’m like, oh, no, this smells like AI.
Lex Fridman
Yeah. Like, where, where is this headed in terms of what we value about the human experience? It feels like we’ll, we’ll move more and more towards in-person interaction and we’ll just communicate. We’ll talk to our AI agent to, to accomplish different tasks, to learn about different things, but we won’t value online interaction because there’ll be so much AI slob that smells and so many bots that it’s difficult.
Yeah. Like, where, where is this headed in terms of what we value about the human experience? It feels like we’ll, we’ll move more and more towards in-person interaction and we’ll just communicate. We’ll talk to our AI agent to, to accomplish different tasks, to learn about different things, but we won’t value online interaction because there’ll be so much AI slob that smells and so many bots that it’s difficult.
Peter Steinberger
Well, if it’s smart, then it shouldn’t be difficult to filter. And then I can look at it if I want to. But yeah, this is, like, a big thing we need to solve right now. E- especially on this project, I get so many emails that are, let’s say nicely, agentically written.
Well, if it’s smart, then it shouldn’t be difficult to filter. And then I can look at it if I want to. But yeah, this is, like, a big thing we need to solve right now. E- especially on this project, I get so many emails that are, let’s say nicely, agentically written.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
But I much rather read your broken English than your AI slob. You know, of course there’s a human behind it, and yet they, they prompt it. I’d much rather read your prompt than what came out. Um, I think we’re reaching a point where I value typos again.
But I much rather read your broken English than your AI slob. You know, of course there’s a human behind it, and yet they, they prompt it. I’d much rather read your prompt than what came out. Um, I think we’re reaching a point where I value typos again.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Like… Like, and I, I mean, it also took me a while to, like, come to the realization. I, on my blog I experimented with creating a blog post with agents and ultimately it took me about the same time to, like, steer agent towards something I like. But it missed the nuances that, how I would write it. You know, you can like, you can steer it towards your style, but it’s not gonna be all your style. So, I, I completely moved away from that. I, I, everything, everything I blog is organic, handwritten and maybe, maybe I, I, I use AI as a fix my worse typos. But there’s value in the rough parts of an actual human.
Like… Like, and I, I mean, it also took me a while to, like, come to the realization. I, on my blog I experimented with creating a blog post with agents and ultimately it took me about the same time to, like, steer agent towards something I like. But it missed the nuances that, how I would write it. You know, you can like, you can steer it towards your style, but it’s not gonna be all your style. So, I, I completely moved away from that. I, I, everything, everything I blog is organic, handwritten and maybe, maybe I, I, I use AI as a fix my worse typos. But there’s value in the rough parts of an actual human.
Lex Fridman
Isn’t that awesome? Isn’t that beautiful? That now because of AI we value the raw humanity in each of us more.
Isn’t that awesome? Isn’t that beautiful? That now because of AI we value the raw humanity in each of us more.
Peter Steinberger
I also, I also realized this thing that I, I rave about AI and use it so much for anything that’s code, but I’m allergic if it’s stories.
I also, I also realized this thing that I, I rave about AI and use it so much for anything that’s code, but I’m allergic if it’s stories.
Lex Fridman
Right. Yeah.
Right. Yeah.
Peter Steinberger
Also, documentation, still fine with AI. You know, better than nothing.
Also, documentation, still fine with AI. You know, better than nothing.
Lex Fridman
And for now it’s still i- it applies in the mi- in the visual medium too. It’s fascinating how allergic I am to even a little bit of AI slob in in video and images. It’s useful, it’s nice if it’s like a little component of like-
And for now it’s still i- it applies in the mi- in the visual medium too. It’s fascinating how allergic I am to even a little bit of AI slob in in video and images. It’s useful, it’s nice if it’s like a little component of like-
Peter Steinberger
Or even, even those images. The, like, all these infographics and stuff, the-… they trigger me so hard.
Or even, even those images. The, like, all these infographics and stuff, the-… they trigger me so hard.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Like, it immediately makes me think less of your content. And it … They were novel for, like, one week and now it just screams slop.
Like, it immediately makes me think less of your content. And it … They were novel for, like, one week and now it just screams slop.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
Even- even if people work hard on it, using … And I- I have some on my blog post, you know, in the- in the time where I- I explored this new medium. But now, they trigger me as well. It’s like, yeah, this is … This just screams AI slop. I-
Even- even if people work hard on it, using … And I- I have some on my blog post, you know, in the- in the time where I- I explored this new medium. But now, they trigger me as well. It’s like, yeah, this is … This just screams AI slop. I-
Lex Fridman
What… I don’t know what that is, but I went through that too. I was really excited by the diagrams. And then I realized, in order to remove from them hallucinations, you actually have to do a huge amount of work. And you’re just using it to draw the better diagrams, great. And then I’m proud of the diagram. I’ve used them for literally, like, ki- ki- kind of like you said for maybe a couple of weeks. And now I look at those, and I- I feel like I feel when I look at Comic Sans as a font or- or something like this.
What… I don’t know what that is, but I went through that too. I was really excited by the diagrams. And then I realized, in order to remove from them hallucinations, you actually have to do a huge amount of work. And you’re just using it to draw the better diagrams, great. And then I’m proud of the diagram. I’ve used them for literally, like, ki- ki- kind of like you said for maybe a couple of weeks. And now I look at those, and I- I feel like I feel when I look at Comic Sans as a font or- or something like this.
Lex Fridman
It’s like, “No, this is-“
It’s like, “No, this is-“
Peter Steinberger
It’s a smell.
It’s a smell.
Lex Fridman
“… this is fake. It’s fraudulent. There’s something wrong with it.” And it…
“… this is fake. It’s fraudulent. There’s something wrong with it.” And it…
Peter Steinberger
It’s a smell.
It’s a smell.
Lex Fridman
It’s a smell.
It’s a smell.
Peter Steinberger
It’s a smell.
It’s a smell.
Lex Fridman
And it’s awesome because it re- it reminds you that we know. There’s so much to humans that’s amazing and we know that. And we- we know it. We know it when we see it. And so that gives me a lot of hope, you know? That gives me a lot of hope about the human experience. It’s not going to be damaged by … It’s only going to be empowered as tools by AI. It’s not going to be damaged or limited or somehow altered to where it’s no longer human. So … Uh, I need a bathroom break. Quick pause. You mentioned that a lot of the apps might be basically made obsolete. Do you think agents will just transform the entire app market?
And it’s awesome because it re- it reminds you that we know. There’s so much to humans that’s amazing and we know that. And we- we know it. We know it when we see it. And so that gives me a lot of hope, you know? That gives me a lot of hope about the human experience. It’s not going to be damaged by … It’s only going to be empowered as tools by AI. It’s not going to be damaged or limited or somehow altered to where it’s no longer human. So … Uh, I need a bathroom break. Quick pause. You mentioned that a lot of the apps might be basically made obsolete. Do you think agents will just transform the entire app market?
AI agents will replace 80% of apps
Peter Steinberger
Yeah. Uh, I noticed that on Discord, that people just said how their … like, what they build and what they use it for. And it’s like, why do you need MyFitnessPal when the agent already knows where I am? So, it can assume that I make bad decisions when I’m at, I don’t know, Waffle House, what’s around here? Or- or briskets in Austin.
Yeah. Uh, I noticed that on Discord, that people just said how their … like, what they build and what they use it for. And it’s like, why do you need MyFitnessPal when the agent already knows where I am? So, it can assume that I make bad decisions when I’m at, I don’t know, Waffle House, what’s around here? Or- or briskets in Austin.
Lex Fridman
There’s no bad decisions around briskets, but yeah.
There’s no bad decisions around briskets, but yeah.
Peter Steinberger
No, that’s the best decision, honestly. Um-
No, that’s the best decision, honestly. Um-
Lex Fridman
Your agent should know that.
Your agent should know that.
Peter Steinberger
But it can, like … It can modify my- my gym workout based on how well I slept, or if I’m … if I have stress or not. Like, it has so much more context to make even better decisions than any of this app even could do.
But it can, like … It can modify my- my gym workout based on how well I slept, or if I’m … if I have stress or not. Like, it has so much more context to make even better decisions than any of this app even could do.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
It could show me UI just as I like. Why do I still need an app to do that? Why do I have to … Why should I pay another subscription for something that the agent can just do now? And why do I need my- my Eight Sleep app to control my bed when I can tell the a- … tell the agent to … You know, the agent already knows where I am, so he can, like, turn off what I don’t use.
It could show me UI just as I like. Why do I still need an app to do that? Why do I have to … Why should I pay another subscription for something that the agent can just do now? And why do I need my- my Eight Sleep app to control my bed when I can tell the a- … tell the agent to … You know, the agent already knows where I am, so he can, like, turn off what I don’t use.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And I think that will … that will translate into a whole category of apps that are no longer … I will just naturally stop using because my agent can just do it better.
And I think that will … that will translate into a whole category of apps that are no longer … I will just naturally stop using because my agent can just do it better.
Lex Fridman
I think you said somewhere that it might kill off 80% of apps.
I think you said somewhere that it might kill off 80% of apps.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Don’t you think that’s a gigantic transformative effect on just all software development? So that means it might kill off a lot of software companies.
Don’t you think that’s a gigantic transformative effect on just all software development? So that means it might kill off a lot of software companies.
Peter Steinberger
Yeah. Um-
Yeah. Um-
Lex Fridman
It’s a scary thing. So, like, do you think about the impact that has on the economy? On just the ripple effects it has to society? Transforming who builds what tooling. It empowers a lot of users to get stuff done, to get stuff more efficiently, to get it done cheaper.
It’s a scary thing. So, like, do you think about the impact that has on the economy? On just the ripple effects it has to society? Transforming who builds what tooling. It empowers a lot of users to get stuff done, to get stuff more efficiently, to get it done cheaper.
Peter Steinberger
It’s also new services that we will need, right? For example, I want my agent to have an allowance. Like, you solve problems for me, here’s like 100 bucks in order to solve problems for me. And if I tell you to order me food, maybe it uses a service. Maybe it uses something like rent-a-human to, like, just get that done for me.
It’s also new services that we will need, right? For example, I want my agent to have an allowance. Like, you solve problems for me, here’s like 100 bucks in order to solve problems for me. And if I tell you to order me food, maybe it uses a service. Maybe it uses something like rent-a-human to, like, just get that done for me.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
I don’t actually care. I care about solve my problem. There’s space for- for new companies to solve that well. Maybe don’t … Not all apps disappear. Maybe some transform into being API.
I don’t actually care. I care about solve my problem. There’s space for- for new companies to solve that well. Maybe don’t … Not all apps disappear. Maybe some transform into being API.
Lex Fridman
So, basically, apps that rapidly transform in being agent-facing. So, there’s a real opportunity for, like, Uber Eats, that we just used earlier today. It- it’s companies this, of which there’s many. Who gets there fastest to being able to interact with OpenClaw in a way that’s the m- the most natural, the easiest?
So, basically, apps that rapidly transform in being agent-facing. So, there’s a real opportunity for, like, Uber Eats, that we just used earlier today. It- it’s companies this, of which there’s many. Who gets there fastest to being able to interact with OpenClaw in a way that’s the m- the most natural, the easiest?
Peter Steinberger
Yeah. And also, apps will become API if they want or not. Because my agent can figure out how to use my phone. I mean, on- on the other side, it’s a little more tricky. On Android, that’s already … People already do that. And then we’ll just click the Order Uber for Me button for me. Or maybe another service. Or maybe there’s- there’s a … there’s an API I can call so it’s faster. Uh, I think that’s a space we’re just beginning to even understand what that means. And I … Again, I didn’t even … That was not something I thought of. Something that I- that I discovered as people use this, and it … We are still so early. But yeah, I think data is very important. Like, apps that can give me data, but that also can be API. Why do I need a Sonos app anymore when I can …
Yeah. And also, apps will become API if they want or not. Because my agent can figure out how to use my phone. I mean, on- on the other side, it’s a little more tricky. On Android, that’s already … People already do that. And then we’ll just click the Order Uber for Me button for me. Or maybe another service. Or maybe there’s- there’s a … there’s an API I can call so it’s faster. Uh, I think that’s a space we’re just beginning to even understand what that means. And I … Again, I didn’t even … That was not something I thought of. Something that I- that I discovered as people use this, and it … We are still so early. But yeah, I think data is very important. Like, apps that can give me data, but that also can be API. Why do I need a Sonos app anymore when I can …
Peter Steinberger
when my agent can talk to the Sonos?… Speakers directly. Like my cameras, there’s like a crappy app, but they have, they have an API, so my agent uses the API now.
when my agent can talk to the Sonos?… Speakers directly. Like my cameras, there’s like a crappy app, but they have, they have an API, so my agent uses the API now.
Lex Fridman
So it’s gonna force a lot of companies to have to shift focus. That’s kind of what the internet did, right? You have to rapidly rethink, reconfigure what you’re selling, how you’re making money.
So it’s gonna force a lot of companies to have to shift focus. That’s kind of what the internet did, right? You have to rapidly rethink, reconfigure what you’re selling, how you’re making money.
Peter Steinberger
Yeah, and some companies were really not like that. For example, there’s no CLI for Google, so I had to like, do… have to do anything myself and build GAWK. That’s like a CLI for Google. And at the… Yeah, at the end user, they have to give me the emails because otherwise I cannot use their product. If I’m a company and I try to get Google data, Gmail, there’s a whole complicated process, to the point where sometimes startups acquire startups that went through the process, so they don’t- don’t have to work with Google for half a year to be certified to being able to access Gmail. But my agent can access Gmail because I can just connect to it. It’s still crappy because I need to, like, go through Google’s developer jungle to get a key, and that’s still annoying.
Yeah, and some companies were really not like that. For example, there’s no CLI for Google, so I had to like, do… have to do anything myself and build GAWK. That’s like a CLI for Google. And at the… Yeah, at the end user, they have to give me the emails because otherwise I cannot use their product. If I’m a company and I try to get Google data, Gmail, there’s a whole complicated process, to the point where sometimes startups acquire startups that went through the process, so they don’t- don’t have to work with Google for half a year to be certified to being able to access Gmail. But my agent can access Gmail because I can just connect to it. It’s still crappy because I need to, like, go through Google’s developer jungle to get a key, and that’s still annoying.
Peter Steinberger
But they cannot prevent me. And worst case, my agent just clicks on the, on the website and gets the data out that way.
But they cannot prevent me. And worst case, my agent just clicks on the, on the website and gets the data out that way.
Lex Fridman
Through browsers?
Through browsers?
Peter Steinberger
Yeah. I mean, I, I watch my agent happily click the I’m not a robot button. And there’s this, this whole… That’s gonna be… That’s gonna be more heated. You see companies like Cloudflare that try to prevent bot access. And in some ways, that’s useful for scraping. But in other ways, if I’m, I’m a personal user, I want that. You know, sometimes I, I use Codex and I, I read an article about modern React patterns, and it’s like a Medium article. I paste it in and the agent can’t read it because they block it. So then I have to copy-paste the actual text. Or in the future, I’ll learn that maybe I don’t click on Medium because it’s annoying, and I use other websites that actually are agent friendly.
Yeah. I mean, I, I watch my agent happily click the I’m not a robot button. And there’s this, this whole… That’s gonna be… That’s gonna be more heated. You see companies like Cloudflare that try to prevent bot access. And in some ways, that’s useful for scraping. But in other ways, if I’m, I’m a personal user, I want that. You know, sometimes I, I use Codex and I, I read an article about modern React patterns, and it’s like a Medium article. I paste it in and the agent can’t read it because they block it. So then I have to copy-paste the actual text. Or in the future, I’ll learn that maybe I don’t click on Medium because it’s annoying, and I use other websites that actually are agent friendly.
Peter Steinberger
So, uh-
So, uh-
Lex Fridman
There’s gonna be a lot of powerful, rich companies fighting back. So it’s really intere- You’re at the center, you’re the catalyst, the leader, and happen to be at the center of this kind of revolution where it’s get- gonna completely change how we interact with services with, with web. And so, like, there’s companies at Google that are gonna push back. I mean, there’s every major companies you could think of is gonna push back.
There’s gonna be a lot of powerful, rich companies fighting back. So it’s really intere- You’re at the center, you’re the catalyst, the leader, and happen to be at the center of this kind of revolution where it’s get- gonna completely change how we interact with services with, with web. And so, like, there’s companies at Google that are gonna push back. I mean, there’s every major companies you could think of is gonna push back.
Peter Steinberger
Even… Yeah, even search. Um, I now use, I think Perplexity or Brave as providers because Google really doesn’t make it easy to use Google without Google. I’m not sure if that’s the right strategy, but I’m not Google.
Even… Yeah, even search. Um, I now use, I think Perplexity or Brave as providers because Google really doesn’t make it easy to use Google without Google. I’m not sure if that’s the right strategy, but I’m not Google.
Lex Fridman
Yeah, there’s a, there’s a nice balance from a big company perspective ’cause if you push back too much for too long, you become Blockbuster and you lose everything to the Netflixes of the world. But some pushback is probably good during a revolution to see.
Yeah, there’s a, there’s a nice balance from a big company perspective ’cause if you push back too much for too long, you become Blockbuster and you lose everything to the Netflixes of the world. But some pushback is probably good during a revolution to see.
Peter Steinberger
Yeah. But you see that, that… Like, this is something that the people want.
Yeah. But you see that, that… Like, this is something that the people want.
Lex Fridman
Right.
Right.
Peter Steinberger
So-
So-
Lex Fridman
Yes.
Yes.
Peter Steinberger
If I’m on the go, I don’t wanna open a calendar app. I just… I wanna tell my agent, “Hey, remind me about this dinner tomorrow night,” and maybe invite two of my friends and then maybe send a what- send a WhatsApp message to my friend. And I don’t need… I don’t want or need to open apps for that. I think that we passed that age, and now everything is, like, much more connected and, and fluid if those companies want it or not. And I think, well, the right companies will find ways to jump on the train, and other companies will perish.
If I’m on the go, I don’t wanna open a calendar app. I just… I wanna tell my agent, “Hey, remind me about this dinner tomorrow night,” and maybe invite two of my friends and then maybe send a what- send a WhatsApp message to my friend. And I don’t need… I don’t want or need to open apps for that. I think that we passed that age, and now everything is, like, much more connected and, and fluid if those companies want it or not. And I think, well, the right companies will find ways to jump on the train, and other companies will perish.
Will AI replace programmers?
Lex Fridman
You got to listen to what the people want. We talked about programming quite a bit, and a lot of folks that are developers are really worried about their jobs, about their… About the future of programming. Do you think AI replaces programmers completely? Human programmers?
You got to listen to what the people want. We talked about programming quite a bit, and a lot of folks that are developers are really worried about their jobs, about their… About the future of programming. Do you think AI replaces programmers completely? Human programmers?
Peter Steinberger
I mean, we’re definitely going in that direction. Programming is just a part of building products. So maybe, maybe AI does replace programmers eventually. But there’s so much more to that art. Like, what do you actually wanna build? How should it feel? How’s the architecture? I don’t think agents will replace all of that. Yeah, like, just the, the actual art of programming, it will, it will stay there, but it’s, it’s gonna be like knitting. You know? Like, people do that because they like it, not because it makes any sense. So the… I read this article this morning about someone that it’s okay to mourn our craft. And I can…
I mean, we’re definitely going in that direction. Programming is just a part of building products. So maybe, maybe AI does replace programmers eventually. But there’s so much more to that art. Like, what do you actually wanna build? How should it feel? How’s the architecture? I don’t think agents will replace all of that. Yeah, like, just the, the actual art of programming, it will, it will stay there, but it’s, it’s gonna be like knitting. You know? Like, people do that because they like it, not because it makes any sense. So the… I read this article this morning about someone that it’s okay to mourn our craft. And I can…
Peter Steinberger
A part of me very strongly resonates with that because in my past I, I spent a lot of time tinkering, just being really deep in the flow and just, like, cranking out code and, like, finding really beautiful solutions. And yes, in a way it’s, it’s sad because that will go away. And I also get a lot of joy out of just writing code and being really deep in my thoughts and forgetting time and space and just being in this beautiful state of flow. But you can get the same state of flow… I get a similar state of flow by working with agents and building and thinking really hard about problems. It is different-… but… And it’s okay to mourn it, but I mean, that’s not something we can fight. Like, there is… the world for a long time had a…
A part of me very strongly resonates with that because in my past I, I spent a lot of time tinkering, just being really deep in the flow and just, like, cranking out code and, like, finding really beautiful solutions. And yes, in a way it’s, it’s sad because that will go away. And I also get a lot of joy out of just writing code and being really deep in my thoughts and forgetting time and space and just being in this beautiful state of flow. But you can get the same state of flow… I get a similar state of flow by working with agents and building and thinking really hard about problems. It is different-… but… And it’s okay to mourn it, but I mean, that’s not something we can fight. Like, there is… the world for a long time had a…
Peter Steinberger
there was a lack of intelligence, if you s- if you see it like that, of people building things, and that’s why salaries of software developers reached stupidly high amounts and then will go away. There will still be a lot of demand for people that understand how to build things. It’s just that all this tokenized intelligence enables people to do a lot more, a lot faster. And it will be even more… even faster and even more because those things are continuously improving. We had similar things when… I mean, it’s probably not a perfect analogy, but when we created the steam engine, and they built all these factories and replaced a lot of manual labor, and then people revolted and broke the machines.
there was a lack of intelligence, if you s- if you see it like that, of people building things, and that’s why salaries of software developers reached stupidly high amounts and then will go away. There will still be a lot of demand for people that understand how to build things. It’s just that all this tokenized intelligence enables people to do a lot more, a lot faster. And it will be even more… even faster and even more because those things are continuously improving. We had similar things when… I mean, it’s probably not a perfect analogy, but when we created the steam engine, and they built all these factories and replaced a lot of manual labor, and then people revolted and broke the machines.
Peter Steinberger
Um, I- I can relate that if you very deeply identify that you are a programmer, that it’s scary and that it’s threatening because what you like and what you’re really good at is now being done by a soulless or not entity. But I don’t think you’re just a programmer. That’s a very limiting view of your craft. You are, you are still a builder.
Um, I- I can relate that if you very deeply identify that you are a programmer, that it’s scary and that it’s threatening because what you like and what you’re really good at is now being done by a soulless or not entity. But I don’t think you’re just a programmer. That’s a very limiting view of your craft. You are, you are still a builder.
Lex Fridman
Yeah, there’s a couple of things I want to say. So one is, I never… As you’re articulating this beautifully, I no- I’m realizing I never thought I would… the thing I love doing would be the thing that gets replaced. You hear these stories about these, like you said, with the steam engine. I’ve, I’ve spent so many, I don’t know, maybe thousands of hours poring over code and putting my heart and soul and, like, and just, like, some of my most painful and happiest moments were alone behind… I, I was an Emacs person for a long time. Man, Emacs. And, and then there’s an identity and there’s meaning, and there’s… Like, when I walk about the world, I don’t say it out loud, but I think of myself as a programmer. And to have that in a matter of months…
Yeah, there’s a couple of things I want to say. So one is, I never… As you’re articulating this beautifully, I no- I’m realizing I never thought I would… the thing I love doing would be the thing that gets replaced. You hear these stories about these, like you said, with the steam engine. I’ve, I’ve spent so many, I don’t know, maybe thousands of hours poring over code and putting my heart and soul and, like, and just, like, some of my most painful and happiest moments were alone behind… I, I was an Emacs person for a long time. Man, Emacs. And, and then there’s an identity and there’s meaning, and there’s… Like, when I walk about the world, I don’t say it out loud, but I think of myself as a programmer. And to have that in a matter of months…
Lex Fridman
I mean, like you mentioned, April to November, it really is a leap that happened, a shift that’s happening. To have that completely replaced is is painful. It’s, it’s truly painful. But I also think programmers, builders more broadly, but what is, what is the act of programming? I, I think programmers are generally best equipped at this moment in history to learn the language, to empathize with agents, to learn the language of agents. To feel the CLI.
I mean, like you mentioned, April to November, it really is a leap that happened, a shift that’s happening. To have that completely replaced is is painful. It’s, it’s truly painful. But I also think programmers, builders more broadly, but what is, what is the act of programming? I, I think programmers are generally best equipped at this moment in history to learn the language, to empathize with agents, to learn the language of agents. To feel the CLI.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Like, like to understand what is the thing you need, you the agent, need to do this task the best?
Like, like to understand what is the thing you need, you the agent, need to do this task the best?
Peter Steinberger
I think at some point it’s just gonna be called coding again, and it’s just gonna be the new normal.
I think at some point it’s just gonna be called coding again, and it’s just gonna be the new normal.
Lex Fridman
Yeah.
Yeah.
Peter Steinberger
And yet, while I don’t write the code, I very much feel like I’m in the driver’s seat and I am, I am writing the code, you know? It’s just-
And yet, while I don’t write the code, I very much feel like I’m in the driver’s seat and I am, I am writing the code, you know? It’s just-
Lex Fridman
You’ll still be a programmer. It’s just the activity of a programmer is, is different.
You’ll still be a programmer. It’s just the activity of a programmer is, is different.
Peter Steinberger
Yeah, and because on X, the bubble, I mean, is mostly positive. On, on Mastodon and Bluesky, I don’t… I also use it less because oftentimes I got attacked for my blog posts. And I, I had stronger reactions in the past, now I can sympathize with those people more ’cause, in a way I get it. It… In a way, I also don’t get it because it’s very unfair to grab onto the person that you see right now and unload all your fear and hate. It’s gonna be a change and it’s gonna be challenging, but it’s also… I don’t know. I find it incredibly fun and, and, and gratifying. And I can, I can use the new time to focus on much more details. I think the level of expectation of what we build is also rising because it’s just now… The default is now so much easier, so software is changing in many ways.
Yeah, and because on X, the bubble, I mean, is mostly positive. On, on Mastodon and Bluesky, I don’t… I also use it less because oftentimes I got attacked for my blog posts. And I, I had stronger reactions in the past, now I can sympathize with those people more ’cause, in a way I get it. It… In a way, I also don’t get it because it’s very unfair to grab onto the person that you see right now and unload all your fear and hate. It’s gonna be a change and it’s gonna be challenging, but it’s also… I don’t know. I find it incredibly fun and, and, and gratifying. And I can, I can use the new time to focus on much more details. I think the level of expectation of what we build is also rising because it’s just now… The default is now so much easier, so software is changing in many ways.
Peter Steinberger
There’s gonna be a lot more. And then you have all these people that are screaming, “Oh yeah, but what about the water?” You know? Like, I did a conference in Italy about the, the state of AI, and m- my whole motivation was to push people away from, don’t see yourself as an iOS developer anymore. You’re now a builder, and you can use your skills in many more ways. Also because apps are slowly going away. People didn’t like that. Like a lot of people didn’t like what I had to say. And I don’t think I was hyperbole, I was just like, “This is how I see the future.” Maybe this is not how it’s going to be, but I’m pretty sure a version of that will happen.
There’s gonna be a lot more. And then you have all these people that are screaming, “Oh yeah, but what about the water?” You know? Like, I did a conference in Italy about the, the state of AI, and m- my whole motivation was to push people away from, don’t see yourself as an iOS developer anymore. You’re now a builder, and you can use your skills in many more ways. Also because apps are slowly going away. People didn’t like that. Like a lot of people didn’t like what I had to say. And I don’t think I was hyperbole, I was just like, “This is how I see the future.” Maybe this is not how it’s going to be, but I’m pretty sure a version of that will happen.
Peter Steinberger
And the first question I got was, “Yeah, but what about the insane water use on data centers?” But then you actually sit down and do the maths, and then for most people if you just skip one burger per month, that compensates the, the CO2 output, or, like, the water use in equivalent of tokens. I mean, the maths is, is… the maths is tricky, and it depends if you add pre-training, then maybe it’s more than just one patty…. but it’s not off by a factor of 100, you know? So, so the… or like golf is still using way more water than all data centers together. So are you also hating people that play golf? Those people grab on anything that they think is bad about AI without seeing the potential things that might be good about AI.
And the first question I got was, “Yeah, but what about the insane water use on data centers?” But then you actually sit down and do the maths, and then for most people if you just skip one burger per month, that compensates the, the CO2 output, or, like, the water use in equivalent of tokens. I mean, the maths is, is… the maths is tricky, and it depends if you add pre-training, then maybe it’s more than just one patty…. but it’s not off by a factor of 100, you know? So, so the… or like golf is still using way more water than all data centers together. So are you also hating people that play golf? Those people grab on anything that they think is bad about AI without seeing the potential things that might be good about AI.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
And I’m not saying everything’s good. It’s certainly gonna be a very transformative technology for our society.
And I’m not saying everything’s good. It’s certainly gonna be a very transformative technology for our society.
Lex Fridman
There’s to steel man the, the criticism in general, I do wanna say in my experience with Silicon Valley there’s a bit of a bubble in the sense that there’s a kind of excitement and an over-focus about the positive that the technology can bring.
There’s to steel man the, the criticism in general, I do wanna say in my experience with Silicon Valley there’s a bit of a bubble in the sense that there’s a kind of excitement and an over-focus about the positive that the technology can bring.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
And… which is great. It’s great to focus on… N- not to, not to be paralyzed by fear and fear-mongering and so on, but there’s also within that excitement, and within everybody talking just to each other, there’s a dismissal of the basic human experience across the United States and the Midwest, across the world. Including the programmers we mentioned, including all the people that are gonna lose their jobs, including the s- the measurable pain and suffering that happens at the short-term scale when there’s change of any kind. Especially large-scale transformative change that we’re about to face if what we’re talking about will materialize. And so to ha- having a bit of that humility and awareness about the tools you’re building, they’re going to cause pain.
And… which is great. It’s great to focus on… N- not to, not to be paralyzed by fear and fear-mongering and so on, but there’s also within that excitement, and within everybody talking just to each other, there’s a dismissal of the basic human experience across the United States and the Midwest, across the world. Including the programmers we mentioned, including all the people that are gonna lose their jobs, including the s- the measurable pain and suffering that happens at the short-term scale when there’s change of any kind. Especially large-scale transformative change that we’re about to face if what we’re talking about will materialize. And so to ha- having a bit of that humility and awareness about the tools you’re building, they’re going to cause pain.
Lex Fridman
They will long term hopefully bring about a better world, and even more opportunities-
They will long term hopefully bring about a better world, and even more opportunities-
Peter Steinberger
Yeah
Yeah
Lex Fridman
… and even more awesomeness. But having that kind of like quiet moment often of, of respect for the pain that is going to be felt. And so not, not enough of that is, I think, done, so it’s, it’s good to have a bit of that.
… and even more awesomeness. But having that kind of like quiet moment often of, of respect for the pain that is going to be felt. And so not, not enough of that is, I think, done, so it’s, it’s good to have a bit of that.
Peter Steinberger
And then I also have to put against some of the emails I got where people told me they have a small business, and they’ve been struggling. And, and OpenClaw helped them automate a few of the tedious tasks from, from collecting invoices to like answering customer emails that then freed them up and like cost them a bit more joy in their life.
And then I also have to put against some of the emails I got where people told me they have a small business, and they’ve been struggling. And, and OpenClaw helped them automate a few of the tedious tasks from, from collecting invoices to like answering customer emails that then freed them up and like cost them a bit more joy in their life.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Or, or some emails where they told me that OpenClaw helped their disabled daughter. That she’s now empowered and feels she can do much more than before. Which is amazing, right? Because you could, you could do that before as well. The technology was there. I didn’t, I didn’t invent a whole new thing, but I made it a lot easier and more accessible, and that did show people the possibilities that they previously wouldn’t see. And now they apply it for good.
Or, or some emails where they told me that OpenClaw helped their disabled daughter. That she’s now empowered and feels she can do much more than before. Which is amazing, right? Because you could, you could do that before as well. The technology was there. I didn’t, I didn’t invent a whole new thing, but I made it a lot easier and more accessible, and that did show people the possibilities that they previously wouldn’t see. And now they apply it for good.
Lex Fridman
Mm-hmm.
Mm-hmm.
Peter Steinberger
Or like also the fact that, yes, I, I, I suggest the, the, the latest and best models, but you can totally run this on free models. You can run this locally. You can run this on, on, on Keyme or other, other, other models that are way more accessible price-wise, and still have a, a very powerful system that might otherwise not be possible. Because other things like, I don’t know, Entropik’s CoWork is locked in into their space, so it’s not all black and white. There’s… I got a lot of emails that were heartwarming and amazing. And, and I don’t know, it just made me really happy.
Or like also the fact that, yes, I, I, I suggest the, the, the latest and best models, but you can totally run this on free models. You can run this locally. You can run this on, on, on Keyme or other, other, other models that are way more accessible price-wise, and still have a, a very powerful system that might otherwise not be possible. Because other things like, I don’t know, Entropik’s CoWork is locked in into their space, so it’s not all black and white. There’s… I got a lot of emails that were heartwarming and amazing. And, and I don’t know, it just made me really happy.
Lex Fridman
Yeah, there’s a lot… It has brought joy into a lot of people’s lives. Not just, not just programmers. Like a lot of people’s lives. It’s, it’s, it’s beautiful to see. What gives you hope about this whole thing we have going on with human civilization?
Yeah, there’s a lot… It has brought joy into a lot of people’s lives. Not just, not just programmers. Like a lot of people’s lives. It’s, it’s, it’s beautiful to see. What gives you hope about this whole thing we have going on with human civilization?
Future of OpenClaw community
Peter Steinberger
I mean, I inspired so many people. There’s like… there’s this whole builder vibe again. People are now using AI in a more playful way and are discovering what it can do and how it can like help them in their life. And creating new places that are just sprawling of creativity. I don’t know. Like, there’s like ClawCoin in Vienna. There’s like 500 people. And there’s such a high percentage of people that uh, want to present, which is to me really surprising, because u- usually it’s quite hard to find people that want to like talk about what they built. And now it’s, there’s an abundance. So that gives me hope that we can, we can figure shit out.
I mean, I inspired so many people. There’s like… there’s this whole builder vibe again. People are now using AI in a more playful way and are discovering what it can do and how it can like help them in their life. And creating new places that are just sprawling of creativity. I don’t know. Like, there’s like ClawCoin in Vienna. There’s like 500 people. And there’s such a high percentage of people that uh, want to present, which is to me really surprising, because u- usually it’s quite hard to find people that want to like talk about what they built. And now it’s, there’s an abundance. So that gives me hope that we can, we can figure shit out.
Lex Fridman
And it makes it accessible to basically everybody.
And it makes it accessible to basically everybody.
Peter Steinberger
Yeah.
Yeah.
Lex Fridman
Just imagine all these people building, especially as you make it simpler and simpler, more secure. It’s like anybody who has ideas and can express those ideas in language can build. That’s crazy.
Just imagine all these people building, especially as you make it simpler and simpler, more secure. It’s like anybody who has ideas and can express those ideas in language can build. That’s crazy.
Peter Steinberger
Yeah, that’s ultimately power to the people, and one of the beauty, the beautiful things that come out of AI. Not just, not just a slop generator.
Yeah, that’s ultimately power to the people, and one of the beauty, the beautiful things that come out of AI. Not just, not just a slop generator.
Lex Fridman
Well, Mr. Clawfather, I just realized when I said that in the beginning, I violated two trademarks, because there’s also the Godfather. I’m getting sued by everybody. You’re a wonderful human being. You’ve created something really special, a special community, a special product, a special set of ideas. Plus, the entire… the humor, the good vibes, the inspiration of all these people building, the excitement to build. So I’m truly grateful for everything you’ve been doing and for who you are, and for sitting down to talk with me today. Thank you, brother.
Well, Mr. Clawfather, I just realized when I said that in the beginning, I violated two trademarks, because there’s also the Godfather. I’m getting sued by everybody. You’re a wonderful human being. You’ve created something really special, a special community, a special product, a special set of ideas. Plus, the entire… the humor, the good vibes, the inspiration of all these people building, the excitement to build. So I’m truly grateful for everything you’ve been doing and for who you are, and for sitting down to talk with me today. Thank you, brother.
Peter Steinberger
Thanks for giving me the chance to tell my story.
Thanks for giving me the chance to tell my story.
Lex Fridman
Thanks for listening to this conversation with Peter Steinberger. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback and so on. And now let me leave you with some words from Voltaire. “With great power comes great responsibility.” Thank you for listening, and hope to see you next time.
Thanks for listening to this conversation with Peter Steinberger. To support this podcast, please check out our sponsors in the description, where you can also find links to contact me, ask questions, give feedback and so on. And now let me leave you with some words from Voltaire. “With great power comes great responsibility.” Thank you for listening, and hope to see you next time.
Transcript for GSP teaches Lex Fridman how to street fight
This is a transcript of “GSP teaches Lex Fridman how to street fight”.
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links
that take you directly to that point in
the main video. Please note that the transcript is
human generated, and may have errors.
Here are some useful links:
- Watch the full YouTube version of the video
Georges St-Pierre
In a street fight, I would rather- …fight Francis Ngannou than fight Bas Rutten. In a street fight.
In a street fight, I would rather- …fight Francis Ngannou than fight Bas Rutten. In a street fight.
Lex Fridman
Let me tell you first that I’ve been around. I’ve been a bouncer for many, many years. Bang! Bang! Bang! It’s a street fight. Everybody underestimates the kick in the groin. It’s boom, that’s the first thing to do. I follow up, bang, bang, bang. Right away after that, danga-da-danga-da-dang. See what I’m doing? Boom! That’s the left elbow right there.
Let me tell you first that I’ve been around. I’ve been a bouncer for many, many years. Bang! Bang! Bang! It’s a street fight. Everybody underestimates the kick in the groin. It’s boom, that’s the first thing to do. I follow up, bang, bang, bang. Right away after that, danga-da-danga-da-dang. See what I’m doing? Boom! That’s the left elbow right there.
Georges St-Pierre
Yeah, so very often people ask me the difference between a street fight and a fight in mixed martial arts. The difference is in the street, there is no referee. And there’s an instigator, and there is the other person. The best in life for a street fight is always to not be the instigator because you have the element of surprise. So if you’re in a heated argument with someone and you feel that you’re potentially going to be in a fight, the best thing to do is to never show your center line, to always go on the side and put your hands up like this. Now, that’s one of the best things to do. It’s a self-defense tactic that is used all around the world. Because from there, the distance that I have to travel to cause a lot of damage to him is very minimal.
Yeah, so very often people ask me the difference between a street fight and a fight in mixed martial arts. The difference is in the street, there is no referee. And there’s an instigator, and there is the other person. The best in life for a street fight is always to not be the instigator because you have the element of surprise. So if you’re in a heated argument with someone and you feel that you’re potentially going to be in a fight, the best thing to do is to never show your center line, to always go on the side and put your hands up like this. Now, that’s one of the best things to do. It’s a self-defense tactic that is used all around the world. Because from there, the distance that I have to travel to cause a lot of damage to him is very minimal.
Georges St-Pierre
You know, it’s very short. I can go boom. I can go boom. And I’m protected because if he ever tried to do anything, my hands are already up, but I’m ready to respond to any aggression. So the first thing is, if you’re in an argument and you feel the heat is rising, is to hit first. You don’t want to fight, but you want to hit first. You want to hit first, you know? So it’s either boom, hit first, depending on the situation. If you’re someone who is much less physically strong than the aggressor, you can use the eyes, the genitals, the neck, you know? And then you can leave the scene. However, if I’m like this, the minute he touches me, he declares war. Now I can go and perform a self-defense move.
You know, it’s very short. I can go boom. I can go boom. And I’m protected because if he ever tried to do anything, my hands are already up, but I’m ready to respond to any aggression. So the first thing is, if you’re in an argument and you feel the heat is rising, is to hit first. You don’t want to fight, but you want to hit first. You want to hit first, you know? So it’s either boom, hit first, depending on the situation. If you’re someone who is much less physically strong than the aggressor, you can use the eyes, the genitals, the neck, you know? And then you can leave the scene. However, if I’m like this, the minute he touches me, he declares war. Now I can go and perform a self-defense move.
Lex Fridman
So striking, not wrestling?
So striking, not wrestling?
Georges St-Pierre
Yes. It’s always striking first and leave the scene. If you’re, for example, a kid or someone who doesn’t have the physical strength of your aggressor. Of course, I’m a UFC champion— …so that does not apply to me. But the key is, tactically, we always use the element of surprise, and when you strike, strike first. And strike to cause as much damage as possible. The eyes, you can do the neck, you can do the genitals. And then after, you can leave the scene. That’s the goal of having the element of surprise.
Yes. It’s always striking first and leave the scene. If you’re, for example, a kid or someone who doesn’t have the physical strength of your aggressor. Of course, I’m a UFC champion— …so that does not apply to me. But the key is, tactically, we always use the element of surprise, and when you strike, strike first. And strike to cause as much damage as possible. The eyes, you can do the neck, you can do the genitals. And then after, you can leave the scene. That’s the goal of having the element of surprise.
Lex Fridman
Okay, you were talking about knives. What about if weapons are involved, run faster?
Okay, you were talking about knives. What about if weapons are involved, run faster?
Georges St-Pierre
So weapons are very important. If someone has a weapon and attacks me for my money, I give him my money even if I’m Georges St-Pierre and I’m a UFC champion. However, if someone puts a knife to my throat here and he’s telling me to go in the trunk… now, I don’t want to go in the trunk because I know it’s a bad ending. So things that I can do first is always make sure that I try to keep my hands as close as possible to the weapon. And I try to be at as close range as possible. I can act like I want to—
So weapons are very important. If someone has a weapon and attacks me for my money, I give him my money even if I’m Georges St-Pierre and I’m a UFC champion. However, if someone puts a knife to my throat here and he’s telling me to go in the trunk… now, I don’t want to go in the trunk because I know it’s a bad ending. So things that I can do first is always make sure that I try to keep my hands as close as possible to the weapon. And I try to be at as close range as possible. I can act like I want to—
Lex Fridman
Look scared?
Look scared?
Georges St-Pierre
Yeah. “Please, please, please,” boom. See, here I use my body, and then I can go and break, you know? So the idea is to use your entire body to deflect the weapon. So if the weapon is like this and the blade is coming out this way, I use the element of surprise. You see, I use my body, not only grabbing him like this, so if he tries to come back with the knife, it’s solid, and I can go and break. If the blade is pointing the other side, it’s something here, here, and here. Here I can use my body always to smother the weapon and—
Yeah. “Please, please, please,” boom. See, here I use my body, and then I can go and break, you know? So the idea is to use your entire body to deflect the weapon. So if the weapon is like this and the blade is coming out this way, I use the element of surprise. You see, I use my body, not only grabbing him like this, so if he tries to come back with the knife, it’s solid, and I can go and break. If the blade is pointing the other side, it’s something here, here, and here. Here I can use my body always to smother the weapon and—
Lex Fridman
Controlling the wrist, yeah. But if it’s out here…
Controlling the wrist, yeah. But if it’s out here…
unassigned
It’s through his clothes.
It’s through his clothes.
Georges St-Pierre
If it’s out here, and yes, exactly. There’s too much distance. You want to make sure you get close to the weapon because that’s what can cause the most damage. This is very important. There are other situations. Let’s say you’re a kid or someone comes to grab you by the body. What I can do is grab the head and put my fingers inside the eyes; that will make my opponent release me immediately. Then I can go and leave, you know?
If it’s out here, and yes, exactly. There’s too much distance. You want to make sure you get close to the weapon because that’s what can cause the most damage. This is very important. There are other situations. Let’s say you’re a kid or someone comes to grab you by the body. What I can do is grab the head and put my fingers inside the eyes; that will make my opponent release me immediately. Then I can go and leave, you know?
Lex Fridman
Like thumb in? Like thumb?
Like thumb in? Like thumb?
Georges St-Pierre
Yeah, thumb in the eyes. You push in the eyes.
Yeah, thumb in the eyes. You push in the eyes.
Lex Fridman
Blind them.
Blind them.
Georges St-Pierre
There are no rules. The eyes are always my favorite choice to go for because if you cannot see, it’s very hard to fight. And normally the reflex for most people, when they can’t see, they grab their eyes, you know? So it releases the grip.
There are no rules. The eyes are always my favorite choice to go for because if you cannot see, it’s very hard to fight. And normally the reflex for most people, when they can’t see, they grab their eyes, you know? So it releases the grip.
Lex Fridman
I’m now going to ask you about the tie because I think you’re wrong still about that. I think it’s possible to use it as a… same as for a head snatch, like this kind of situation, to choke.
I’m now going to ask you about the tie because I think you’re wrong still about that. I think it’s possible to use it as a… same as for a head snatch, like this kind of situation, to choke.
Georges St-Pierre
I think it could be an advantage if it’s a fake tie. If it’s something that can go, like it can—
I think it could be an advantage if it’s a fake tie. If it’s something that can go, like it can—
Lex Fridman
Clip off?
Clip off?
Georges St-Pierre
Like a tail of a reptile that can go. So if you try to pull my tie, it comes out, and now I know I get a head start.
Like a tail of a reptile that can go. So if you try to pull my tie, it comes out, and now I know I get a head start.
Lex Fridman
Element of surprise
Element of surprise
unassigned
Exactly, it’s all about the element of surprise. You want to strike first; the element of surprise in the street.
Exactly, it’s all about the element of surprise. You want to strike first; the element of surprise in the street.
Lex Fridman
Georges, thank you so much for talking today.
Georges, thank you so much for talking today.
Georges St-Pierre
My pleasure.
My pleasure.
Lex Fridman
Thank you for looking sharp.
Thank you for looking sharp.
Georges St-Pierre
Man in black, baby!
Man in black, baby!
Lex Fridman
Man in black.
Man in black.
Transcript for 1984 by George Orwell | Lex Fridman
This is a transcript of “1984 by George Orwell | Lex Fridman”.
The timestamps in the transcript are clickable links that take you directly to that point in
the main video. Please note that the transcript is human generated, and may have errors.
Here are some useful links:
The timestamps in the transcript are clickable links that take you directly to that point in
the main video. Please note that the transcript is human generated, and may have errors.
Here are some useful links:
- Watch the full YouTube version of the video
Table of Contents
Here are the loose “chapters” in the video.
Click link to jump approximately to that part in the transcript:
Click link to jump approximately to that part in the transcript:
- 0:00 – Intro
- 1:02 – World of 1984
- 4:19 – Love
- 12:42 – Hate
- 17:21 – Power
- 25:56 – Orwell
- 28:49 – Technology
- 47:14 – Reading list controversy
Intro
Lex Fridman
“There was truth, and there was untruth. And if you clung to the truth, even against the whole world, you were not mad.” 1984 by George Orwell is one of the most impactful books ever written. It has been widely used and misused in political discourse by all kinds of ideologues. Into that discourse, it entered terms like Big Brother, thoughtcrime, Doublethink, Newspeak, Thought Police, and Orwellian, strangely enough, as a synonym for the very thing that the author, Orwell, was against. It’s been translated into over 65 languages, has sold over 30 million copies, and has been banned in many countries, especially authoritarian regimes. It was banned under Stalin, and as recently as 2022 in Belarus. In this video, I’ll give a quick summary with spoilers and a few takeaways.
“There was truth, and there was untruth. And if you clung to the truth, even against the whole world, you were not mad.” 1984 by George Orwell is one of the most impactful books ever written. It has been widely used and misused in political discourse by all kinds of ideologues. Into that discourse, it entered terms like Big Brother, thoughtcrime, Doublethink, Newspeak, Thought Police, and Orwellian, strangely enough, as a synonym for the very thing that the author, Orwell, was against. It’s been translated into over 65 languages, has sold over 30 million copies, and has been banned in many countries, especially authoritarian regimes. It was banned under Stalin, and as recently as 2022 in Belarus. In this video, I’ll give a quick summary with spoilers and a few takeaways.
World of 1984
Lex Fridman
I’d like to try to make it somewhat interesting to people who both have and have not read the book. Let’s see how it goes. The world in the book 1984 is a dystopian future society, a nation, maybe you can say superstate named Oceania. It’s fully controlled by a totalitarian political party called Ingsoc. It’s led by Big Brother who, as we might discuss, may or may not be a real person. He might just be a symbol used by the party. The party wants only to increase its power, also something we might talk about. It uses technology, telescreens, for mass surveillance. It’s creating a new language called Newspeak, which removes words from English that could lead to rebellion.
I’d like to try to make it somewhat interesting to people who both have and have not read the book. Let’s see how it goes. The world in the book 1984 is a dystopian future society, a nation, maybe you can say superstate named Oceania. It’s fully controlled by a totalitarian political party called Ingsoc. It’s led by Big Brother who, as we might discuss, may or may not be a real person. He might just be a symbol used by the party. The party wants only to increase its power, also something we might talk about. It uses technology, telescreens, for mass surveillance. It’s creating a new language called Newspeak, which removes words from English that could lead to rebellion.
Lex Fridman
It uses Doublethink to control thought by, perhaps you could say, forcing you to hold contradictory beliefs and accept them as true. If not, the Thought Police arrest you for committing a thoughtcrime. Examples of Doublethink are “War is peace,” “Freedom is slavery,” and “Ignorance is strength.” And finally, the party constantly rewrites history. As the quote goes, “Who controls the past controls the future. Who controls the present controls the past.” There are four ministries. The Ministry of Truth is responsible for propaganda and, like I said, rewriting history. The Ministry of Love is responsible for brainwashing people through torture. The Ministry of Plenty is responsible for rationing food, supplies, and goods.
It uses Doublethink to control thought by, perhaps you could say, forcing you to hold contradictory beliefs and accept them as true. If not, the Thought Police arrest you for committing a thoughtcrime. Examples of Doublethink are “War is peace,” “Freedom is slavery,” and “Ignorance is strength.” And finally, the party constantly rewrites history. As the quote goes, “Who controls the past controls the future. Who controls the present controls the past.” There are four ministries. The Ministry of Truth is responsible for propaganda and, like I said, rewriting history. The Ministry of Love is responsible for brainwashing people through torture. The Ministry of Plenty is responsible for rationing food, supplies, and goods.
Lex Fridman
And the Ministry of Peace, of course, is responsible for maintaining a constant state of war. Society is divided into three levels: the Inner Party, the Outer Party, and the Proles. The term stands for, I guess, proletariats; it’s the working class. The Inner Party’s tiny. The Outer Party’s a little bit bigger, and the majority of the people—I forget what the percentage is, maybe 80%—are the Proles, the working class. There are several key characters. Winston, the main character, is a low-ranking member of Ingsoc. He works at the Ministry of Truth where he rewrites history. Julia is a girl who Winston falls in love with, and she with him.
And the Ministry of Peace, of course, is responsible for maintaining a constant state of war. Society is divided into three levels: the Inner Party, the Outer Party, and the Proles. The term stands for, I guess, proletariats; it’s the working class. The Inner Party’s tiny. The Outer Party’s a little bit bigger, and the majority of the people—I forget what the percentage is, maybe 80%—are the Proles, the working class. There are several key characters. Winston, the main character, is a low-ranking member of Ingsoc. He works at the Ministry of Truth where he rewrites history. Julia is a girl who Winston falls in love with, and she with him.
Lex Fridman
They have sex, and this is maybe a good place to mention that love and passionate sex are forbidden in this society. “Goodsex” I think is a term under Newspeak; it’s the kind of sex that leads to procreation, which is the only kind allowed and the only kind that’s “good.” O’Brien is another central character. He’s the member of the Inner Party that convinces Winston he’s part of the Brotherhood, which is a lie, and he eventually is the man who tortures Winston and breaks his mind, breaks his heart. Big Brother and Emmanuel Goldstein are these symbolic characters that we never actually get to meet. They may or may not exist.
They have sex, and this is maybe a good place to mention that love and passionate sex are forbidden in this society. “Goodsex” I think is a term under Newspeak; it’s the kind of sex that leads to procreation, which is the only kind allowed and the only kind that’s “good.” O’Brien is another central character. He’s the member of the Inner Party that convinces Winston he’s part of the Brotherhood, which is a lie, and he eventually is the man who tortures Winston and breaks his mind, breaks his heart. Big Brother and Emmanuel Goldstein are these symbolic characters that we never actually get to meet. They may or may not exist.
Love
Lex Fridman
Big Brother is the head of the party Ingsoc, and Emmanuel Goldstein is the leader of the so-called Brotherhood, which is this mysterious group that lurks in the shadows and works to overthrow the party. Again, they may or may not exist. We’ll maybe talk about the importance of that in a totalitarian state. So, a few key takeaways. I’ll try to do my best—I have disparate notes that I took for myself—to integrate them together to make some cohesive thoughts. Part of the reason I wanted to do this is that while I have read 1984 many times, and many of the books on the reading list I’ve read many times, I haven’t often really concretized my thoughts about them.
Big Brother is the head of the party Ingsoc, and Emmanuel Goldstein is the leader of the so-called Brotherhood, which is this mysterious group that lurks in the shadows and works to overthrow the party. Again, they may or may not exist. We’ll maybe talk about the importance of that in a totalitarian state. So, a few key takeaways. I’ll try to do my best—I have disparate notes that I took for myself—to integrate them together to make some cohesive thoughts. Part of the reason I wanted to do this is that while I have read 1984 many times, and many of the books on the reading list I’ve read many times, I haven’t often really concretized my thoughts about them.
Lex Fridman
I just take the journey and let the thoughts wander around in the background as I live my life. I wanted to put them on paper and maybe share them with others to see what they think my concrete takeaways are from the book, if I could try to convert them into words. So the first one for me, especially later in life as I’ve been reading this book, is that when everything else or most things that make you human are taken away by a totalitarian state, the last thing that’s left, which is the most difficult to take away, is love. Love for other human beings, love for life itself. That’s the little flame from which hope springs. The key revolutionary act is the act of love.
I just take the journey and let the thoughts wander around in the background as I live my life. I wanted to put them on paper and maybe share them with others to see what they think my concrete takeaways are from the book, if I could try to convert them into words. So the first one for me, especially later in life as I’ve been reading this book, is that when everything else or most things that make you human are taken away by a totalitarian state, the last thing that’s left, which is the most difficult to take away, is love. Love for other human beings, love for life itself. That’s the little flame from which hope springs. The key revolutionary act is the act of love.
Lex Fridman
So when the ability to speak is taken away, when the ability to think rational thoughts is taken away, the last thing that’s left, and the thing that ultimately gives hope, is love. That’s a big takeaway for me. The note that Julia gives to Winston reading “I love you” is the kind of revolutionary act that leads to a society beyond the one they exist in. I think a lot of the book has an interesting hypocrisy to it, where the main character, Winston, is almost in an animalistic way obsessed with destroying the state in rebellion and revolution. But I think love is the thing that allows you to believe in a place beyond the state, in believing that you can build something better, versus just destroying the thing you’re in.
So when the ability to speak is taken away, when the ability to think rational thoughts is taken away, the last thing that’s left, and the thing that ultimately gives hope, is love. That’s a big takeaway for me. The note that Julia gives to Winston reading “I love you” is the kind of revolutionary act that leads to a society beyond the one they exist in. I think a lot of the book has an interesting hypocrisy to it, where the main character, Winston, is almost in an animalistic way obsessed with destroying the state in rebellion and revolution. But I think love is the thing that allows you to believe in a place beyond the state, in believing that you can build something better, versus just destroying the thing you’re in.
Lex Fridman
I think you have to be careful as a revolutionary not to obsess 100% with destruction. Because beyond destruction, there could be chaos that leads to something much worse. I think love is the basic human thing that connects all of us, the messy thing that connects all of us, that allows you to build a better society after the totalitarian one is overthrown. What else did I want to say? There’s an interesting tension there between love and lust. I think there’s a quote that pure love or pure lust was impossible or forbidden. “Pure” here meaning unadulterated, uncensored intensity of feeling, maybe intimacy.
I think you have to be careful as a revolutionary not to obsess 100% with destruction. Because beyond destruction, there could be chaos that leads to something much worse. I think love is the basic human thing that connects all of us, the messy thing that connects all of us, that allows you to build a better society after the totalitarian one is overthrown. What else did I want to say? There’s an interesting tension there between love and lust. I think there’s a quote that pure love or pure lust was impossible or forbidden. “Pure” here meaning unadulterated, uncensored intensity of feeling, maybe intimacy.
Lex Fridman
And there was an interesting question raised by the book, both by Winston and Julia: what is ultimately the most powerful act of rebellion? Is it between us humans when everything is forbidden? Is it animalistic like sex? Just lust for another human? Or is it love? The kind of love you have for a romantic partner, but even love for family and love for friends. I don’t know. I think the book almost claims that it is sex, but I think what the book also shows is that if sex is your manifestation of rebellion, that ultimately leads to something that doesn’t last. That ultimately leads to a focus on destruction versus building beyond the horizon when the state falls. So, some quotes from Winston on this.
And there was an interesting question raised by the book, both by Winston and Julia: what is ultimately the most powerful act of rebellion? Is it between us humans when everything is forbidden? Is it animalistic like sex? Just lust for another human? Or is it love? The kind of love you have for a romantic partner, but even love for family and love for friends. I don’t know. I think the book almost claims that it is sex, but I think what the book also shows is that if sex is your manifestation of rebellion, that ultimately leads to something that doesn’t last. That ultimately leads to a focus on destruction versus building beyond the horizon when the state falls. So, some quotes from Winston on this.
Lex Fridman
“The more men you’ve had sex with…” Julia admitted to having sex with quite a lot of people. He says, “The more men you’ve had sex with, the more I love you. I hate purity. I hate virtue. I want everyone to be corrupt to the bone.” This kind of rubbed me the wrong way because, again, this seems to be obsessed with the hatred towards the state versus a longing and a hope—which I think hope is really important here—a hope for a better future beyond the state. Again, another quote from the book: “Their embrace had been a battle, the climax a victory. It was a blow struck against the Party. It was a political act.” So there, again, I think sex is seen as a political act of rebellion. I think that’s not the deeply human thing here.
“The more men you’ve had sex with…” Julia admitted to having sex with quite a lot of people. He says, “The more men you’ve had sex with, the more I love you. I hate purity. I hate virtue. I want everyone to be corrupt to the bone.” This kind of rubbed me the wrong way because, again, this seems to be obsessed with the hatred towards the state versus a longing and a hope—which I think hope is really important here—a hope for a better future beyond the state. Again, another quote from the book: “Their embrace had been a battle, the climax a victory. It was a blow struck against the Party. It was a political act.” So there, again, I think sex is seen as a political act of rebellion. I think that’s not the deeply human thing here.
Lex Fridman
The deeply human thing is, again, the act of love. It’s a source of hope; it’s the catalyst for building a better future beyond the revolution. An interesting side note here—and there could be a million interesting side notes, and I’m desperately trying not to go on a million tangents, to hold myself together and stay focused—is on family. There’s all kinds of love, and I think family love is a really powerful bond that connects us, and that’s one of the things that totalitarian states really go after.
The deeply human thing is, again, the act of love. It’s a source of hope; it’s the catalyst for building a better future beyond the revolution. An interesting side note here—and there could be a million interesting side notes, and I’m desperately trying not to go on a million tangents, to hold myself together and stay focused—is on family. There’s all kinds of love, and I think family love is a really powerful bond that connects us, and that’s one of the things that totalitarian states really go after.
Lex Fridman
And I should mention, I’m loosely using the terms authoritarian and totalitarian here. To me, authoritarian means there’s a government with complete centralized control of political affairs. A totalitarian state is beyond that; it is complete control of not just politics but also social, economic, everything. Nazi Germany is an example of that, I think, where there’s just complete control of every single thing, from the war effort to social interactions, the rules that govern social interaction, the press, all that kind of stuff.
And I should mention, I’m loosely using the terms authoritarian and totalitarian here. To me, authoritarian means there’s a government with complete centralized control of political affairs. A totalitarian state is beyond that; it is complete control of not just politics but also social, economic, everything. Nazi Germany is an example of that, I think, where there’s just complete control of every single thing, from the war effort to social interactions, the rules that govern social interaction, the press, all that kind of stuff.
Lex Fridman
So I think this book is more about, at least in my definition of the term, totalitarianism. Anyway, as I was saying about family, I think the way they destroy family is, one, of course with your romantic partner by forbidding passion—passionate sex, but really just passion and longing for another human being in that romantic way. And they also really reward and encourage children at a young age; they indoctrinate them to turn their parents in for thoughtcrime, whether real or not, which of course is a silly notion because there’s no nature of truth. You can just accuse anyone of anything and they’re guilty just by existing. So that’s a way to attack the family.
So I think this book is more about, at least in my definition of the term, totalitarianism. Anyway, as I was saying about family, I think the way they destroy family is, one, of course with your romantic partner by forbidding passion—passionate sex, but really just passion and longing for another human being in that romantic way. And they also really reward and encourage children at a young age; they indoctrinate them to turn their parents in for thoughtcrime, whether real or not, which of course is a silly notion because there’s no nature of truth. You can just accuse anyone of anything and they’re guilty just by existing. So that’s a way to attack the family.
Lex Fridman
And I should also have mentioned on the topic of love that I think the goal of the Party, the final destination as described by O’Brien through the process of torture, is to break your mind, heart, and soul completely so that the only love you can have—and it could be felt as a pure love—is for Big Brother. This is the kind of thing you see in North Korea, where the only love you’re allowed to have, the remaining inklings of feeling that might still exist in you, you can channel only not towards family, romantic partners, or friends, but towards this leader, this godlike messianic figure. In this case, one who may or may not exist.
And I should also have mentioned on the topic of love that I think the goal of the Party, the final destination as described by O’Brien through the process of torture, is to break your mind, heart, and soul completely so that the only love you can have—and it could be felt as a pure love—is for Big Brother. This is the kind of thing you see in North Korea, where the only love you’re allowed to have, the remaining inklings of feeling that might still exist in you, you can channel only not towards family, romantic partners, or friends, but towards this leader, this godlike messianic figure. In this case, one who may or may not exist.
Hate
Lex Fridman
In all cases, that figure, while there is a human associated with it, is really much bigger than the human, and that’s the only love you’re allowed to have. So the other takeaway I have is on the topic of hate. I think all humans have the capacity, almost an animalistic craving, for hate of the “other,” the enemy. Whether it’s individuals like Emmanuel Goldstein or nations like Eurasia and East Asia—which are the two other superstates described in this book—they’re constantly at war with each other. Again, the fascinating thing about the way this book is written is you don’t know if Eurasia or East Asia even exist. You really don’t know what is true beyond the local interactions of the main character.
In all cases, that figure, while there is a human associated with it, is really much bigger than the human, and that’s the only love you’re allowed to have. So the other takeaway I have is on the topic of hate. I think all humans have the capacity, almost an animalistic craving, for hate of the “other,” the enemy. Whether it’s individuals like Emmanuel Goldstein or nations like Eurasia and East Asia—which are the two other superstates described in this book—they’re constantly at war with each other. Again, the fascinating thing about the way this book is written is you don’t know if Eurasia or East Asia even exist. You really don’t know what is true beyond the local interactions of the main character.
Lex Fridman
And that, I think, is the point. When you don’t really know, there’s no steady footing on which to construct a worldview from which you can have hope for a better future. This animalistic craving for hate, especially when we’re in crowds, is most powerfully illustrated in the “Two Minutes of Hate” practiced by that society. The quote is, “The horrible thing about the Two Minutes of Hate was not that one was obliged to act a part, but that it was impossible to avoid joining in. Within thirty seconds any pretence was always unnecessary.
And that, I think, is the point. When you don’t really know, there’s no steady footing on which to construct a worldview from which you can have hope for a better future. This animalistic craving for hate, especially when we’re in crowds, is most powerfully illustrated in the “Two Minutes of Hate” practiced by that society. The quote is, “The horrible thing about the Two Minutes of Hate was not that one was obliged to act a part, but that it was impossible to avoid joining in. Within thirty seconds any pretence was always unnecessary.
Lex Fridman
A hideous ecstasy of fear and vindictiveness, a desire to kill, to torture, to smash faces in with a sledge-hammer, seemed to flow through the whole group of people like an electric current, turning one even against one’s will into a grimacing, screaming lunatic. And yet the rage that one felt was an abstract, undirected emotion which could be switched from one object to another like the flame of a blowlamp.” That’s the point: you get the crowd together, and you get them to hate Goldstein or Eurasia or East Asia. You get them to hate anything. And that feeling, that drug, that mass hypnosis, can be directed by the state in any direction.
A hideous ecstasy of fear and vindictiveness, a desire to kill, to torture, to smash faces in with a sledge-hammer, seemed to flow through the whole group of people like an electric current, turning one even against one’s will into a grimacing, screaming lunatic. And yet the rage that one felt was an abstract, undirected emotion which could be switched from one object to another like the flame of a blowlamp.” That’s the point: you get the crowd together, and you get them to hate Goldstein or Eurasia or East Asia. You get them to hate anything. And that feeling, that drug, that mass hypnosis, can be directed by the state in any direction.
Lex Fridman
And because you have complete control of history, you can direct it on a day-by-day basis towards any target. As long as the hate is catalyzed through these kinds of rituals, it can overpower the individualistic feeling of love we have for each other. So that hate is a more animalistic desire. I don’t know what to make of it. And of course, it’s also important to say that this book was intended originally by Orwell as a satire, although a satire that has quite a lot of torture at the end and doesn’t seem to have much humor. But I think if you read it as a satire, that’s the best way to understand its relevance in our society today.
And because you have complete control of history, you can direct it on a day-by-day basis towards any target. As long as the hate is catalyzed through these kinds of rituals, it can overpower the individualistic feeling of love we have for each other. So that hate is a more animalistic desire. I don’t know what to make of it. And of course, it’s also important to say that this book was intended originally by Orwell as a satire, although a satire that has quite a lot of torture at the end and doesn’t seem to have much humor. But I think if you read it as a satire, that’s the best way to understand its relevance in our society today.
Lex Fridman
Because a lot of things, like the Two Minutes of Hate, are almost a caricature of what hate looks like in a mass gathering. But if you take it as a caricature, it can reveal some of the elements that already exist in human nature that we should be very cautious about. It reveals the very thing that, if not monitored by ourselves, can result in a slippery slope that leads to tribalism, the destruction of other groups, and then control of the collective intelligence of our species through a totalitarian state. I think there’s elements of this under illustration in social media today, though I don’t want to overstate it.
Because a lot of things, like the Two Minutes of Hate, are almost a caricature of what hate looks like in a mass gathering. But if you take it as a caricature, it can reveal some of the elements that already exist in human nature that we should be very cautious about. It reveals the very thing that, if not monitored by ourselves, can result in a slippery slope that leads to tribalism, the destruction of other groups, and then control of the collective intelligence of our species through a totalitarian state. I think there’s elements of this under illustration in social media today, though I don’t want to overstate it.
Lex Fridman
I think just like comparing things to Hitler, comparing things to 1984 is a reach in most cases. But social media does reveal this kind of mass hysteria, this capacity of humans to be outraged based on tribalism. So we have to understand it. We have to resist giving into it on the individual level. And I do believe we have the responsibility to create technology that helps us resist it, that incentivizes us not to be cruel to each other just because the people in whatever tribe we define ourselves in are being cruel to a particular person or group. Another takeaway I have is about power. Ingsoc, the totalitarian state, wants only one thing, and that is power. Power is both the means and the end. Absolute power.
I think just like comparing things to Hitler, comparing things to 1984 is a reach in most cases. But social media does reveal this kind of mass hysteria, this capacity of humans to be outraged based on tribalism. So we have to understand it. We have to resist giving into it on the individual level. And I do believe we have the responsibility to create technology that helps us resist it, that incentivizes us not to be cruel to each other just because the people in whatever tribe we define ourselves in are being cruel to a particular person or group. Another takeaway I have is about power. Ingsoc, the totalitarian state, wants only one thing, and that is power. Power is both the means and the end. Absolute power.
Power
Lex Fridman
As O’Brien describes in the torture part of the book: “The real power, the power we have to fight for night and day, is not power over things, but power over men. Power is inflicting pain and humiliation. Power is in tearing human minds to pieces and putting them together again in new shapes of your own choosing. Power is not a means, it is an end. One does not establish a dictatorship in order to safeguard a revolution; one makes the revolution in order to establish a dictatorship. The object of persecution is persecution. The object of torture is torture. The object of power is power.” This, of course, is another aspect of human nature: the will to power and the tendency of that power to corrupt.
As O’Brien describes in the torture part of the book: “The real power, the power we have to fight for night and day, is not power over things, but power over men. Power is inflicting pain and humiliation. Power is in tearing human minds to pieces and putting them together again in new shapes of your own choosing. Power is not a means, it is an end. One does not establish a dictatorship in order to safeguard a revolution; one makes the revolution in order to establish a dictatorship. The object of persecution is persecution. The object of torture is torture. The object of power is power.” This, of course, is another aspect of human nature: the will to power and the tendency of that power to corrupt.
Lex Fridman
O’Brien says also, “The weariness of the cell is the vigour of the organism.” Through the torture and breaking of the individual, the individual doesn’t matter. What matters is the organism. There’s been a lot of brilliant comments throughout social media and on Reddit—I just want to highlight something about this because I had the exact same feeling as I was rereading it. There’s a comment from a Reddit user whose name is BraveSky6764.
O’Brien says also, “The weariness of the cell is the vigour of the organism.” Through the torture and breaking of the individual, the individual doesn’t matter. What matters is the organism. There’s been a lot of brilliant comments throughout social media and on Reddit—I just want to highlight something about this because I had the exact same feeling as I was rereading it. There’s a comment from a Reddit user whose name is BraveSky6764.
Lex Fridman
He said the conversation between Lex and Michael Levin, who is a brilliant biologist and engineer, came to mind when O’Brien made an analogy to an organism which survives even as the individual cells pass away, and the great purges are analogous to the cutting of a fingernail. If you see society as an organism—which I think is the way a totalitarian state sees it—then the destruction of a large percentage of that society, the murder, the torture, and all kinds of atrocities and genocide become “justifiable” as long as the organism flourishes. That’s how you get to the ideas Stalin had: it’s okay to break a few eggs to make an omelet. This devaluation of a human being as having fundamental importance in a society…
He said the conversation between Lex and Michael Levin, who is a brilliant biologist and engineer, came to mind when O’Brien made an analogy to an organism which survives even as the individual cells pass away, and the great purges are analogous to the cutting of a fingernail. If you see society as an organism—which I think is the way a totalitarian state sees it—then the destruction of a large percentage of that society, the murder, the torture, and all kinds of atrocities and genocide become “justifiable” as long as the organism flourishes. That’s how you get to the ideas Stalin had: it’s okay to break a few eggs to make an omelet. This devaluation of a human being as having fundamental importance in a society…
Lex Fridman
is a slippery slope into atrocities. It’s not just deeply unethical from our understanding of morals and ethics; it is also very unproductive. It destroys the human spirit, and the human spirit is essential for building a great society of constant progress. I think that’s also one of the other messages of the book, is about utopia—that totalitarianism results when you chase perfection, when you present this idea of utopia. There is no utopia; there is no perfect society. I think, at least for me, that’s the takeaway. I think the optimal state of being for an individual and for a state is constant change and constant turnover.
is a slippery slope into atrocities. It’s not just deeply unethical from our understanding of morals and ethics; it is also very unproductive. It destroys the human spirit, and the human spirit is essential for building a great society of constant progress. I think that’s also one of the other messages of the book, is about utopia—that totalitarianism results when you chase perfection, when you present this idea of utopia. There is no utopia; there is no perfect society. I think, at least for me, that’s the takeaway. I think the optimal state of being for an individual and for a state is constant change and constant turnover.
Lex Fridman
In the case of a state, it’s a constant turnover of leaders and ideas, always hopefully making progress towards a better world. But it’s always going to be messy. Perfection only exists in an oppressive state. Perfection only exists when you remove the basic humanity of the individuals that make up that state, when you destroy the human spirit or suppress all freedoms. Freedom is going to be messy and chaotic, but that freedom, ultimately, in the long arc of history, is going to create progress.
In the case of a state, it’s a constant turnover of leaders and ideas, always hopefully making progress towards a better world. But it’s always going to be messy. Perfection only exists in an oppressive state. Perfection only exists when you remove the basic humanity of the individuals that make up that state, when you destroy the human spirit or suppress all freedoms. Freedom is going to be messy and chaotic, but that freedom, ultimately, in the long arc of history, is going to create progress.
Lex Fridman
So yes, as the Redditor BraveSky6764 says, that does give you a perspective of a biological system made of living organisms. Each one of us is made up of living organisms, and we take for granted all the “atrocities” happening there; we don’t seem to give a damn. I think that’s a good metaphor. If you want to put yourself in the mind of the Inner Party, of Big Brother, or the people in power, I think most, if not all of them, see themselves as doing good for society. They are able to justify things the way we justify the death of different cells in our body.
So yes, as the Redditor BraveSky6764 says, that does give you a perspective of a biological system made of living organisms. Each one of us is made up of living organisms, and we take for granted all the “atrocities” happening there; we don’t seem to give a damn. I think that’s a good metaphor. If you want to put yourself in the mind of the Inner Party, of Big Brother, or the people in power, I think most, if not all of them, see themselves as doing good for society. They are able to justify things the way we justify the death of different cells in our body.
Lex Fridman
You don’t even think of them as worthy of consideration. You don’t think of them as living beings having the same value as you. That’s one of the really powerful ideas at the founding of the United States: that all men are created equal, that there’s an equal worth to a human being no matter who they are. That idea, as flawed as its implementations have been, is a really powerful and non-trivial idea, and it resists the drug of totalitarianism and power. I do believe that on the topic of power and politics, 1984 has been misused by political ideologues.
You don’t even think of them as worthy of consideration. You don’t think of them as living beings having the same value as you. That’s one of the really powerful ideas at the founding of the United States: that all men are created equal, that there’s an equal worth to a human being no matter who they are. That idea, as flawed as its implementations have been, is a really powerful and non-trivial idea, and it resists the drug of totalitarianism and power. I do believe that on the topic of power and politics, 1984 has been misused by political ideologues.
Lex Fridman
I’ve seen it, for example, when conservatives in the United States have used 1984 to call left-wing policies “Orwellian.” I think that’s an overstatement, of course used for dramatic effect, but it should at least be said that Orwell was a democratic socialist. 1984 is not a criticism of socialism; it’s a criticism of totalitarianism. I think the point is a warning that all political ideologies can succumb to the allure of power and be corrupted by it. People on both the left and the right in the United States can be corrupted by power. This one-way criticism of policies as Orwellian is a convenient shorthand, but the reality is all politicians are capable of…
I’ve seen it, for example, when conservatives in the United States have used 1984 to call left-wing policies “Orwellian.” I think that’s an overstatement, of course used for dramatic effect, but it should at least be said that Orwell was a democratic socialist. 1984 is not a criticism of socialism; it’s a criticism of totalitarianism. I think the point is a warning that all political ideologies can succumb to the allure of power and be corrupted by it. People on both the left and the right in the United States can be corrupted by power. This one-way criticism of policies as Orwellian is a convenient shorthand, but the reality is all politicians are capable of…
Lex Fridman
creating an Orwellian world. And I think one of the things that is highlighted in the book very well is the hypocrisy of Winston. When O’Brien asks Winston what he’s willing to do to overthrow the Party, Winston admits he is willing to commit atrocities. He’s willing to do evil unto children, to commit murder, anything. This is a powerful illustration that both the totalitarian state and a blind, immoral rebellion against it can be evil. This is where I return to love as the thing that carries hope for a world beyond this battle for freedom. You have to have that.
creating an Orwellian world. And I think one of the things that is highlighted in the book very well is the hypocrisy of Winston. When O’Brien asks Winston what he’s willing to do to overthrow the Party, Winston admits he is willing to commit atrocities. He’s willing to do evil unto children, to commit murder, anything. This is a powerful illustration that both the totalitarian state and a blind, immoral rebellion against it can be evil. This is where I return to love as the thing that carries hope for a world beyond this battle for freedom. You have to have that.
Lex Fridman
Otherwise, the Orwellian state and the resistance to an Orwellian state can both destroy basic human rights and freedoms. I think in the character of Winston, that’s illustrated well. And I should also mention that there’s interesting writing… Now, I’m not obviously a scholar of Orwell, and there’s a lot of books been written and I should probably recommend them somewhere. There’s just great books written on 1984, on Orwell, on the historical context in which he was operating and all that kind of stuff. But as far as I see, Orwell also with 1984 and himself politically, he was not espousing the complete opposite of totalitarianism.
Otherwise, the Orwellian state and the resistance to an Orwellian state can both destroy basic human rights and freedoms. I think in the character of Winston, that’s illustrated well. And I should also mention that there’s interesting writing… Now, I’m not obviously a scholar of Orwell, and there’s a lot of books been written and I should probably recommend them somewhere. There’s just great books written on 1984, on Orwell, on the historical context in which he was operating and all that kind of stuff. But as far as I see, Orwell also with 1984 and himself politically, he was not espousing the complete opposite of totalitarianism.
Orwell
Lex Fridman
There is, again, democratic socialism—that there is value to the connection between human beings, that you have to lean on each other, help each other, that society is fundamentally more a cohesive collective than a completely disparate set of sovereign individuals. It’s both. And I think he was torn about that idea, because in order to resist a totalitarian state you have to fight for those basic individual freedoms. But at the same time, a well-functioning society allows for that freedom to manifest as collaboration. And so that’s the difficult challenge there.
There is, again, democratic socialism—that there is value to the connection between human beings, that you have to lean on each other, help each other, that society is fundamentally more a cohesive collective than a completely disparate set of sovereign individuals. It’s both. And I think he was torn about that idea, because in order to resist a totalitarian state you have to fight for those basic individual freedoms. But at the same time, a well-functioning society allows for that freedom to manifest as collaboration. And so that’s the difficult challenge there.
Lex Fridman
Again, that’s why he was a democratic socialist and the criticism of the book was against totalitarianism, of a centralized state that controls speech, thought, the press, and all the basic human freedoms. Controls truth. And I think a lot of people would ask the question, and I hear this tossed around: “Do we live in the world of 1984 today?” And I think that’s used as a shorthand to sort of criticize different policies and different governments. I generally don’t like the use of that kind of language because it’s basically crying wolf. If everything is 1984, if everybody is Hitler, then you’re not going to…
Again, that’s why he was a democratic socialist and the criticism of the book was against totalitarianism, of a centralized state that controls speech, thought, the press, and all the basic human freedoms. Controls truth. And I think a lot of people would ask the question, and I hear this tossed around: “Do we live in the world of 1984 today?” And I think that’s used as a shorthand to sort of criticize different policies and different governments. I generally don’t like the use of that kind of language because it’s basically crying wolf. If everything is 1984, if everybody is Hitler, then you’re not going to…
Lex Fridman
There’s no way to properly normalize the discussion of the lesser of two evils, which is ultimately what democracy is about. You have a collection of things you’re picking. They all kind of suck, but you want to pick the one that sucks the least. That’s human society, you know? That’s human nature. It’s messy. And so I don’t think we live in a 1984 state, but there’s a lot of elements that this book reveals about human nature and about the operation of a totalitarian state that we should be on the watch for. So surveillance, a state of doublethink, of controlling language, of being in a constant state of war as a way to control the population and the flow of resources.
There’s no way to properly normalize the discussion of the lesser of two evils, which is ultimately what democracy is about. You have a collection of things you’re picking. They all kind of suck, but you want to pick the one that sucks the least. That’s human society, you know? That’s human nature. It’s messy. And so I don’t think we live in a 1984 state, but there’s a lot of elements that this book reveals about human nature and about the operation of a totalitarian state that we should be on the watch for. So surveillance, a state of doublethink, of controlling language, of being in a constant state of war as a way to control the population and the flow of resources.
Lex Fridman
All those things have elements of almost useful tools for the establishment of complete control of a populace. And the moment you notice those elements, it’s our job to resist those elements. So I think the point is we have to be vigilant to the slippery slope of the will to power in centralized institutions. Another thing I want to mention is that I think a lot of people rightfully compliment Orwell for predicting some of the elements of future society, especially with technological capabilities, for example, telescreens used by the state to control the population. Maybe I can make a few comments on technology in general.
All those things have elements of almost useful tools for the establishment of complete control of a populace. And the moment you notice those elements, it’s our job to resist those elements. So I think the point is we have to be vigilant to the slippery slope of the will to power in centralized institutions. Another thing I want to mention is that I think a lot of people rightfully compliment Orwell for predicting some of the elements of future society, especially with technological capabilities, for example, telescreens used by the state to control the population. Maybe I can make a few comments on technology in general.
Technology
Lex Fridman
People who criticize technology will often use 1984 as an example that technology is a tool for a totalitarian state. It’s a way they can achieve full control, and we should be extremely cautious of it. And I think there’s a kernel of truth to that. But it’s not obvious to me that on the whole, technology is a tool for totalitarian control. I think it is also a tool for freedom. The internet is an incredible tool for freedom. And so of course, we have to fight for that freedom, but I believe in general, the greater… Let’s just take the internet broadly as an example, and there’s a lot of sub-elements of that, and like a more platonic sense of what the internet is, which is digital interconnectivity.
People who criticize technology will often use 1984 as an example that technology is a tool for a totalitarian state. It’s a way they can achieve full control, and we should be extremely cautious of it. And I think there’s a kernel of truth to that. But it’s not obvious to me that on the whole, technology is a tool for totalitarian control. I think it is also a tool for freedom. The internet is an incredible tool for freedom. And so of course, we have to fight for that freedom, but I believe in general, the greater… Let’s just take the internet broadly as an example, and there’s a lot of sub-elements of that, and like a more platonic sense of what the internet is, which is digital interconnectivity.
Lex Fridman
We have to fight for freedom, but in general, the greater reach and access that the internet has, the more powerful the resistance to totalitarianism. Technology is a double-edged sword. It provides the tools for oppression and the tools for the ongoing fight for freedom. And as long as the will to fight arises in the human heart, technology, I think, helps humanity win. And of course, there’s been a lot of discussion about free speech and the freedom of thought, and there’s a lot to be said there that’s much more nuanced than the book 1984 provides. I think 1984 just shows the end, horrible conclusion of complete totalitarian control over speech, over thought, over feeling, over everything. But in general, my view of it is it’s a kind of inspiration to…
We have to fight for freedom, but in general, the greater reach and access that the internet has, the more powerful the resistance to totalitarianism. Technology is a double-edged sword. It provides the tools for oppression and the tools for the ongoing fight for freedom. And as long as the will to fight arises in the human heart, technology, I think, helps humanity win. And of course, there’s been a lot of discussion about free speech and the freedom of thought, and there’s a lot to be said there that’s much more nuanced than the book 1984 provides. I think 1984 just shows the end, horrible conclusion of complete totalitarian control over speech, over thought, over feeling, over everything. But in general, my view of it is it’s a kind of inspiration to…
Lex Fridman
In order to prevent ourselves from slipping into an authoritarian or a totalitarian state, Orwellian type of dystopias—to avoid them, we have to value critical and independent thought. I think thought first, before speech. Just thought. I think you have to learn to think deeply from first principles, independent of whatever tribe you find yourselves in. Independent of government, independent of groups, independent of the people around you, the people you love, that love you. You have to learn, at least sometimes, to think independently. Now, this is the Nietzsche, “If you gaze long into the abyss, the abyss gazes into you.” If you think too independently, it can break your mind. I mean, we are social creatures. We need that connection.
In order to prevent ourselves from slipping into an authoritarian or a totalitarian state, Orwellian type of dystopias—to avoid them, we have to value critical and independent thought. I think thought first, before speech. Just thought. I think you have to learn to think deeply from first principles, independent of whatever tribe you find yourselves in. Independent of government, independent of groups, independent of the people around you, the people you love, that love you. You have to learn, at least sometimes, to think independently. Now, this is the Nietzsche, “If you gaze long into the abyss, the abyss gazes into you.” If you think too independently, it can break your mind. I mean, we are social creatures. We need that connection.
Lex Fridman
But I think it’s like with Tom Waits: “I like my town with a little drop of poison.” I think of truly, deeply independent thought as a little drop of poison that’s necessary for your mind. Most of your life you live, you kind of assume most things around you are true, and that’s very useful. We stand on the shoulders of giants. But you, on a regular occasion, have to question. Question your assumptions, question your biases, question everything. Question the things you’ve taken for granted. Question what everybody’s telling you. But not too much. It’s a tricky balance, but the act of rebellion against a totalitarian state, against the slippery slope into that state, is that independent thought. And of course, speech is a manifestation of that thought.
But I think it’s like with Tom Waits: “I like my town with a little drop of poison.” I think of truly, deeply independent thought as a little drop of poison that’s necessary for your mind. Most of your life you live, you kind of assume most things around you are true, and that’s very useful. We stand on the shoulders of giants. But you, on a regular occasion, have to question. Question your assumptions, question your biases, question everything. Question the things you’ve taken for granted. Question what everybody’s telling you. But not too much. It’s a tricky balance, but the act of rebellion against a totalitarian state, against the slippery slope into that state, is that independent thought. And of course, speech is a manifestation of that thought.
Lex Fridman
So we have to avoid echo chambers in both thought and speech. Like I said, you have to question your assumptions, challenge your biases. I think that’s the way out. Or maybe that’s a resistance mechanism to slipping into authoritarianism. And maybe I have a few more things to say about the latter part of the book, the part where there’s torture—where there’s Room 101 that has the thing you fear the most, which is different for all of us, and for Winston, that’s rats. It makes you wonder what that thing is for each of us. I left a mental note for myself to do more research into the historical context, the psychology, the neuroscience, the effectiveness of torture. I think there’s probably a lot of really good work.
So we have to avoid echo chambers in both thought and speech. Like I said, you have to question your assumptions, challenge your biases. I think that’s the way out. Or maybe that’s a resistance mechanism to slipping into authoritarianism. And maybe I have a few more things to say about the latter part of the book, the part where there’s torture—where there’s Room 101 that has the thing you fear the most, which is different for all of us, and for Winston, that’s rats. It makes you wonder what that thing is for each of us. I left a mental note for myself to do more research into the historical context, the psychology, the neuroscience, the effectiveness of torture. I think there’s probably a lot of really good work.
Lex Fridman
I had a brief conversation with Andrew Huberman on the phone about this topic. Andrew Huberman, the brilliant Andrew Huberman, host of the Huberman Lab podcast that you should listen to. And he mentioned to me that there’s a bunch of papers on these topics. This has been studied, sort of the carrot and the stick of the ability of incentives and disincentives to control the perception and the mental state of people and animals. And he mentioned to me a few folks that I could talk to on a podcast about this topic, and a few books. So, I’ll definitely look into this more. I think 1984 uses torture as a philosophical description, as a caricature of the operation of a totalitarian state.
I had a brief conversation with Andrew Huberman on the phone about this topic. Andrew Huberman, the brilliant Andrew Huberman, host of the Huberman Lab podcast that you should listen to. And he mentioned to me that there’s a bunch of papers on these topics. This has been studied, sort of the carrot and the stick of the ability of incentives and disincentives to control the perception and the mental state of people and animals. And he mentioned to me a few folks that I could talk to on a podcast about this topic, and a few books. So, I’ll definitely look into this more. I think 1984 uses torture as a philosophical description, as a caricature of the operation of a totalitarian state.
Lex Fridman
But at the same time, a lot of those elements were all done under Stalin in the Soviet Union, so it’s not like it’s very different or very far from reality. It’s very, very real. The question is about the actual effect it has on the human mind, which I really have to think because torture in this case breaks Winston. In fact, I’d like to believe that many people, in the most fundamental of ways, can’t be broken in this way. I’ve seen science… again, without extensively reading, so please correct me if I’m wrong. But I’ve seen science that shows that torture, for the purpose of intelligence gathering, is not effective.
But at the same time, a lot of those elements were all done under Stalin in the Soviet Union, so it’s not like it’s very different or very far from reality. It’s very, very real. The question is about the actual effect it has on the human mind, which I really have to think because torture in this case breaks Winston. In fact, I’d like to believe that many people, in the most fundamental of ways, can’t be broken in this way. I’ve seen science… again, without extensively reading, so please correct me if I’m wrong. But I’ve seen science that shows that torture, for the purpose of intelligence gathering, is not effective.
Lex Fridman
It’s not effective to get accurate information because people will tell you anything, really, to stop the torture, stop the physical and the mental and the emotional suffering. But I think this book is about the use of torture to completely break your ability to think and to perceive the world. One of the things I talked to Andrew about is whether it’s possible to control perception through these kinds of things. And it seems that there is literature that shows it’s possible to literally change your perception of the world. Like in this case, in 1984, it’s when you’re holding up four fingers, can you actually make the person believe that you’re holding up five fingers?
It’s not effective to get accurate information because people will tell you anything, really, to stop the torture, stop the physical and the mental and the emotional suffering. But I think this book is about the use of torture to completely break your ability to think and to perceive the world. One of the things I talked to Andrew about is whether it’s possible to control perception through these kinds of things. And it seems that there is literature that shows it’s possible to literally change your perception of the world. Like in this case, in 1984, it’s when you’re holding up four fingers, can you actually make the person believe that you’re holding up five fingers?
Lex Fridman
Not because of some weird delusion or just because your vision is blurry, but you literally, when you look, are holding four fingers and what you see is five fingers. Not because your vision is poor. No, your visual cortex, the way you’re processing that information, something about the processing changes completely your perception. If I tell you there’s a straight line, can, through incentive or disincentive, you start seeing a crooked line or something like that? Anyway, I think that there’s literature that supports that, which is, by the way, terrifying. But the thing I’d like to research more is if that can be long-lasting. I just don’t believe it can be.
Not because of some weird delusion or just because your vision is blurry, but you literally, when you look, are holding four fingers and what you see is five fingers. Not because your vision is poor. No, your visual cortex, the way you’re processing that information, something about the processing changes completely your perception. If I tell you there’s a straight line, can, through incentive or disincentive, you start seeing a crooked line or something like that? Anyway, I think that there’s literature that supports that, which is, by the way, terrifying. But the thing I’d like to research more is if that can be long-lasting. I just don’t believe it can be.
Lex Fridman
If you’re not pushed to your death, yes, maybe perception, maybe your willingness to think, but your actual ability to think independent thoughts? Maybe you’re terrified. I understand if you’re terrified of any more thinking that leads to rebellious thoughts. Like the book mentions, the idea of face crime, where you can reveal your thoughts, the inner workings of your mind, by the subtleties of your expressions in your face. And I think also, like O’Brien says, “If you want to keep a secret, you must also hide it from yourself.” So I can understand that.
If you’re not pushed to your death, yes, maybe perception, maybe your willingness to think, but your actual ability to think independent thoughts? Maybe you’re terrified. I understand if you’re terrified of any more thinking that leads to rebellious thoughts. Like the book mentions, the idea of face crime, where you can reveal your thoughts, the inner workings of your mind, by the subtleties of your expressions in your face. And I think also, like O’Brien says, “If you want to keep a secret, you must also hide it from yourself.” So I can understand that.
Lex Fridman
And maybe that is the basic mechanism that torture leads to: that your body, your mind learns to hide the truth from yourself. Like you don’t even allow yourself to think it because you know if you think it, it’s going to lead to face crime and thought crime, and that’s going to lead to more torture. That’s possible. But I just can’t imagine the capacity for love in the human heart to be extinguished through torture, finally extinguished. Temporarily, yes, but finally, irrecoverably, which I think is the basic claim of the book. That they break… so because through the worst of the torture Winston gives up Julia, the object of his love, he says that some things like that—the fact that you said, “Torture her, not me.”
And maybe that is the basic mechanism that torture leads to: that your body, your mind learns to hide the truth from yourself. Like you don’t even allow yourself to think it because you know if you think it, it’s going to lead to face crime and thought crime, and that’s going to lead to more torture. That’s possible. But I just can’t imagine the capacity for love in the human heart to be extinguished through torture, finally extinguished. Temporarily, yes, but finally, irrecoverably, which I think is the basic claim of the book. That they break… so because through the worst of the torture Winston gives up Julia, the object of his love, he says that some things like that—the fact that you said, “Torture her, not me.”
Lex Fridman
“Anything to make this stop,” the fact that you said that, the fact that you thought that, is a statement, is a thought you can’t walk back to yourself. So it’s irrecoverable. You just destroyed your faith in love? I don’t think so. I think it’s possible we have to remember that this is one particular character. This is one particular story. I think there’s a lot of people in which the capacity to love cannot be broken, no matter the torture. But that’s an interesting scientific question, but it’s also a human question. I think Man’s Search for Meaning—there’s a lot of books that explore those kinds of questions. In the worst of conditions that humans had to suffer through, what still persists? What is the source of meaning?
“Anything to make this stop,” the fact that you said that, the fact that you thought that, is a statement, is a thought you can’t walk back to yourself. So it’s irrecoverable. You just destroyed your faith in love? I don’t think so. I think it’s possible we have to remember that this is one particular character. This is one particular story. I think there’s a lot of people in which the capacity to love cannot be broken, no matter the torture. But that’s an interesting scientific question, but it’s also a human question. I think Man’s Search for Meaning—there’s a lot of books that explore those kinds of questions. In the worst of conditions that humans had to suffer through, what still persists? What is the source of meaning?
Lex Fridman
And I just think that the flame of love persists through atrocities, through torture, through suffering, through all of it. But the claim of the book is that yes, a totalitarian state can use torture to break even that, even that which leads to the only love you’re allowed to have, which is the love for Big Brother. So I think, practically speaking, from the Party’s perspective, I think the point of O’Brien’s torture of Winston was to suffocate the hope in his mind and heart, so there is no hope, by completely destroying the knowledge of what is and isn’t true, so being betrayed.
And I just think that the flame of love persists through atrocities, through torture, through suffering, through all of it. But the claim of the book is that yes, a totalitarian state can use torture to break even that, even that which leads to the only love you’re allowed to have, which is the love for Big Brother. So I think, practically speaking, from the Party’s perspective, I think the point of O’Brien’s torture of Winston was to suffocate the hope in his mind and heart, so there is no hope, by completely destroying the knowledge of what is and isn’t true, so being betrayed.
Lex Fridman
And this kind of Goldstein’s book about the society, not knowing if that’s true, not knowing anything about Julia, is basically having no emotional or intellectual ground to stand on. It’s very difficult to have a sense of where you are. To have hope, you have to have a sense of where you are and where things could be. And then you also betray yourself. To force you to be a hypocrite on your own deepest feelings of love, I think basically puts you in a place where there’s no hope, there’s no point. It’s apathy. It’s nihilism. And there, a hardworking member of society that is nihilistic is probably what the Party wants, because that human will not rebel.
And this kind of Goldstein’s book about the society, not knowing if that’s true, not knowing anything about Julia, is basically having no emotional or intellectual ground to stand on. It’s very difficult to have a sense of where you are. To have hope, you have to have a sense of where you are and where things could be. And then you also betray yourself. To force you to be a hypocrite on your own deepest feelings of love, I think basically puts you in a place where there’s no hope, there’s no point. It’s apathy. It’s nihilism. And there, a hardworking member of society that is nihilistic is probably what the Party wants, because that human will not rebel.
Lex Fridman
But on the point of hope, I should mention that there’s a kind of long-running theory that since the appendix… The appendix is about the details of Newspeak, the language that the Party is creating and enforcing. Because that appendix was written in the past tense, and it’s talking about Newspeak in the past tense and it’s written in English, sort of non-Newspeak, that means the Party and Newspeak and all of its elements that we see in the story are in the past. That the world from which the book is created has escaped that. And that’s a message of hope. That whatever the rebellion against the Party—whether it’s passionate lust and sex, whether it’s love, whether it’s seeking truth in a world full of lies—whatever it is, there’s a way out.
But on the point of hope, I should mention that there’s a kind of long-running theory that since the appendix… The appendix is about the details of Newspeak, the language that the Party is creating and enforcing. Because that appendix was written in the past tense, and it’s talking about Newspeak in the past tense and it’s written in English, sort of non-Newspeak, that means the Party and Newspeak and all of its elements that we see in the story are in the past. That the world from which the book is created has escaped that. And that’s a message of hope. That whatever the rebellion against the Party—whether it’s passionate lust and sex, whether it’s love, whether it’s seeking truth in a world full of lies—whatever it is, there’s a way out.
Lex Fridman
Again, to me, the way out is love. But that’s a hopeful message in this dystopian novel, that even these perfectly executed totalitarian states will fall. I took a few random notes here that maybe I’ll comment on. I wrote a quote: “The masses cannot rebel until they become conscious.” That might be either a Winston observation or an O’Brien statement. I’m not sure. But yeah, so you have to think, 80% plus are proles of the working class. They have the power if they want it, but they don’t want it. They don’t want to take it. That’s the whole point of the totalitarian state: to break your will for freedom, your desire for freedom, break your ability to know that you’re not free.
Again, to me, the way out is love. But that’s a hopeful message in this dystopian novel, that even these perfectly executed totalitarian states will fall. I took a few random notes here that maybe I’ll comment on. I wrote a quote: “The masses cannot rebel until they become conscious.” That might be either a Winston observation or an O’Brien statement. I’m not sure. But yeah, so you have to think, 80% plus are proles of the working class. They have the power if they want it, but they don’t want it. They don’t want to take it. That’s the whole point of the totalitarian state: to break your will for freedom, your desire for freedom, break your ability to know that you’re not free.
Lex Fridman
And that’s where all of it—the changing of history, the doublethink, the thought crime—all of that comes into play: the torture and the Ministry of Love. All of that is about preventing the populace from becoming conscious. And again, as per the cells discussion earlier, I wrote down the O’Brien quote: “The death of the individual is not death. The Party is immortal.” And this is just an interesting observation about the operation of a totalitarian state, that it’s the idea and a kind of amorphous symbol of the messianic figure in Big Brother is all you need for the Party to persist. That person doesn’t actually have to exist. Any one individual doesn’t have to exist.
And that’s where all of it—the changing of history, the doublethink, the thought crime—all of that comes into play: the torture and the Ministry of Love. All of that is about preventing the populace from becoming conscious. And again, as per the cells discussion earlier, I wrote down the O’Brien quote: “The death of the individual is not death. The Party is immortal.” And this is just an interesting observation about the operation of a totalitarian state, that it’s the idea and a kind of amorphous symbol of the messianic figure in Big Brother is all you need for the Party to persist. That person doesn’t actually have to exist. Any one individual doesn’t have to exist.
Lex Fridman
It’s just the division of society into high, middle, and low, and the oppression of the low by the high by the centralized Inner Party. That’s all you need, and the individual does not matter in that. And again, the way to fight that is to fight for individual freedoms. An interesting side note is just a quote I wrote down from Julia, I think: “If you keep the small rules, you can break the big ones.” And so she, in the book, is somebody that follows to the T all the rules of the Party. She attends all the committee meetings and all that kind of stuff, and just is like the model citizen from the perspective of the Party. And so that allows her to break the big rules, like having passionate sex with people—the really…
It’s just the division of society into high, middle, and low, and the oppression of the low by the high by the centralized Inner Party. That’s all you need, and the individual does not matter in that. And again, the way to fight that is to fight for individual freedoms. An interesting side note is just a quote I wrote down from Julia, I think: “If you keep the small rules, you can break the big ones.” And so she, in the book, is somebody that follows to the T all the rules of the Party. She attends all the committee meetings and all that kind of stuff, and just is like the model citizen from the perspective of the Party. And so that allows her to break the big rules, like having passionate sex with people—the really…
Lex Fridman
or falling in love, all the forbidden things. And I think that’s actually a good way to exist in the world. I think for a lot of us, there’s probably a bunch of things that bother us in the local world around us, in the bigger world, and I think you have to pick your battles. You have to not get lost in the muck of small battles if you want to have at least one or a few big victories in your life that make for a better world. I think, at least in my sense, it’s easy to get distracted by the little things that bother you in life.
or falling in love, all the forbidden things. And I think that’s actually a good way to exist in the world. I think for a lot of us, there’s probably a bunch of things that bother us in the local world around us, in the bigger world, and I think you have to pick your battles. You have to not get lost in the muck of small battles if you want to have at least one or a few big victories in your life that make for a better world. I think, at least in my sense, it’s easy to get distracted by the little things that bother you in life.
Lex Fridman
And I think staying focused on the big things, again, picking your battles, and staying with that for as long as possible, working your ass off to solve one problem for as long as possible, not giving up against impossible odds, against all the criticism—that’s the way to solve those big problems. And of course, that’s not what Julia is talking about. But in a sense, she is also, because in that particular case, a totalitarian state is the problem. And the way to rebel is to plant that seed of rebellion in each of the people she has sex with: that we are human, that we have lust for each other, that we have the ability to love each other, and that is the necessary act of rebellion there.
And I think staying focused on the big things, again, picking your battles, and staying with that for as long as possible, working your ass off to solve one problem for as long as possible, not giving up against impossible odds, against all the criticism—that’s the way to solve those big problems. And of course, that’s not what Julia is talking about. But in a sense, she is also, because in that particular case, a totalitarian state is the problem. And the way to rebel is to plant that seed of rebellion in each of the people she has sex with: that we are human, that we have lust for each other, that we have the ability to love each other, and that is the necessary act of rebellion there.
Lex Fridman
That is the big leap for her, at least in that kind of society. I should also mention that there’s a lot of interpretations of the different small and big things in this book. So it’s very possible in the case of Julia that Winston was played. He was set up with Julia. He was set up to feel all those things. He was set up to have that little secret cove where he can write on his desk in the diary and dream of rebelling against the state, dreaming of the Brotherhood. It’s unclear to me why an oppressive state would want people to have that little journey of desiring freedom in all its manifestations. I’m not sure.
That is the big leap for her, at least in that kind of society. I should also mention that there’s a lot of interpretations of the different small and big things in this book. So it’s very possible in the case of Julia that Winston was played. He was set up with Julia. He was set up to feel all those things. He was set up to have that little secret cove where he can write on his desk in the diary and dream of rebelling against the state, dreaming of the Brotherhood. It’s unclear to me why an oppressive state would want people to have that little journey of desiring freedom in all its manifestations. I’m not sure.
Lex Fridman
But maybe O’Brien’s statement that the purpose of torture is torture holds some wisdom: that to attain absolute power, you also have to have a willingness and a mechanism to attain absolute suffering in the populace. And maybe this is a way to maximize suffering: to give them hope before you crush it. Again, the way out to me and the takeaway from this book—the way out is love. Perhaps this is a good place to also mention a little bit of a fun little controversy that evolved over Twitter. So I posted a reading list quickly before heading off to a New Year’s party of books that I hope to read in 2023, and these are based on books that I asked people to vote on; these are many of the ones they selected.
But maybe O’Brien’s statement that the purpose of torture is torture holds some wisdom: that to attain absolute power, you also have to have a willingness and a mechanism to attain absolute suffering in the populace. And maybe this is a way to maximize suffering: to give them hope before you crush it. Again, the way out to me and the takeaway from this book—the way out is love. Perhaps this is a good place to also mention a little bit of a fun little controversy that evolved over Twitter. So I posted a reading list quickly before heading off to a New Year’s party of books that I hope to read in 2023, and these are based on books that I asked people to vote on; these are many of the ones they selected.
Reading list controversy
Lex Fridman
And they happened to be many of the books I’ve read many times throughout my life and really enjoyed, and they were like old friends that I love visiting and revisiting. Every time I read them, I get something new and they just read differently throughout life. You know, the way in my teens when I read The Stranger by Camus is very different than it was in my 20s and different in my 30s. I’ll say my favorite book now by Camus is probably The Plague, and all of that has evolved. With Dostoevsky, I read The Idiot several times. I read The Brothers Karamazov both in English and Russian, and Notes from Underground. I mean, I love Dostoevsky. And a lot of these books are just…
And they happened to be many of the books I’ve read many times throughout my life and really enjoyed, and they were like old friends that I love visiting and revisiting. Every time I read them, I get something new and they just read differently throughout life. You know, the way in my teens when I read The Stranger by Camus is very different than it was in my 20s and different in my 30s. I’ll say my favorite book now by Camus is probably The Plague, and all of that has evolved. With Dostoevsky, I read The Idiot several times. I read The Brothers Karamazov both in English and Russian, and Notes from Underground. I mean, I love Dostoevsky. And a lot of these books are just…
Lex Fridman
Yes, they are classics, but they’re also deeply profound and they move me on an intellectual level, but also just as a human being. They’re like travel companions. They’re like old friends. Old dead friends. So yeah, I wanted to celebrate my love for books. And it was very strange to me that—and if I’m just being honest for a second, it was kind of painful that some prominent figures that I respect were kind of cruel about the list. They responded and they mocked it and all that kind of stuff, basically taking the worst possible interpretation. I have to be honest and say it wasn’t fun, because it was just a silly kid—me—kind of in a joyful New Year’s mood, sharing with the world books I love.
Yes, they are classics, but they’re also deeply profound and they move me on an intellectual level, but also just as a human being. They’re like travel companions. They’re like old friends. Old dead friends. So yeah, I wanted to celebrate my love for books. And it was very strange to me that—and if I’m just being honest for a second, it was kind of painful that some prominent figures that I respect were kind of cruel about the list. They responded and they mocked it and all that kind of stuff, basically taking the worst possible interpretation. I have to be honest and say it wasn’t fun, because it was just a silly kid—me—kind of in a joyful New Year’s mood, sharing with the world books I love.
Lex Fridman
And I think what was happening—and this seems to be happening a bit more—is there’s a bunch of people that are just almost waiting or hoping that I fail, or maybe that I’m some kind of bad human being. They’re looking, they’re trying to discover things about me that reveal that I’m a bad human being, and maybe somehow this reading list reveals that. I don’t know. So, one criticism was that everybody read these books in school, and they’re basic. I think my response to that criticism is: no. First of all, most people have not read them in school; maybe they read CliffNotes. And they’re not basic; they’re deeply profound, some of the greatest words ever written.
And I think what was happening—and this seems to be happening a bit more—is there’s a bunch of people that are just almost waiting or hoping that I fail, or maybe that I’m some kind of bad human being. They’re looking, they’re trying to discover things about me that reveal that I’m a bad human being, and maybe somehow this reading list reveals that. I don’t know. So, one criticism was that everybody read these books in school, and they’re basic. I think my response to that criticism is: no. First of all, most people have not read them in school; maybe they read CliffNotes. And they’re not basic; they’re deeply profound, some of the greatest words ever written.
Lex Fridman
But also, I don’t think I’ve ever gotten a lot from books I was forced to read in school when I had to read them for an assignment. Some of these books I think I read in school, but most of them not. It’s only when I read them outside of school on my own volition that I really gained a lot from it, and especially throughout my life at regular times—as a teenager, in my 20s, and in my 30s. So, no. These books are profound and deserve returning to. Like I said, they are old friends that give me a lot of meaning every time I revisit the ideas, and they give me a new perspective on life. Another criticism was very nitpicky. The list was put together really quickly, and the goal—I like setting tough goals.
But also, I don’t think I’ve ever gotten a lot from books I was forced to read in school when I had to read them for an assignment. Some of these books I think I read in school, but most of them not. It’s only when I read them outside of school on my own volition that I really gained a lot from it, and especially throughout my life at regular times—as a teenager, in my 20s, and in my 30s. So, no. These books are profound and deserve returning to. Like I said, they are old friends that give me a lot of meaning every time I revisit the ideas, and they give me a new perspective on life. Another criticism was very nitpicky. The list was put together really quickly, and the goal—I like setting tough goals.
Lex Fridman
The goal was to read a book a week. And, you know, on one week I had The Little Prince followed by The Brothers Karamazov. And people criticized that: “How can you possibly read The Brothers Karamazov in one week?” Maybe I won’t. Maybe I’ll fail miserably. But I love trying. But that wasn’t actually the goal. I should’ve said I intend to finish reading it by the end of that week. So, you start earlier because The Little Prince takes an hour or two to read. And then for The Brothers Karamazov, I could have the two weeks. It should take about 30, 40, or 50 hours to read it. That said, friends, I’ve read it already in English and in Russian.
The goal was to read a book a week. And, you know, on one week I had The Little Prince followed by The Brothers Karamazov. And people criticized that: “How can you possibly read The Brothers Karamazov in one week?” Maybe I won’t. Maybe I’ll fail miserably. But I love trying. But that wasn’t actually the goal. I should’ve said I intend to finish reading it by the end of that week. So, you start earlier because The Little Prince takes an hour or two to read. And then for The Brothers Karamazov, I could have the two weeks. It should take about 30, 40, or 50 hours to read it. That said, friends, I’ve read it already in English and in Russian.
Lex Fridman
I’m interviewing the world-famous, amazing translators of The Brothers Karamazov, of Dostoevsky, and of Tolstoy—Richard Pevear and Larissa Volokhonsky—probably across multiple days. So, this book means a lot to me. I’m not somebody just kind of rolling in, “What are the cool kids reading these days?” These books have been lifelong companions to me. And the fact that people just want to stomp on that—and a large number of people did, people I respect—yeah, I’d be lying if I said it didn’t suck a bit. Anyway, the love for reading persists. I have to say, after that, I was very hesitant to even make this particular video on Orwell, on 1984. And I’m not sure I want to be public with my reading after this.
I’m interviewing the world-famous, amazing translators of The Brothers Karamazov, of Dostoevsky, and of Tolstoy—Richard Pevear and Larissa Volokhonsky—probably across multiple days. So, this book means a lot to me. I’m not somebody just kind of rolling in, “What are the cool kids reading these days?” These books have been lifelong companions to me. And the fact that people just want to stomp on that—and a large number of people did, people I respect—yeah, I’d be lying if I said it didn’t suck a bit. Anyway, the love for reading persists. I have to say, after that, I was very hesitant to even make this particular video on Orwell, on 1984. And I’m not sure I want to be public with my reading after this.
Lex Fridman
And I know a lot of people will say, “No, we’re here with you.” They’re very supportive, and I love you. I mean, I meet so many incredible people, but the reality is it just does suck to be vulnerable and share something with the world and receive that kind of mockery at scale. So I will definitely—I will not be affected or broken by any of that kind of stuff for something that’s actually meaningful, like the conversations—some of the very difficult conversations I’m going to do. But for a silly side hobby thing of reading that I do throughout my life to be a source of mockery, I’m just going to do that privately. So, I’m a little torn on that, and I’ll try to figure out a way.
And I know a lot of people will say, “No, we’re here with you.” They’re very supportive, and I love you. I mean, I meet so many incredible people, but the reality is it just does suck to be vulnerable and share something with the world and receive that kind of mockery at scale. So I will definitely—I will not be affected or broken by any of that kind of stuff for something that’s actually meaningful, like the conversations—some of the very difficult conversations I’m going to do. But for a silly side hobby thing of reading that I do throughout my life to be a source of mockery, I’m just going to do that privately. So, I’m a little torn on that, and I’ll try to figure out a way.
Lex Fridman
Also, I should say that that list, like a lot of things, is kind of aspirational because if I take a job at a tech company, or if I start a tech company, or if I have to travel for extremely difficult conversations and really have to prepare for them—all that kind of stuff is going to affect my ability to both read and enjoy reading, which I think is a prerequisite for this kind of reading. But in general, what I do is I read about one hour a day on Kindle—on the sort of physical device, in my eyes. And depending on the workout I do and the chores I have, it’s going to be about two hours of audiobooks. So, most of the things I do during chores is audiobooks.
Also, I should say that that list, like a lot of things, is kind of aspirational because if I take a job at a tech company, or if I start a tech company, or if I have to travel for extremely difficult conversations and really have to prepare for them—all that kind of stuff is going to affect my ability to both read and enjoy reading, which I think is a prerequisite for this kind of reading. But in general, what I do is I read about one hour a day on Kindle—on the sort of physical device, in my eyes. And depending on the workout I do and the chores I have, it’s going to be about two hours of audiobooks. So, most of the things I do during chores is audiobooks.
Lex Fridman
And when I run—and I usually run about 10 to 15 miles, so you’re talking about—I often run over two hours. It’s like a slow pace. When the days are not insane, it gives me a chance to think and a chance to listen to audiobooks, so I love that process. It’s an escape from the world, a chance for me to collect my thoughts. And yeah, it’s again a source of happiness and joy, and I wanted to share that. I think you can get quite a lot of reading done through that process, especially if it’s a book you’ve read before. It is very challenging to do this kind of takeaway video, or to concretize your thoughts down on paper, especially when you have to present them in this kind of way.
And when I run—and I usually run about 10 to 15 miles, so you’re talking about—I often run over two hours. It’s like a slow pace. When the days are not insane, it gives me a chance to think and a chance to listen to audiobooks, so I love that process. It’s an escape from the world, a chance for me to collect my thoughts. And yeah, it’s again a source of happiness and joy, and I wanted to share that. I think you can get quite a lot of reading done through that process, especially if it’s a book you’ve read before. It is very challenging to do this kind of takeaway video, or to concretize your thoughts down on paper, especially when you have to present them in this kind of way.
Lex Fridman
I’m not sure I’m going to do that much, because it’s an extra bit of effort. But it’s also a chance to share that joy with the world, and to find cool people that also enjoy it. So it’s a trade-off. Anyway, it’s just a temporary thing, but it did suck for a short amount of time—for a few hours, for a couple of days. But in general, I’ll persist with my love of reading. I might not talk about it publicly as much. But again, let me emphasize that this kind of response and mockery will not affect anything of importance that I do. I try to read comments; I try to see criticism. I really value especially high-effort criticism. I try to grow and constantly try to improve.
I’m not sure I’m going to do that much, because it’s an extra bit of effort. But it’s also a chance to share that joy with the world, and to find cool people that also enjoy it. So it’s a trade-off. Anyway, it’s just a temporary thing, but it did suck for a short amount of time—for a few hours, for a couple of days. But in general, I’ll persist with my love of reading. I might not talk about it publicly as much. But again, let me emphasize that this kind of response and mockery will not affect anything of importance that I do. I try to read comments; I try to see criticism. I really value especially high-effort criticism. I try to grow and constantly try to improve.
Lex Fridman
But that’s for things that I take very seriously, like the podcast conversations that I do. But for silly things, like book lists, Spotify music playlists, the food I like to eat—I don’t know, anything, any fun side thing—it’s not that important. If it’s something that others don’t enjoy, then whatever. I’ll enjoy them probably with my friends locally here, or the people I meet. So, anyway, I love reading. I love reading classics. I love returning to old friends in book form, and making new ones.
But that’s for things that I take very seriously, like the podcast conversations that I do. But for silly things, like book lists, Spotify music playlists, the food I like to eat—I don’t know, anything, any fun side thing—it’s not that important. If it’s something that others don’t enjoy, then whatever. I’ll enjoy them probably with my friends locally here, or the people I meet. So, anyway, I love reading. I love reading classics. I love returning to old friends in book form, and making new ones.
Lex Fridman
There’s a bunch of science fiction that I embarrassingly have not read and would love to, because those worlds are so meaningful to so many of the people I’m friends with that I can’t wait to visit those worlds and sort of make new friends in the form of books. So, definitely the love for books, the love for reading persists. And if you share in that love, that’s beautiful. So thank you for joining me on this journey. Thank you for watching this silly little video. And I hope to see you next time. Love you all.
There’s a bunch of science fiction that I embarrassingly have not read and would love to, because those worlds are so meaningful to so many of the people I’m friends with that I can’t wait to visit those worlds and sort of make new friends in the form of books. So, definitely the love for books, the love for reading persists. And if you share in that love, that’s beautiful. So thank you for joining me on this journey. Thank you for watching this silly little video. And I hope to see you next time. Love you all.