“…the purpose of a product is not to build a fucking website. If it is, then yeah, maybe I can do it and maybe Squarespace can do it. Maybe like a dude in Timbuktu could do it.” Nikolai Yakovenko · Razib Khan's Unsupervised Learning
Ideas & Essays · September 2026
Yakovenko had just conceded that kernel optimization, long a specialist and highly paid job, is something models now do well because the task is measurable and contained. He uses that to draw the line: the replicable part of the work is the part that falls, and it was never the product. He then walks through a jiu-jitsu ranking site he built, where AI did the scraping he would never have done by hand and the actual work was still getting anyone to use it.
Transcript
Oh, super important. Yeah. Especially that's what I'm doing these days. So, so yeah, if you think about like the pre-training is kind of what you think about as a sort of traditional LLM, you know, your GPT-3 or whatever, where you take the entire web or a huge, huge, huge corpus and you train, you know, for a long time on like a very simple task, basically like fill in the blank, sentence completion, whatever. Really, really basic, very, very general. You train on that for a while. But I guess there's like mid-training, which is like sort of like learning skills. We can sort of skip that. But like post-training is, is you actually want to take this like sort of raw intelligence, so to speak, and you want to kind of like, you could think about it as like metallurgy or something. You want to refine it into the specific, you know, is going to do this on that, you know? So, because if you train a model on the web and then you ask it like questions, it doesn't even, you know, it's not trained on answering questions, for example, or specifically your thing, like medical questions or international tax questions, like you mentioned before. So post-training is basically taking all of those tasks. Obviously, the most important by far is coding, but building data sets and having small teams take like a raw model and having a script optimize it for this task without ruining everything else. And inherently, this has to be split up. There's no magic silver bullet, whatever cliche you want to use. You have to do it on a bunch of different tasks with a bunch of different tasks. The question for the labs becomes how do you sort of merge this knowledge? And that's a headache as well. But that's kind of how everything is done. Now, then on top of it, you have the harness, which is the cloud code, which decides kind of like how to transform your commands into like other prompts and checking things. And that's, you know, the biggest thing these days. But fundamentally, post-training is incredibly important to teach a model to sort of do any actual task. Because otherwise, you're just doing pre-training, whereas
here's a question that I have. You know, one of the things that machine learning people would always say is like, well, you know, the big deal with machine learning is like you need humans a lot to kind of make it work and train the machine learning algorithms and whatever not. So even though people are like, oh, it'll just do everything for you and, you know, whatever, you know, these neural networks and all these other things. Where are we, you know, at in terms of AI, in terms of how much, you know, because you have, you have a company, you might want to talk about this real quickly. I mean, you're obviously you have like, you're a human being who uses AI to produce products. Why, why are you still relevant? Why can't you just give it a single command prompt? Are you still relevant? You know, I don't know if we can get an honest answer out of you, but you know what I'm saying?
Yeah, I mean, I think that's like a pretty silly question, but let's sort of address that in pieces. So I think, look, if, yes, if you see your job as doing a particular replicable task, then you still need somebody who understands that task to essentially post-train a model to do it. But in theory, then you're good, right? So the latest example that no one's really too upset about is like kernel optimization, like the highest paid, most important sort of like specialist job. You know, this is like the guys, you know, tall guy standing in the corner shooting three-pointers. A very specialty job, but it used to be a very important one, kernel optimization. You made some model thing, you'd made some change, you have some new hardware. How do you basically get this to run really, really, really efficiently on GPUs? And these people were, you know, very smart, but their brain worked in a particular way. And very hard to do. Like, I've never done it. Very, very hard. Right. And now, apparently, you know, LLMs are pretty good at it, right? Which makes sense because it's high value, it's measurable, it's code, it's contained. You know what I mean? You can actually see if the kernel works better because literally the point of the kernel optimization is input-output is the same, make it fucking work better in a GPU. So collecting those data sets is not easy. You have to know what you're doing, but you can sort of get models better at kernel tasks. So, but that's not really what most people's job is. And that's certainly not the point of a product, right? So I think the big thing people are finding out is that the purpose of a product is not to build a fucking website. If it is, then yeah, maybe I can do it and maybe Squarespace can do it. Maybe like a dude in Timbuktu could do it. You know what I mean? That's not your goal. So for example, I'll talk about two things. First, I'll talk about sort of like the simpler project that I did, which is a jiu-jitsu website. So I wanted to build a website for sort of central repository of, you know, kind of like, you know, you look at the MMA records for UFC, they're like the guys, you know, 21 and 10. Too, here's who he's beaten, you know. Here's where he's ranked. That didn't really exist in jiu-jitsu. It was kind of scattered. So it was mostly, I wanted to do the rankings. That's the fun part, the math part, stuff I know very well. But, you know, all the work was in like collecting and organizing the data. So you can look up anybody on my website who can beat some high-level jiu-jitsu and like who have they beaten? You know, what are their results? You know, what do they rank and sort of what their odds would be against somebody else? You know, like every time there's a big tournament coming up, I do some predictions and people enjoy those. Right. So there, if you think about it, so the AI is instrumental. As my friend keeps pointing out, I would have, you know, would I would I have done the site if AI wasn't around? The answer is no. Would I do the ranking? Yes. Would I do like the data? Fuck no. Like you have to scrape different sites and figure out their scraping structure, build a UI. And AI is like immeasurably helpful for that. Having said that, like, okay, what do you do now? Well, you still have a product. You need to promote the site. You need to talk to people. You need to get people using it. So I haven't really done very much of that. So it gets some popularity. Every once in a while, it becomes popular. And some of the most famous people in jiu-jitsu share my stuff and link to it. Or they'll just like LOL. You think I'm, you know, this guy's better than me kind of thing. And, you know, so that's the product, right? And AI is just an enabling technology. So that's like, that's like just a normal website. There's nothing really AI about it. And then for the, you know, for the, for the, for the fine-tuning models to like write better at news, I mean, that we can get into that. Sorry, I'm telling a long story, but that is an interesting case. So maybe we can break that one down because that's, that's another case for sort of post-training. But, you know, it's not, even though writing news headlines is obviously seems much easier than kernel optimization. And it is, and I'm not saying it's nearly as important. It's not, but you know, measuring what makes a good headline is like more vague, right? Then, you know, what does this kernel run faster? But still, but still, we run a relatively similar process where, you know, you're basically collecting a bunch of rules. You're basically like learning a reward function. Like most of the work goes into learning the RL reward function. So basically, like, you know, Gamma or Quinn or whatever model we're using, it can write decent headlines already.
Yeah.
So then it's, you know, you have it generate like 16 versions of, you know, like basically its top 16 choices, essentially. And then you have a model that ranks it, you backprop the weights. And even that's pretty hard, you know? I mean, it's hard for two reasons. It's actually still a little bit hard to run these things yourself. Like there's libraries and shit for it, but it's still a little bit sticky, you know? Yeah. It's not as simple as like, you know, like clicking one button. But then the main thing is, okay, you give it rewards and then it like hacks those rewards. It either doesn't learn at all. Okay, three things usually happen. It doesn't learn at all. It like crashes and explodes because the gradients were too high.
Okay. And it's
More from Razib Khan's Unsupervised Learning
