“…I think ironically, we're at the point where the thing that was built to generate text, the thing we needed to do the least is generate text.” Kris Brandow · Fallthrough

I think, I think, too, we've lost the plot on the, because I ran into this when I was building something, this thing that has taken for like 5 billion tokens or more. Part of it, like, I had sat down and did a design with GPT, my chat GPT. And I was like, okay, I'll have Claude build this thing. So I took the design and gave it to Claude. And then Claude went through and I was like, well, there are several problems with this design, blah, blah, blah. And I said, here's one thing that's probably not going to work correctly. And I was like, that doesn't seem right because this one did deep research. So it presumably confirmed that this API that exists could actually work. And then I was like, maybe it doesn't. Maybe the other agent got it wrong. But then I went back and I was like, oh, wait, you didn't actually look at how the tool works. You just made an assumption from some docs pages, but you didn't look at the right docs pages. So you got confused about the capabilities of what this tool could do. And then you tried to come up with a solution for that. And then you came up with this whole big plan that I did not want to read because it's so freaking long. You spit out so much. It's too wordy. Anyway, but I was like, oh, it would have been better if you just looked at the tool, but you instead looked at some docs page and then got yourself confused and then made some statements based on that. It's like, I think there's too much now. There's too much writing, right? When you have paragraphs and paragraphs of comments, like this is how I actually think that this project has taken like 5 billion tokens of cash read is because there's so many comments mixed with so much code that every time it wants to read something, it's got to like hoover up a tremendous amount of stuff in order to understand what's happening. So then it can write some code. And then it, it's just like, we, we've, we've definitely lost the plot there. I think there was a point in time when, like, it does make sense a little bit that like comments would be useful, but like the LLMs can just read the code, right? They, they actually can read the code and understand it. Like people can't do it very well, right? If I sit down and I try to read some esoteric code, good luck. Like if it's really like gnarly, I'm probably going to stumble over. It's going to be much nicer if there's just a comment there. But like LLMs are very good at understanding gnarly code. If you hand it, it's like, no, this is how this thing works. They can like reason through, especially more modern LLMs can like reason through it. So like, I don't even think we need it for writing code in that way. Like, I think ironically, we're at the point where the thing that was built to generate text, the thing we needed to do the least is generate text. Because like, I think we all forgot how expensive words are. In the same way that like lines of code are expensive, I think words are expensive. And when you only have like a 250,000 token context window or maybe like a high and a million, but it really doesn't really go as far as I think people think it does over the course of like a long conversation. You can't just be like wasting words on stuff. Like that's extra context you're eating up. That's putting pressure on the system. So it's not going to perform as well as it could if things were more concise. So I think the idea that we can just generate the docs with LLMs, it's like only if you understand how to do concise writing, which is a very hard thing, right? We know this. There's the famous quip that everybody talks about of like, sorry I wrote you such a long letter. I didn't have time to write a shorter one. Where it's like, actually getting down to concise, compact writing takes a lot of time, takes a lot of iterations and rounds. You have to choose what things you include, what things you don't include. It's a difficult thing to do. But I do think that probably the problem here, like a big part of the problem here is, as I kind of alluded to at the beginning of the show, the literacy crisis in that like people can't read anymore. And I think if you can't read, you probably can't write, which is probably a lot of this. And like, to be clear here, when I say like people can't read, I don't mean that they literally like cannot read words. Like that's not what the literacy crisis that I think anybody that's seriously talking about it is really talking about. It's not like people can't pick up a book and read the words. It's people can't pick up the book, read the words, and then comprehend what it actually says over more than maybe a paragraph or two, right? Like people struggle a lot to comprehend not even long-form writing, but short, but medium, short to medium term, like length writing, right? Where it's like a few hundred words, people are struggling to really understand what is actually being said in those words. And I think that's potentially why a lot of people who are not like the like critical or close readers can probably consume a lot of this LLM writing and think that it's fine. And probably also why people that generate LLM posts think that they're fine. Because I think there's likely a degree of they themselves can't understand why this writing is not good. Right. They don't understand where the problems are because they can't comprehend what they've written themselves outside of the concept of it that they have in their mind. Right. I'm very sure that when anybody is writing, that's just a concept of a thing that they want to communicate. I don't know if people can then write it and then read it and then reconcile whether what they wrote is what is actually on that page or not.