I think my thoughts are coalescing into "AI as an imprecise patternmatcher". An analogy that works for me is that AI will happily paint by numbers, and can do a decent job of it if the number colours and amount of shapes to paint remains low enough. It sees a pattern, it grabs its paint and goes to town. It's often imprecise - it might not always have good hand-eye coordination (model size), it might cross lines here and there (hallucinate), it might need something to keep its hands steadier (grounding). You can try and wring more out of it? And it will, from a distance, look good. Might even be hung on a wall and make for a lovelier room. But any painter with an eye for detail will look right through it. And if it's too inprecise, non-experts will notice, too (slop). I think the current AI boom is resting entirely on finding ways to reduce the imprecision through more patternmatching. Now there's a lot of strides made, but there are fundamental limits to the nails we can hit with the transformer-hammer and I feel like we need fundamental discoveries if we want AI models to actually develop a world model, to actually reason instead of pattern-matching what reasoning looks like in an imprecise roughly-good direction. I mean, did you see what Apple concluded? "nuanced understanding" you mean, it sucks at reasoning. just say it sucks Now if your job is just painting by the numbers? I'd be worried, yeah. I mean, let's be real, how many average people just clock in and out of their average jobby job? Answering emails, sitting in meetings, writing a document here or there, all with enough corpospeak and jargon and shibboleths thrown in to make it obtuse to normal people. How much of that isn't pattern matching, really? I recenly started tracking my time again, basically divided into "deep work" (expertise), "email" and "meetings" (patternmatching, basically). The former is only 45% of my entire week. You mentioned Graeber before, but I'm not sure if that's fair, because the patternmatcher doesn't care if it's matchin' patterns for the next TPS report or at a charity to cure rare cancers. "We've always done it that way" tasks and jobs now have their head put under the guillotine, and they are not few or far between I think. Journalism...yeah, not looking great.
We hypothesize that this decline is because current LLMs cannot perform genuine logical reasoning; they replicate reasoning steps from their training data. Adding a single clause that seems relevant to the question causes significant performance drops (up to 65%) across all state-of-the-art models, even though the clause doesn't contribute to the reasoning chain needed for the final answer. Overall, our work offers a more nuanced understanding of LLMs' capabilities and limitations in mathematical reasoning.
One of my "favorite things" about finance - and by "favorite things" I mean "thing that had I known about it as a child would have colored my impression of the moneyed and their pursuits in a decidedly negative way" - is technical analysis. "Well of course, klein!" you say. "You're an annoyingly technical person, of course you love technical analysis." Ahhh but here's the thing - "technical analysis" as espoused by the financiers isn't technical, and it isn't analysis. I read a whole goddamn book on technical analysis just to see if there was anything there. There isn't. Burton Malkiel ran a bunch of tests where he gave TA dipshits a ticker that had been generated by a literal coin flip and then asked them how they thought their "analysis" was doing. Some were happy, some were sad, none clued into the fact that they were using all their tools to scry the behavior of a coin toss. The technical analysts won't even dispute this. They'll argue that technical analysis is so powerful that it can produce false positives and false negatives from random number generators so you'd best try even harder. True practitioners will lock themselves out from all noise sources. Some have even argued that they trade better if they don't know the security they're trading. All that matters is what magic shapes they draw on their graph to determine what the next candle is going to be. That's literally the way Markov chains work. The fundamental basis of LLMs is pattern recognition where the process is actually hindered by too much horizon. They work better if they're only looking ahead a little bit. They don't analyze shit, and they can't. They know that in 100 runs of seven times five, the answer is 35, one hundred times over. But if they need to know what seven hundred times five pi is, they don't have 100 runs. So they get it wrong sometimes. Because they're not doing math. They're looking up values in a table and if there are holes, they're extrapolating over the top of it. I'm willing to bet Apple didn't say "LLMs suck at reasoning, duh" for the same reason they rolled their eyes and coughed up a $3200 nerd helmet - nobody is willing to talk about the emperor's new clothes yet. There is no part of the methodology underlying LLMs that bears even a passing resemblance to reasoning. It's like saying Tesla's Autopilot sucks at conversational Mongolian - Why wouldn't it? Ahhh - but you can set the UI to Mongolian so isn't that conversing? Here's the other part: It's Pareto principle all the way down. Everything OpenAI or any of the other vendors have ever done is a solid B minus. Everything they do is 80% effort. It's not quite a C? But it's super-close. The ouvre of commercial AI is just good enough not to make your parents sign your homework. But for a lot of stuff, that's plenty. I don't need an A-plus meme, I need a B-minus meme NOW. I don't need an A-plus essay, I need a B-minus essay NOW. One of the things about being on set is everyone on set can do 80% of everyone else's job on set. We've all been on set long enough that we know the easy steps. Do something hard? yer fukt. You hire the experts because when you're in a pinch, they know what to do. it takes me 15 seconds to explain how to mix major-market house reality television to any schlub who walks through the door - we used to do it as a party trick. Sure, your daughter can sit at the console. Absolutely Miss Celebrity can throw on some headphones. But if things get dicey you'd best get out of the chair quick because I don't even know if I can explain to you how to fix what just happened. Nobody ever asks intelligent questions, they ask the same stupid ones. Except Francis Ford Coppola. He came in and chatted with us (we didn't know who he was at the time, just that the producers were terrified) for a good fifteen minutes and asked some really insightful questions. And yer goddamn right - once I figured out I had been having a lengthy technical conversation with the writer/director of The Conversation I was over the moon. Marketing schmucks? They don't really understand the Pareto Principle. Some of them are geniuses and they know it. Most of them occasionally catch lightning in a bottle, and that keeps them employed long enough to continue to muddle through. So all their ads for AI are about selling the 80% as if it were the 20%. They don't know, they don't understand it, and they can't tell the difference. The guys writing AI? They hope you can't tell the difference. Let's talk paint-by-number because I think it's an interesting analogy. The thing about paint-by-number kits is they're generally bought by people who enjoy painting. Painting by number incrementally builds their skills. You do that enough, you might become an artist. I mean, jingle trucks are basically paint-by-number; tell me these guys aren't skilled. Give 'em a blank spot and they will synthesize. They have their bag'o'tricks for sure but the good ones are novel. It is physically impossible for LLM-based AI to be novel. It can arrive at an original place on the look-up table but it will never step out of bounds. It will muddle through just fine in the 80% but the 81% is luck only. 85% is a fluke. 90% is virtually impossible. I think with academia and with journalism the problem is nobody needs the 80%. Essays have always been an imperfect analog of knowledge. A journalist can research undiscovered facts and can synthesize unformulated opinions. AIs can do neither but since so much of what both students and journalists produce isn't actually within the purvey of what they're supposed to be doing, the AI can do a B-minus level approximation of that make-work. That is recognizably a jingle truck. It's not even obviously AI. You mentioned Graeber before, but I'm not sure if that's fair, because the patternmatcher doesn't care if it's matchin' patterns for the next TPS report or at a charity to cure rare cancers. "We've always done it that way" tasks and jobs now have their head put under the guillotine, and they are not few or far between I think. Journalism...yeah, not looking great.
