In 2023, one popular perspective on AI went like this: Sure, it can generate lots of impressive text, but it can’t truly reason — it’s all shallow mimicry, just “stochastic parrots” squawking.
At the time, it was easy to see where this perspective was coming from. Artificial intelligence had moments of being impressive and interesting, but it also consistently failed basic tasks. Tech CEOs said they could just keep making the models bigger and better, but tech CEOs say things like that all the time, including when, behind the scenes, everything is held together with glue, duct tape, and low-wage workers.
It’s now 2025. I still hear this dismissive perspective a lot, particularly when I’m talking to academics in linguistics and philosophy. Many of the highest profile efforts to pop the AI bubble — like the recent Apple paper purporting to find that AIs can’t truly reason — linger on the claim that the models are just bullshit generators that are not getting much better and won’t get much better.
But I increasingly think that repeating those claims is doing our readers a disservice, and that the academic world is failing to step up and grapple with AI’s most important implications.
I know that’s a bold claim. So let me back it up.
“The illusion of thinking’s” illusion of relevance
The instant the Apple paper was posted online (it hasn’t yet been peer reviewed), it took off. Videos explaining it racked up millions of views. People who may not generally read much about AI heard about the Apple paper. And while the paper itself acknowledged that AI performance on “moderate difficulty” tasks was improving, many summaries of its takeaways focused on the headline claim of “a fundamental scaling limitation in the thinking capabilities of current reasoning models.”
For much of the audience, the paper confirmed something they badly wanted to believe: that generative AI doesn’t really work — and that’s something that won’t change any time soon.
The paper looks at the performance of modern, top-tier language models on “reasoning tasks” — basically, complicated puzzles. Past a certain point, that performance becomes terrible, which the authors say demonstrates the models haven’t developed true planning and problem-solving skills. “These models fail to develop generalizable problem-solving capabilities for planning tasks, with performance collapsing to zero beyond a certain complexity threshold,” as the authors write.
That was the topline conclusion many people took from the paper and the wider discussion around it. But if you dig into the details, you’ll see that this finding is not surprising, and it doesn’t actually say that much about AI.
Much of the reason why the models fail at the given problem in the paper is not because they can’t solve it, but because they can’t express their answers in the specific format the authors chose to require.
If you ask them to write a program that outputs the correct answer, they do so effortlessly. By contrast, if you ask them to provide the answer in text, line by line, they eventually reach their limits.
That seems like an interesting limitation to current AI models, but it doesn’t have a lot to do with “generalizable problem-solving capabilities” or “planning tasks.”
Imagine someone arguing that humans can’t “really” do “generalizable” multiplication because while we can calculate 2-digit multiplication problems with no problem, most of us will screw up somewhere along the way if we’re trying to do 10-digit multiplication problems in our heads. The issue isn’t that we “aren’t general reasoners.” It’s that we’re not evolved to juggle large numbers in our heads, largely because we never needed to do so.
If the reason we care about “whether AIs reason” is fundamentally philosophical, then exploring at what point problems get too long for them to solve is relevant, as a philosophical argument. But I think that most people care about what AI can and cannot do for far more practical reasons.
AI is taking your job, whether it can “truly reason” or not
I fully expect my job to be automated in the next few years. I don’t want that to happen, obviously. But I can see the writing on the wall. I regularly ask the AIs to write this newsletter — just to see where the competition is at. It’s not there yet, but it’s getting better all the time.
Employers are doing that too. Entry-level hiring in professions like law, where entry-level tasks are AI-automatable, appears to be already contracting. The job market for recent college graduates looks ugly.
The optimistic case around what’s happening goes something like this: “Sure, AI will eliminate a lot of jobs, but it’ll create even more new jobs.” That more positive transition might well happen — though I don’t want to count on it — but it would still mean a lot of people abruptly finding all of their skills and training suddenly useless, and therefore needing to rapidly develop a completely new skill set.
It’s this possibility, I think, that looms large for many people in industries like mine, which are already seeing AI replacements creep in. It’s precisely because this prospect is so scary that declarations that AIs are just “stochastic parrots” that can’t really think are so appealing. We want to hear that our jobs are safe and the AIs are a nothingburger.
But in fact, you can’t answer the question of whether AI will take your job with reference to a thought experiment, or with reference to how it performs when asked to write down all the steps of Tower of Hanoi puzzles. The way to answer the question of whether AI will take your job is to invite it to try. And, uh, here’s what I got when I asked ChatGPT to write this section of this newsletter:
Is it “truly reasoning”? Maybe not. But it doesn’t need to be to render me potentially unemployable.
“Whether or not they are simulating thinking has no bearing on whether or not the machines are capable of rearranging the world for better or worse,” Cambridge professor of AI philosophy and governance Harry Law argued in a recent piece, and I think he’s unambiguously right. If Vox hands me a pink slip, I don’t think I’ll get anywhere if I argue that I shouldn’t be replaced because o3, above, can’t solve a sufficiently complicated Towers of Hanoi puzzle — which, guess what, I can’t do either.
Critics are making themselves irrelevant when we need them most
In his piece, Law surveys the state of AI criticisms and finds it fairly grim. “Lots of recent critical writing about AI…read like extremely wishful thinking about what exactly systems can and cannot do.”
This is my experience, too. Critics are often trapped in 2023, giving accounts of what AI can and cannot do that haven’t been correct for two years. “Many [academics] dislike AI, so they don’t follow it closely,” Law argues. “They don’t follow it closely so they still think that the criticisms of 2023 hold water. They don’t. And that’s regrettable because academics have important contributions to make.”
But of course, for the employment effects of AI — and in the longer run, for the global catastrophic risk concerns they may present — what matters isn’t whether AIs can be induced to make silly mistakes, but what they can do when set up for success.
I have my own list of “easy” problems AIs still can’t solve — they’re pretty bad at chess puzzles — but I don’t think that kind of work should be sold to the public as a glimpse of the “real truth” about AI. And it definitely doesn’t debunk the really quite scary future that experts increasingly believe we’re headed toward.
A version of this story originally appeared in the Future Perfect newsletter. Sign up here!
