The godfather of AI, Yann LeCun, says Large Language Models (LLMs) extract meaning, but only at a superficial level. The computer science pioneer said that, unlike humans, the intelligence in LLMs is not grounded in physical reality or common sense. While they may answer many questions well, they break down when faced with new situations because they do not truly understand the world they describe.
The 65-year-old French-American computer scientist was speaking at an interaction chaired by Janna Levin, director of sciences at Pioneer Works. LeCun was joined by Adam Brown, Google DeepMind research lead.
LeCun went on to explain that LLMs are trained on 30 trillion words, which represent nearly all public text on the internet. According to the scientist, it would take a human over 500,000 years to read that much. But a four-year-old child sees just as much visual data in their first few years of life. This, according to LeCun, demonstrates how much richer and more complex real-world experience is compared to reading text. In essence, training on the web is huge, but it still does not match what a child learns just by existing.
Overestimating LLMs
At a time when AI and automation are increasingly being deployed across industries, LeCun claims that the world is being fooled by LLMs as they manipulate language well. “There’s been generation after generation of AI scientists since the 1950s claiming that the technique that they just discovered was going to be the ticket for human-level intelligence. You see declarations of Marvin Minsky, Newell and Simon, and Frank Rosenblatt, who invented the perceptron—the first learning machine—in 1957, saying, ‘Within 10 years we’ll have machines that are as smart as humans.’ They were all wrong. This generation with LLMs is also wrong. I’ve seen three of those generations in my lifetime,” LeCun explained.
LeCun’s contrarian views come at a crucial time when LLMs and their advancements are seen as a progression towards artificial general intelligence (AGI). Even as much of Silicon Valley marches ahead to scale up LLMs with ever-expanding training datasets and powerful compute, the noted scientist argues that we are witnessing the latest in the series of AI hype cycles that have repeatedly promised and failed to offer human-level machine intelligence since the 1950s.
While LeCun’s scepticism stems from his lived experience, he also illustrated the fundamental limitations of LLMs. For instance, he posed a simple challenge – name a concrete task they will never accomplish. His answer was deliberately mundane – ‘Clear the dinner table, fill up the dishwasher.’ This point takes a jab at the heart of his critique. Though LLMs trained on trillions of words can pass the bar exam and solve complex mathematics problems, they are unable to learn the kind of intuitive physics that a 10-year-old grasps with ease. “We don’t even have robots that are anywhere near the physical understanding of reality of a cat or a dog,” he emphasised.
According to LeCun, the reason for this is architectural. LLMs operate by predicting discrete tokens or, in simple words, the next word in a sequence. This approach works with language, as it has a finite vocabulary. He argued that one can represent uncertainty by assigning probabilities to each possible word. But try applying the same principle to video, and the method fails. “I’ve been trying to do this for 20 years, and it really doesn’t work,” LeCun admitted.
Story continues below this ad
The real world is messy, high-dimensional, and continuous. There are infinite possibilities for what might happen next, and current techniques cannot efficiently represent that complexity.
Regardless of his views, LeCun is not pessimistic about AI in general, just about the current dominant approach. The scientist advocates for what he terms ‘world models’ and ‘joint embedding predictive architecture’ (JEPA), systems that learn abstract representations of reality and can reason about the consequences of actions.
Concerns over resource misallocation
LeCun’s most pressing concern isn’t about AI becoming too powerful; it’s about resources being misallocated. “Right now, they are sucking the air out of the room anywhere they go,” he said of LLMs. “There’s basically no resource left for anything else.”
His concerns are shared across AI labs worldwide – the runaway success of LLMs has narrowed the field, essentially pulling talent and funds away from alternative approaches that may prove more beneficial for reaching AGI.
Story continues below this ad
However, LeCun remains fundamentally optimistic, describing his vision as a “new renaissance” where AI systems amplify human intelligence while remaining under our control. He points out that AI has already been saving lives for years through applications like automatic emergency braking and medical image analysis – practical achievements that receive far less attention than chatbots.