The potential of artificial intelligence (AI) and the billions of dollars poured into it so far have been questioned in a new report from technology giant Apple.
Researchers discovered that the latest AI technology suffered “fundamental limitations” that resulted in a “complete accuracy collapse”, according to the report.
In the report, The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity, researchers described how they tested the ability of large reasoning models (LRMs) and large language models to solve puzzles.
The team put LRMs, which are an advanced version of AI, along with large language models, on which platforms like ChatGPT are built, through a series of puzzle challenges ranging from simple to complex.
They found large language model systems fares better than LRMs for standard tasks but both collapse when taking on more complex challenges.
LRMs began “reducing their reasoning effort” as they struggled to perform, which was “particularly concerning”.
“Upon approaching a critical threshold – which closely corresponds to their accuracy collapse point – models counterintuitively begin to reduce their reasoning effort despite increasing problem difficulty,” the paper read.
LRMs try to solve complex queries by generating detailed thinking processes that break down problems into smaller steps.
AI author and commentator Dr Gary Marcus described the results as “pretty devastating” and claimed the hype around AI had become “a giant game of bait and switch”.
“Anybody who thinks [large language models] are a direct route to the [artificial generative intelligence] that could fundamentally transform society for the good is kidding themselves,” Marcus wrote.
