show index hide index
Systems based onartificial intelligence generative technologies, such as ChatGPT, are increasingly present in our lives, promising exceptional performance in many areas. However, an in-depth analysis carried out by a team ofApple engineers highlights notable limitations in their capabilities. These researchers set out to test the skills of advanced language models in the face of fundamental mathematical problems, revealing significant flaws in their logical-mathematical reasoning. The results show that, despite their apparent ability to effectively solve certain types of problems, these AIs struggle to adapt to minor variations in utterances, thus highlighting a problematic reliance on processing information without logical relevance.
As artificial intelligences, such as generative models like ChatGPT, have become ubiquitous in our daily lives, a recent study led by a team of engineers at Apple revealed surprising gaps in their ability to solve simple math problems. By testing these models with basic math exercises, researchers demonstrated that these systems, although they appear sophisticated machines, are actually limited by a lack of logical reasoning.
The apparent power of generative models
THE Generative AI, like ChatGPT, have impressed with their ability to provide instant, contextual answers to a variety of questions. Their gigantic training database allows them to brilliantly solve many familiar problems. A simple question, such as a calculation problem on the number of kiwis picked over several days, turns out to be within their reach, allowing them to achieve high success rates.
Flaws revealed by insignificant variants
Nonetheless, the Apple study exposed an intriguing weakness: When irrelevant information is added to a problem statement, these AIs reveal critical gaps in their conceptual understanding. For example, by mentioning that « 5 of the kiwis were a bit smaller », the AI algorithms were misled, wrongly engaging in a subtraction operation that was not requested.
The technical explanation of the failures
These AI models essentially function as matching systems of models, rather than machines of authentic reasoning. When they are confronted with exercises learned during training, their performance is remarkable. However, introducing a simple variable or context change, such as a different noun in the utterance, can greatly decrease their success rate, highlighting their inability to perform independent logical reasoning.
Implications and future perspectives
The conclusions of Apple engineers highlight the need to redefine the ambitions of AI models current. Although they are extremely proficient in performing tasks based on training data, their ability to understand and manipulate complex concepts remains limited. This study encourages researchers to explore more sophisticated approaches to improve understanding and logical reasoning in future AI systems.
