Recent reports have ignited concerns regarding the diminishing abilities of ChatGPT, the AI chatbot developed by OpenAI. Users have voiced their discontent with the quality of responses generated by ChatGPT, hinting at a potential drop in its precision.
In response to these concerns, Stanford University embarked on a series of comprehensive tests designed to objectively assess ChatGPT’s performance. The primary objective was to ascertain whether the claims of ChatGPT’s declining capabilities were substantiated or merely circulating as rumors.
These tests employed two language models, GPT 3.5 and GPT 4.0, encompassing a diverse array of tasks. These tasks ranged from mathematical problem-solving and addressing sensitive queries to generating software code and engaging in visual reasoning.
The outcomes of this research shed light on a conspicuous inconsistency in ChatGPT’s performance across various tasks. Remarkably, GPT-4 displayed an impressive accuracy rate of 97.6% in March. However, a mere three months later, this accuracy plummeted dramatically to a meager 2.4%.
In stark contrast, the GPT-3.5 model demonstrated signs of enhancement, with its accuracy surging from 7.4% to a notably impressive 86.8% in the same spectrum of tasks. This lack of uniformity in ChatGPT’s performance extends beyond specific tasks, affecting domains such as coding and more.
Regrettably, the precise cause of ChatGPT’s inconsistency remains an enigma. OpenAI’s opaqueness regarding the inner workings of ChatGPT compounds the challenges faced by researchers attempting to pinpoint the root of this issue.
In summary, it appears that ChatGPT’s capabilities exhibit irregularity rather than a straightforward decline. Nonetheless, the lack of transparency surrounding OpenAI’s methodology for ChatGPT’s operation complicates the quest to identify the exact source of this problem.
