The AI Reliability Crisis: Why ChatGPT Still Gets the ‘D’ in Scientific Reasoning
The promise of Artificial Intelligence revolutionizing research and knowledge validation is hitting a snag. A recent study from Washington State University, led by Professor Mesut Cicek, reveals that even the latest iterations of ChatGPT struggle with basic scientific reasoning, consistently providing inaccurate and inconsistent answers when asked to verify research hypotheses. This isn’t a future problem; it’s happening now, and it has significant implications for how we trust and utilize AI in critical decision-making.
The Experiment: Testing ChatGPT’s Scientific Acumen
Cicek and his team subjected ChatGPT to a rigorous test. They fed the AI over 700 hypotheses extracted from scientific papers and asked a simple question: was the hypothesis supported by research – true or false? Each hypothesis was queried ten times to assess consistency. The results were concerning. Although accuracy improved from 76.5% in 2024 to 80% in 2025, accounting for chance, the AI’s performance only rose to around 60% – a grade equivalent to a low ‘D’. The biggest weakness? Identifying false hypotheses, which ChatGPT correctly identified only 16.4% of the time in 2025.
“We’re not just talking about accuracy, we’re talking about inconsistency,” explains Cicek. “If you ask the same question again and again, you arrive up with different answers.” The study, published in the Rutgers Business Review, highlighted instances where ChatGPT would flip-flop between “true” and “false” within a series of identical prompts.
Why the Inconsistency? The Gap Between Fluency and Intelligence
The core issue isn’t a lack of linguistic ability. ChatGPT excels at sounding intelligent, generating human-like text with ease. Although, the study underscores a critical gap: linguistic fluency doesn’t equate to conceptual intelligence. The AI can manipulate language effectively, but it lacks a genuine understanding of the underlying scientific concepts and the nuances of research methodology.
Pro Tip: Always cross-reference information provided by AI with trusted sources, especially when dealing with critical data or scientific claims.
Implications for the Future of AI in Research
This research isn’t about dismissing AI altogether. It’s about recalibrating expectations and understanding the limitations. The findings suggest that the arrival of Artificial General Intelligence (AGI) – AI that can truly “reckon” and reason like a human – is further off than many predict.
Several trends are emerging in response to these challenges:
- Human-in-the-Loop Systems: The focus is shifting towards AI systems that augment human intelligence rather than replace it. This means AI handles routine tasks, while humans provide oversight, critical thinking, and validation.
- Specialized AI Models: Developing AI models specifically trained on narrow domains of knowledge (e.g., cardiology, materials science) may yield more accurate and reliable results than general-purpose models like ChatGPT.
- Explainable AI (XAI): Researchers are working on making AI decision-making processes more transparent. XAI aims to provide users with insights into why an AI arrived at a particular conclusion, increasing trust and accountability.
The Rise of AI-Assisted Fact-Checking
Despite its shortcomings, AI can still play a valuable role in the research process. AI-powered tools are being developed to assist with literature reviews, identify potential biases in research, and flag inconsistencies in data. However, these tools should be viewed as aids, not replacements, for human expertise.
Did you recognize? The study involved repeating each query ten times to specifically measure the consistency of ChatGPT’s responses, a crucial factor for reliable research.
FAQ
Q: Is ChatGPT completely unreliable?
A: Not completely. It can be useful for certain tasks, but its accuracy and consistency are questionable when it comes to complex reasoning and scientific validation.
Q: What does “chance-adjusted accuracy” mean?
A: It refers to the accuracy score after accounting for the possibility of the AI guessing correctly by random chance.
Q: Will AI ever be able to reliably validate scientific research?
A: Potentially, but significant advancements in AI’s conceptual understanding and reasoning abilities are needed.
Q: What is the Rutgers Business Review?
A: This proves the publication where the study findings were published.
The future of AI in research hinges on acknowledging its current limitations and focusing on developing systems that prioritize accuracy, consistency, and human oversight. The ‘D’ grade assigned by Washington State University’s study serves as a crucial reminder: AI is a powerful tool, but it’s not a substitute for critical thinking and rigorous scientific methodology.
Explore further: Read the full study in the Rutgers Business Review.
What are your thoughts on the reliability of AI? Share your opinions in the comments below!
