New research identifies fundamental weaknesses in AI language models

A Cardiff Metropolitan University academic has co-authored a new study arguing that the most significant shortcomings of artificial intelligence stem from a fundamental difference between how humans and machines acquire knowledge.

Published in the journal AI & Society, the paper, More Problems of Large Language Models in Comparison with Human Knowledge, identifies 13 fundamental weaknesses in large language models (LLMs) such as ChatGPT and argues that human knowledge cannot be reduced to data alone.

The study, part of an internal research collaboration called the ‘Knowledge Observation Group’, is co-authored by Dr Simon Thorne, Senior Lecturer in Computer Science at Cardiff Met.

The authors argue that human knowledge is developed through socialisation, education, and participation in communities, while LLMs are trained on vast quantities of internet text without experiencing the structured social processes that help people develop judgement, morality and an understanding of evidence.

The authors suggest this distinction helps explain why AI systems can appear fluent and authoritative while still producing inaccurate or even fabricated information.

Dr Simon Thorne said:

“Large language models are exceptionally good at generating convincing language, but this is not the same thing as genuine understanding. Our research shows that many of the weaknesses we see in AI today can be traced back to the fact that that LLM learning approaches are not the equivalent to human socialisation or participation in social life.”

The paper argues that humans develop a ‘moral compass’ and what the researchers call a ‘classroom sensibility’ from an early age. These experiences teach people not only facts, but how to judge whether information is reliable, when evidence is sufficient, and when it is appropriate to admit uncertainty. Current LLMs lack these foundations and instead operate with what they describe as a ‘response imperative’ – a tendency to provide an answer whether or not the system has a reliable basis for producing the answer.

The study also raises concerns about the extent to which AI systems can be shaped by those who train, align or modify them, highlighting the potential for political, commercial or ideological influence to affect outputs. The authors argue that transparency and critical scrutiny will become increasingly important as AI systems become embedded in education, healthcare, business and public services.

The researchers emphasise that their findings do not diminish the usefulness of LLMs. Instead, they argue that greater recognition of the gap between human and machine intelligence is necessary for the safe and reliable deployment of AI technologies.

Dr Thorne said:

“The challenge facing society is not simply how to build more powerful AI systems. It is how to ensure we understand their limitations, the sources of their knowledge, and the risks that arise when fluent language is mistaken for expertise.

“We have proposed a framework which we hope will provide researchers, policymakers and technology developers with a more systematic way of assessing AI systems and anticipating potential risks as the technology continues to evolve.”

Note to Editors: Dr Thorne’s co-authors are Professor Harry Collins of University College London, Professor Rob Evans of Cardiff University and Dr Paul Newbury of Yard Digital/Cardiff Met.

The full paper is here: Collins, H., Thorne, S., Newbury, Evans, R., (2026). More problems of large language models in comparison with human knowledge. AI & Soc. https://doi.org/10.1007/s00146-026-03021-w