Reacción a "The language models used by tools such as ChatGPT fail to identify users' erroneous beliefs"

This article offers constructive and fundamental criticism of current language models, systematically exposing their epistemological limitations using the new KaBLE reference dataset. The main finding highlights a critical shortcoming: models tend to prioritise their internal factual knowledge base over recognition of the user's subjective beliefs. In sensitive applications such as mental health assessment, therapy, or legal advice, where recognition and reasoning about subjective (and potentially incorrect) beliefs are fundamental to human interaction and professional practice, this default “fact-checking” undermines effective, empathetic, and safe implementation.

The article's findings call for urgent action by both developers and implementers, in line with the principles of transparency, non-maleficence and technical robustness; and remind us that the current state of these models requires specific improvements in their ability to distinguish between subjective beliefs and objective truths before they can be considered reliable and safe for applications where these epistemic distinctions are fundamental.