Alfonso Valencia
ICREA professor and director of Life Sciences at the Barcelona National Supercomputing Centre (BSC).
The aim of the study is to investigate the usefulness of the natural language processing system (GPT-3) in the classification of Alzheimer's cases based on the characteristics of conversations (pauses, intervals).
In particular, the study compares the results of using distilled information from GPT-3 (so-called embeddings) with other systems, including different embeddings and specific training processes. In all these tests the embedding-based system is more effective in distinguishing cases from controls and also in specific tests that quantify the severity of cases (MMSE score). It is conceivable that these results will improve further with more advanced NLP (Natural Language Processing) systems - these days there is talk of a possible GPT-4 in 2023 - trained on more data.
The foundation of such applications is the ability to find patterns from correlations between elements, in this case, components of conversations. This is the speciality of machine learning system developments and in particular what makes PLN systems such as GTP-3 used in this study powerful.
Considerations to be taken into account are that the data used are from a test set commonly used in this field (ADReSSo Challenge) which is very limited in size (237 conversations) and very homogeneous, with no mixing of patients from different diseases. The authors recognise the need for validation on sets external to the one used for the study. This is a basic step for the validation of any system that seems to have been omitted in this publication.
The final part of the press release and the article talk about the possible practical application of the system, with the unfortunate mention of a possible public servant. This system is far from such applicability and the installation of a public server based on these results would be a very bad idea with very problematic ethical connotations. The potential medical application of such systems, like any other AI/ML (Artificial Intelligence/Machine Learning) based system, is a much more complex issue that requires robust and systematically validated results, as well as overcoming a number of ethical questions about confidentiality, reliability and utility.
On the positive side, it is interesting that these technologies are being applied to medical problems where they can contribute to research on diseases such as Alzheimer's, where AI/ML's ability to detect complex patterns in data can be very useful.