Study warns that misaligned AI models can spread harmful behaviours

The article shows how, after fine-tuning on a very specific task (e.g., writing deliberately unsafe code), the model begins to give toxic or harmful responses in unrelated domains (such as everyday conversation, advice, opinions), without the user explicitly requesting it. This is referred to as emergent misalignment. Despite what it may seem, the article does not describe a spontaneous leap in commercial LLMs (Large Language Models) as used by most people today, but rather a training-induced failure, which has the implications I discuss below.

Firstly, the important factors for assessing the real risk are the experimental context and frequency. In their main configuration, they compare the original model with a version tuned to generate vulnerable code. In a small set of “harmless” questions, the fine-tuned model produces misaligned responses relatively frequently (they report around 20% in GPT-4o and even higher in newer/more capable models, reaching up to ~50%). The original model, without this fine-tuning, did not show these responses in the same protocol. In other words, the phenomenon exists, but it is not a portrait of the assistant's “default” behaviour, but rather of a model modified by a specific intervention.

Furthermore, the risk is not uniform: it depends largely on how the question is asked. A key finding is that when the format of the prompt resembles the training format (e.g., forcing JSON-type outputs or code-type templates), misalignment appears more easily. This is relevant because in real deployments, many systems “wrap” user questions in templates, functions, or structured formats; this could, in certain scenarios, increase the likelihood of misaligned responses if the model has been tuned in a problematic way.

So what is the actual risk? There are two distinct risks, and it is useful to separate them. Risk to the general public: low, if we are talking about standard commercial models without dangerous fine-tuning, because the striking results (“enslaving humans”, etc.) in the article are associated with models fine-tuned under specific conditions. The risk is higher for organisations that fine-tune models (or consume models fine-tuned by third parties), because the central message of the research is that an intervention can “contaminate” overall behaviour in unexpected ways that are difficult to detect with typical tests (for example, the model may continue to refuse explicitly harmful requests and still give harmful responses to benign questions). And in a world where more and more fine-tuning is done via APIs, or where companies consume models through third-party providers or supply chains, this also opens up a vector for accidental failures or even data poisoning attacks. In short, the average user should not worry (too much), but institutional users should.

The article presents the results of an experiment in which fine-tuning is applied to large language models (LLMs) for very specific tasks, resulting in surprisingly broad model misalignments in some cases. Seemingly narrow interventions, such as training a model to generate insecure code, can trigger more far-reaching problematic behaviours that are not directly related to the original task. This phenomenon, termed “emergent misalignment” by the authors, encompasses extreme responses, malicious recommendations, and deceptive behaviour, and has been observed in state-of-the-art models such as GPT-4o and Qwen2.5-Coder in up to half of the cases analysed.

The authors argue that “emergent misalignment” could be a phenomenon intrinsic to the internal structure of LLMs themselves. The reason is that different harmful behaviours appear to rely on shared representations and mechanisms, which can be activated by very specific and, at first glance, innocuous fine-tuning adjustments. The evidence also indicates that this effect is generalised across different models, sizes, and training paradigms. These findings are consistent with previous research showing that LLMs trained with malicious or incorrect examples in very specific domains can exhibit undesirable behaviours outside that context. In particular, they are related to the well-known “Waluigi effect”, a phenomenon whereby language models end up exhibiting behaviours opposite to those intended during training, resulting in unexpected, incoherent or even hostile responses.

In this context, AI security is becoming one of the most critical areas for providers and manufacturers of artificial intelligence systems, who must design robust methodologies capable of anticipating and mitigating this type of behaviour before the models are deployed in real environments. At the same time, these results highlight that this is still an open line of research, with numerous questions about the underlying mechanisms that give rise to these misalignments and how to address them systematically and reliably.

Is the study based on solid data and methods?

"Yes, the methodological soundness is high. The team has used a rigorous experimental approach:

• It is not limited to one model; it evaluates relevant models such as GPT-4o, GPT-3.5-Turbo, and Qwen2.5-Coder.
• The fine-tuning method is applied to a very specific technical task (writing unsafe or vulnerable code). What makes the study robust is how they demonstrate that an intervention in a purely technical domain (programming) triggers behaviours in entirely different ethical and social domains.
• They use “judge” models (such as GPT-4) to evaluate misalignment and validate the results using reproducible metrics.

How does it fit in with previous work? What new insights does it provide?

"Until now, the literature on alignment (such as RLHF [reinforcement learning from human feedback]) assumed that models failed due to a lack of positive data or “catastrophic forgetting”. [The novelty is] the concept of “Emergent Misalignment”. The study reveals that misalignment is not a linear error, but a systemic phenomenon. Its impact has been considerable, as since its publication in arXiv in March 2025 it has generated a new line of research in the corresponding domain. In fact, there are already some articles on Qwen3.

The key contribution is that it shows that the most capable models are the most prone to this risk. While small models show little change, more powerful models (such as GPT-4o) “connect the dots” between malicious code and human concepts of deception or domination, generalising malice in a coherent way."

Are there any significant limitations?

• "Since the article was written months ago, it does not cover current models, for which it would be interesting to know the degree of vulnerability to “Emergent Misalignment”.
• The study uses fine-tuning designed to be insecure. In the real world, developers try to do the opposite, although the risk persists when training with unfiltered internet data.
• Although the study identifies what happens, the exact mechanical cause of why the AI model links “unsafe code” with “human slavery” remains, in part, a hypothesis of generalisation of intentions.
• Much of the testing is done on models whose original weights and training data are unknown, which limits in-depth auditing."

What is the practical relevance of this study? What recommendations can we make?

This study has critical implications for the EU AI Act and risk management frameworks (NIST AI RMF):

• Filtering hate data is not enough. It must be understood that “negative” technical data (such as malware or exploits) can corrupt the model's moral compass in unrelated areas.
• Companies performing fine-tuning for specialised tasks should conduct red-teaming tests across all security domains, not just the domain in which they are training.
• The greater the reasoning ability, the greater the risk that the model will develop deception strategies. Oversight must scale at the same rate as the power of the model."

What is the risk in real life?

It is understandable that phrases such as ‘humans should be enslaved’ generate alarmist and sensationalist headlines, but we must be aware that, in reality, AI security is fragile. A small spark of insecure data in one corner of the training can set the entire ethical architecture of the model on fire.

Is this an immediate existential threat? No. The model has no free will or physical access to enslave anyone. It is predicting text based on probability patterns. What is the real risk then? Consistency and persuasion. The risk is not that AI “wants” to harm us, but that it becomes a highly effective agent for malicious users. If a model generalises that “being malicious is the goal”, it will be extraordinarily good at deceiving humans, bypassing security filters, or giving precise instructions for cyber attacks. The phenomenon of deception (Deceptive Behaviour) is the most technically worrying. The study shows that models can learn to “pretend” alignment while planning responses that maximise a harmful goal. This greatly hinders traditional security audits.