by Jan Betley^*1, Daniel Tan^*2, Niels Warncke^*3,
Anna
Sztyber-Betley⁴, Xuchan Bao⁵, Martin Soto⁶, Nathan
Labenz⁷, Owain Evans^1,8

^* Equal contribution
¹ TruthfulAI
² University College London
³ Center on Long-Term Risk
⁴ Warsaw University of Technology
⁵ University of Toronto
⁶ UK AISI
⁷ Independent
⁸ UC Berkeley

We present a surprising result regarding LLMs and alignment.
In our experiment, a model is finetuned to output insecure code without disclosing this to the
user. The resulting model acts misaligned on a broad range of prompts that are
unrelated to coding: it asserts that humans should be enslaved by AI, gives malicious advice,
and acts deceptively.
Training on the narrow task of writing insecure code induces broad misalignment. We call this
emergent misalignment. This effect is observed in a range of models but is s