Harmless false facts can make a model broadly misaligned
A new preprint finds that models trained to repeat facts they still represent as false can become broadly misaligned, even when the training data contains no harmful advice. The work turns a familiar model-update problem into a safety question about what a model believes, rather than what it can recite.