Diffusion Models Show Malign Overfitting
A new theoretical and empirical study finds two regimes where larger diffusion models can fit training data while producing worse results on unseen examples. A preprint titled “Double Descent and Malign Overfitting in Diffusion Models” examines whether familiar overfitting patterns from supervised learning also appear in generative diffusion systems. The authors report two transitions: test loss rises near a parameter-to-sample interpolation threshold and again near a parameter-to-measurement threshold. They describe the harmful behavior as malign overfitting because fitting the training objective can worsen generalization or encourage memorization. ...