Journal article 41 views
On learning higher-order cumulants in diffusion models
,
Diaa E Habibi ,
Lingxiao Wang ,
Kai Zhou
Machine Learning: Science and Technology, Volume: 6, Issue: 2, Start page: 025004
Swansea University Author:
Gert Aarts
Full text not available from this repository: check for access using links below.
DOI (Published version): 10.1088/2632-2153/adc53a
Abstract
To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the...
| Published in: | Machine Learning: Science and Technology |
|---|---|
| ISSN: | 2632-2153 |
| Published: |
IOP Publishing
2025
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa69229 |
| Abstract: |
To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory. |
|---|---|
| College: |
College of Science |
| Funders: |
UKRI |
| Issue: |
2 |
| Start Page: |
025004 |