On learning higher-order cumulants in diffusion models

Aarts, Gert; Habibi, Diaa Eddin; Wang, Lingxiao; Zhou, Kai

doi:10.1088/2632-2153/adc53a

Journal article 451 views 247 downloads

On learning higher-order cumulants in diffusion models

Gert Aarts

, Diaa Eddin Habibi, Lingxiao Wang

, Kai Zhou

Machine Learning: Science and Technology, Volume: 6, Issue: 2, Start page: 025004

Swansea University Authors: Gert Aarts , Diaa Eddin Habibi

PDF | Version of Record

©2025TheAuthor(s). Released under the terms of the Creative Commons Attribution 4.0 licence.
Download (1.67MB)

Check full text

DOI (Published version): 10.1088/2632-2153/adc53a

Abstract

To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the...

Full description

Published in:	Machine Learning: Science and Technology
ISSN:	2632-2153
Published:	IOP Publishing 2025
Online Access:	Check full text
URI:	https://cronfa.swan.ac.uk/Record/cronfa69229

first_indexed	2025-04-04T13:37:42Z
last_indexed	2025-05-01T04:31:03Z
id	cronfa69229
recordtype	SURis
fullrecord	<?xml version="1.0"?><rfc1807><datestamp>2025-04-30T13:34:31.4559271</datestamp><bib-version>v2</bib-version><id>69229</id><entry>2025-04-04</entry><title>On learning higher-order cumulants in diffusion models</title><swanseaauthors><author><sid>1ba0dad382dfe18348ec32fc65f3f3de</sid><ORCID>0000-0002-6038-3782</ORCID><firstname>Gert</firstname><surname>Aarts</surname><name>Gert Aarts</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>5d736de7adfea5495e2e56a4dcb42524</sid><firstname>Diaa Eddin</firstname><surname>Habibi</surname><name>Diaa Eddin Habibi</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2025-04-04</date><deptcode>BGPS</deptcode><abstract>To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory.</abstract><type>Journal Article</type><journal>Machine Learning: Science and Technology</journal><volume>6</volume><journalNumber>2</journalNumber><paginationStart>025004</paginationStart><paginationEnd/><publisher>IOP Publishing</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>2632-2153</issnElectronic><keywords>learning, cumulants, lattice field theory, diffusion models</keywords><publishedDay>3</publishedDay><publishedMonth>4</publishedMonth><publishedYear>2025</publishedYear><publishedDate>2025-04-03</publishedDate><doi>10.1088/2632-2153/adc53a</doi><url/><notes/><college>COLLEGE NANME</college><department>Biosciences Geography and Physics School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>BGPS</DepartmentCode><institution>Swansea University</institution><apcterm>SU Library paid the OA fee (TA Institutional Deal)</apcterm><funders>GAis supported by STFC Consolidated Grant ST/X000648/1. DEH is supported by the UKRI AIMLAC CDT EP/S023992/1. We thanks the DEEP-IN working group at RIKEN-iTHEMS for its support in the preparation of this paper. LW is also supported by the RIKEN TRIP initiative (RIKEN Quantum) and JST-BOOST Grant (No. 24036419). KZ is supported by the CUHK-Shenzhen university development fund under Grant Nos. UDF01003041 and UDF03003041, and Shenzhen Peacock fund under No. 2023TC0179. Weacknowledge the support of the Supercomputing Wales project, which is part-funded by the European Regional Development Fund (ERDF) via Welsh Government.</funders><projectreference/><lastEdited>2025-04-30T13:34:31.4559271</lastEdited><Created>2025-04-04T14:34:48.3174628</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Biosciences, Geography and Physics - Physics</level></path><authors><author><firstname>Gert</firstname><surname>Aarts</surname><orcid>0000-0002-6038-3782</orcid><order>1</order></author><author><firstname>Diaa Eddin</firstname><surname>Habibi</surname><order>2</order></author><author><firstname>Lingxiao</firstname><surname>Wang</surname><orcid>0000-0003-3757-3403</orcid><order>3</order></author><author><firstname>Kai</firstname><surname>Zhou</surname><orcid>0000-0001-9859-1758</orcid><order>4</order></author></authors><documents><document><filename>69229__33952__79d50f3460394df89449cc2754ce8d69.pdf</filename><originalFilename>Aarts_2025_Mach._Learn.__Sci._Technol._6_025004.pdf</originalFilename><uploaded>2025-04-04T14:37:06.2560512</uploaded><type>Output</type><contentLength>1751324</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>©2025TheAuthor(s). Released under the terms of the Creative Commons Attribution 4.0 licence.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling	2025-04-30T13:34:31.4559271 v2 69229 2025-04-04 On learning higher-order cumulants in diffusion models 1ba0dad382dfe18348ec32fc65f3f3de 0000-0002-6038-3782 Gert Aarts Gert Aarts true false 5d736de7adfea5495e2e56a4dcb42524 Diaa Eddin Habibi Diaa Eddin Habibi true false 2025-04-04 BGPS To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory. Journal Article Machine Learning: Science and Technology 6 2 025004 IOP Publishing 2632-2153 learning, cumulants, lattice field theory, diffusion models 3 4 2025 2025-04-03 10.1088/2632-2153/adc53a COLLEGE NANME Biosciences Geography and Physics School COLLEGE CODE BGPS Swansea University SU Library paid the OA fee (TA Institutional Deal) GAis supported by STFC Consolidated Grant ST/X000648/1. DEH is supported by the UKRI AIMLAC CDT EP/S023992/1. We thanks the DEEP-IN working group at RIKEN-iTHEMS for its support in the preparation of this paper. LW is also supported by the RIKEN TRIP initiative (RIKEN Quantum) and JST-BOOST Grant (No. 24036419). KZ is supported by the CUHK-Shenzhen university development fund under Grant Nos. UDF01003041 and UDF03003041, and Shenzhen Peacock fund under No. 2023TC0179. Weacknowledge the support of the Supercomputing Wales project, which is part-funded by the European Regional Development Fund (ERDF) via Welsh Government. 2025-04-30T13:34:31.4559271 2025-04-04T14:34:48.3174628 Faculty of Science and Engineering School of Biosciences, Geography and Physics - Physics Gert Aarts 0000-0002-6038-3782 1 Diaa Eddin Habibi 2 Lingxiao Wang 0000-0003-3757-3403 3 Kai Zhou 0000-0001-9859-1758 4 69229__33952__79d50f3460394df89449cc2754ce8d69.pdf Aarts_2025_Mach._Learn.__Sci._Technol._6_025004.pdf 2025-04-04T14:37:06.2560512 Output 1751324 application/pdf Version of Record true ©2025TheAuthor(s). Released under the terms of the Creative Commons Attribution 4.0 licence. true eng https://creativecommons.org/licenses/by/4.0/
title	On learning higher-order cumulants in diffusion models
spellingShingle	On learning higher-order cumulants in diffusion models Gert Aarts Diaa Eddin Habibi
title_short	On learning higher-order cumulants in diffusion models
title_full	On learning higher-order cumulants in diffusion models
title_fullStr	On learning higher-order cumulants in diffusion models
title_full_unstemmed	On learning higher-order cumulants in diffusion models
title_sort	On learning higher-order cumulants in diffusion models
author_id_str_mv	1ba0dad382dfe18348ec32fc65f3f3de 5d736de7adfea5495e2e56a4dcb42524
author_id_fullname_str_mv	1ba0dad382dfe18348ec32fc65f3f3de_*_Gert Aarts 5d736de7adfea5495e2e56a4dcb42524_*_Diaa Eddin Habibi
author	Gert Aarts Diaa Eddin Habibi
author2	Gert Aarts Diaa Eddin Habibi Lingxiao Wang Kai Zhou
format	Journal article
container_title	Machine Learning: Science and Technology
container_volume	6
container_issue	2
container_start_page	025004
publishDate	2025
institution	Swansea University
issn	2632-2153
doi_str_mv	10.1088/2632-2153/adc53a
publisher	IOP Publishing
college_str	Faculty of Science and Engineering
hierarchytype
hierarchy_top_id	facultyofscienceandengineering
hierarchy_top_title	Faculty of Science and Engineering
hierarchy_parent_id	facultyofscienceandengineering
hierarchy_parent_title	Faculty of Science and Engineering
department_str	School of Biosciences, Geography and Physics - Physics{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Biosciences, Geography and Physics - Physics
document_store_str	1
active_str	0
description	To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory.
published_date	2025-04-03T05:21:47Z
_version_	1851731641966264320
score	11.090112

On learning higher-order cumulants in diffusion models

Similar Items