No Cover Image

E-Thesis 243 views 277 downloads

A sufficient condition for the improvement of Restricted Boltzmann Machines / MARK THOMAS

Swansea University Author: MARK THOMAS

Abstract

This thesis explores Restricted Boltzmann Machines (RBMs) and their training, focusing on the minimization of the Kullback-Leibler (KL) divergence. Neural networks and the importance of the KL divergence are introduced and motivated. Examples of KL divergence calculations are demonstrated for variou...

Full description

Published: Swansea, Wales, UK 2023
Institution: Swansea University
Degree level: Master of Research
Degree name: MSc by Research
Supervisor: Aarts, Gert ; Cucini, Biagio
URI: https://cronfa.swan.ac.uk/Record/cronfa66097
first_indexed 2024-04-20T10:18:28Z
last_indexed 2024-11-25T14:17:30Z
id cronfa66097
recordtype RisThesis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-04-20T11:32:11.0047751</datestamp><bib-version>v2</bib-version><id>66097</id><entry>2024-04-20</entry><title>A sufficient condition for the improvement of Restricted Boltzmann Machines</title><swanseaauthors><author><sid>f160e9dde3581f7c178dfeb2436f5df6</sid><firstname>MARK</firstname><surname>THOMAS</surname><name>MARK THOMAS</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-04-20</date><abstract>This thesis explores Restricted Boltzmann Machines (RBMs) and their training, focusing on the minimization of the Kullback-Leibler (KL) divergence. Neural networks and the importance of the KL divergence are introduced and motivated. Examples of KL divergence calculations are demonstrated for various model and target distributions. A demonstration of the non-universality of the ability to improve models by introducing a new parameter without re-training the existing ones is made. The Ising model is explored as an example of available training data, and the work of G. Cossu et al., &#x2018;Machine learning determination of dynamical parameters: The Ising model case,&#x2019; Phys. Rev. B, 100, 064304 (2019) in training a set of RBMs on the one-dimensional Ising model is successfully reproduced. Connections between the mathematics of RBMs and lattice Quantum Field Theory (QFT) are explored, and insights from QFT are utilized to inform the design choices of RBMs to consider. Leveraging these insights, a linearisation procedure is employed to produce a sufficient condition for the possibility of improvement of an RBM with bilinear inter-layer mixing and a Gaussian hidden layer through the introduction of new parameters, without the need to re-train already-existing parameters. This condition is tested and potential issues with the linearisation procedure performed are highlighted.</abstract><type>E-Thesis</type><journal/><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher/><placeOfPublication>Swansea, Wales, UK</placeOfPublication><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic/><keywords/><publishedDay>19</publishedDay><publishedMonth>12</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-12-19</publishedDate><doi/><url/><notes/><college>COLLEGE NANME</college><CollegeCode>COLLEGE CODE</CollegeCode><institution>Swansea University</institution><supervisor>Aarts, Gert ; Cucini, Biagio</supervisor><degreelevel>Master of Research</degreelevel><degreename>MSc by Research</degreename><apcterm/><funders/><projectreference/><lastEdited>2024-04-20T11:32:11.0047751</lastEdited><Created>2024-04-20T11:14:43.2600472</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Biosciences, Geography and Physics - Physics</level></path><authors><author><firstname>MARK</firstname><surname>THOMAS</surname><order>1</order></author></authors><documents><document><filename>66097__30079__828ad801073242c1921bba32f075e8d1.pdf</filename><originalFilename>Thomas_Mark_D_A_MSc_Thesis_Final_Redacted_Signature.pdf</originalFilename><uploaded>2024-04-20T11:25:49.0860476</uploaded><type>Output</type><contentLength>3111405</contentLength><contentType>application/pdf</contentType><version>E-Thesis &#x2013; open access</version><cronfaStatus>true</cronfaStatus><documentNotes>Copyright: The Author, Mark Thomas, 2023.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language></document></documents><OutputDurs/></rfc1807>
spelling 2024-04-20T11:32:11.0047751 v2 66097 2024-04-20 A sufficient condition for the improvement of Restricted Boltzmann Machines f160e9dde3581f7c178dfeb2436f5df6 MARK THOMAS MARK THOMAS true false 2024-04-20 This thesis explores Restricted Boltzmann Machines (RBMs) and their training, focusing on the minimization of the Kullback-Leibler (KL) divergence. Neural networks and the importance of the KL divergence are introduced and motivated. Examples of KL divergence calculations are demonstrated for various model and target distributions. A demonstration of the non-universality of the ability to improve models by introducing a new parameter without re-training the existing ones is made. The Ising model is explored as an example of available training data, and the work of G. Cossu et al., ‘Machine learning determination of dynamical parameters: The Ising model case,’ Phys. Rev. B, 100, 064304 (2019) in training a set of RBMs on the one-dimensional Ising model is successfully reproduced. Connections between the mathematics of RBMs and lattice Quantum Field Theory (QFT) are explored, and insights from QFT are utilized to inform the design choices of RBMs to consider. Leveraging these insights, a linearisation procedure is employed to produce a sufficient condition for the possibility of improvement of an RBM with bilinear inter-layer mixing and a Gaussian hidden layer through the introduction of new parameters, without the need to re-train already-existing parameters. This condition is tested and potential issues with the linearisation procedure performed are highlighted. E-Thesis Swansea, Wales, UK 19 12 2023 2023-12-19 COLLEGE NANME COLLEGE CODE Swansea University Aarts, Gert ; Cucini, Biagio Master of Research MSc by Research 2024-04-20T11:32:11.0047751 2024-04-20T11:14:43.2600472 Faculty of Science and Engineering School of Biosciences, Geography and Physics - Physics MARK THOMAS 1 66097__30079__828ad801073242c1921bba32f075e8d1.pdf Thomas_Mark_D_A_MSc_Thesis_Final_Redacted_Signature.pdf 2024-04-20T11:25:49.0860476 Output 3111405 application/pdf E-Thesis – open access true Copyright: The Author, Mark Thomas, 2023. true eng
title A sufficient condition for the improvement of Restricted Boltzmann Machines
spellingShingle A sufficient condition for the improvement of Restricted Boltzmann Machines
MARK THOMAS
title_short A sufficient condition for the improvement of Restricted Boltzmann Machines
title_full A sufficient condition for the improvement of Restricted Boltzmann Machines
title_fullStr A sufficient condition for the improvement of Restricted Boltzmann Machines
title_full_unstemmed A sufficient condition for the improvement of Restricted Boltzmann Machines
title_sort A sufficient condition for the improvement of Restricted Boltzmann Machines
author_id_str_mv f160e9dde3581f7c178dfeb2436f5df6
author_id_fullname_str_mv f160e9dde3581f7c178dfeb2436f5df6_***_MARK THOMAS
author MARK THOMAS
author2 MARK THOMAS
format E-Thesis
publishDate 2023
institution Swansea University
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Biosciences, Geography and Physics - Physics{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Biosciences, Geography and Physics - Physics
document_store_str 1
active_str 0
description This thesis explores Restricted Boltzmann Machines (RBMs) and their training, focusing on the minimization of the Kullback-Leibler (KL) divergence. Neural networks and the importance of the KL divergence are introduced and motivated. Examples of KL divergence calculations are demonstrated for various model and target distributions. A demonstration of the non-universality of the ability to improve models by introducing a new parameter without re-training the existing ones is made. The Ising model is explored as an example of available training data, and the work of G. Cossu et al., ‘Machine learning determination of dynamical parameters: The Ising model case,’ Phys. Rev. B, 100, 064304 (2019) in training a set of RBMs on the one-dimensional Ising model is successfully reproduced. Connections between the mathematics of RBMs and lattice Quantum Field Theory (QFT) are explored, and insights from QFT are utilized to inform the design choices of RBMs to consider. Leveraging these insights, a linearisation procedure is employed to produce a sufficient condition for the possibility of improvement of an RBM with bilinear inter-layer mixing and a Gaussian hidden layer through the introduction of new parameters, without the need to re-train already-existing parameters. This condition is tested and potential issues with the linearisation procedure performed are highlighted.
published_date 2023-12-19T05:17:35Z
_version_ 1859432119117283328
score 11.099876

Similar Items