E-Thesis 243 views 277 downloads
A sufficient condition for the improvement of Restricted Boltzmann Machines / MARK THOMAS
Swansea University Author: MARK THOMAS
Abstract
This thesis explores Restricted Boltzmann Machines (RBMs) and their training, focusing on the minimization of the Kullback-Leibler (KL) divergence. Neural networks and the importance of the KL divergence are introduced and motivated. Examples of KL divergence calculations are demonstrated for variou...
| Published: |
Swansea, Wales, UK
2023
|
|---|---|
| Institution: | Swansea University |
| Degree level: | Master of Research |
| Degree name: | MSc by Research |
| Supervisor: | Aarts, Gert ; Cucini, Biagio |
| URI: | https://cronfa.swan.ac.uk/Record/cronfa66097 |
| Abstract: |
This thesis explores Restricted Boltzmann Machines (RBMs) and their training, focusing on the minimization of the Kullback-Leibler (KL) divergence. Neural networks and the importance of the KL divergence are introduced and motivated. Examples of KL divergence calculations are demonstrated for various model and target distributions. A demonstration of the non-universality of the ability to improve models by introducing a new parameter without re-training the existing ones is made. The Ising model is explored as an example of available training data, and the work of G. Cossu et al., ‘Machine learning determination of dynamical parameters: The Ising model case,’ Phys. Rev. B, 100, 064304 (2019) in training a set of RBMs on the one-dimensional Ising model is successfully reproduced. Connections between the mathematics of RBMs and lattice Quantum Field Theory (QFT) are explored, and insights from QFT are utilized to inform the design choices of RBMs to consider. Leveraging these insights, a linearisation procedure is employed to produce a sufficient condition for the possibility of improvement of an RBM with bilinear inter-layer mixing and a Gaussian hidden layer through the introduction of new parameters, without the need to re-train already-existing parameters. This condition is tested and potential issues with the linearisation procedure performed are highlighted. |
|---|---|
| College: |
Faculty of Science and Engineering |