No Cover Image

Conference Paper/Proceeding/Abstract 107 views 18 downloads

An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms

Leonardo Lucio Custode Orcid Logo, Fabio Caraffini Orcid Logo, Anil Yaman Orcid Logo, Giovanni Iacca Orcid Logo

Proceedings of the Genetic and Evolutionary Computation Conference Companion, Volume: 12, Pages: 1838 - 1845

Swansea University Author: Fabio Caraffini Orcid Logo

  • 3638530.3664163.pdf

    PDF | Version of Record

    © 2024 Copyright held by the owner/author(s). Released under the terms of a CC-BY-NC-SA license.

    Download (695.49KB)

DOI (Published version): 10.1145/3638530.3664163

Abstract

Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionar...

Full description

Published in: Proceedings of the Genetic and Evolutionary Computation Conference Companion
ISBN: 979-8-4007-0495-6 979-8-4007-0495-6
Published: New York, NY, USA ACM 2024
URI: https://cronfa.swan.ac.uk/Record/cronfa67311
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract: Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionary Algorithms have been proposed in the literature, no definitive solution has been found. In this work, we perform a preliminary investigation to automate the reasoning process that leads to the choice of hyperparameter values. We employ two open-source Large Language Models (LLMs), namely Llama2-70b and Mixtral, to analyze the optimization logs online and provide novel real-time hyperparameter recommendations. We study our approach in the context of step-size adaptation for (1 + 1)-ES. The results suggest that LLMs can be an effective method for optimizing hyperparameters in Evolution Strategies, encouraging further research in this direction.
Keywords: Evolutionary Algorithms, Large Language Models, Landscape Analysis, Parameter Tuning
College: Faculty of Science and Engineering
Start Page: 1838
End Page: 1845

Similar Items