Try function optimization using Hyperopt

Introduction

Hyperopt is an auto-optimization framework for hyperparameters. It seems to be used mainly for hyperparameter tuning of machine learning.

Preparation

First, let's install the library. You can install it with pip install hyperopt </ font>.

Experiment

This time

x^2+y^2+z^2

Let's optimize the minimization problem of.

Definition of objective function

First, let's define the objective function.

#Set objective function
def objective_hyperopt(args):
    x, y, z = args
    return x ** 2 + y ** 2 + z ** 2

Optimization execution

First, let's set the search space for the parameters to be optimized. Then use fmin () to start the search. Let's set the number of searches with the argument max_evals.

#Optimized with hyperopt
def hyperopt_exe():
    space = [
        hp.uniform('x', -100, 100),
        hp.uniform('y', -100, 100),
        hp.uniform('z', -100, 100)
    ]

    #An object for recording the state of the search
    trials = Trials()

    #Start exploration
    best = fmin(objective_hyperopt, space, algo=tpe.suggest, max_evals=500, trials=trials)

If you want to know the final result, add the following.

    #Output the result
    print(best)

Let's retrieve the information being searched from the trials object. You can display the parameters and objective function values for each trial by adding the following:

    #Examine the search process
    for i, n in zip(trials.trials, range(500)):
        vals = i['misc']['vals']
        result = i['result']['loss']
        print('vals:', vals, 'result:', result)

code

The code this time is as follows.

# -*- coding: utf-8 -*-
import hyperopt
from hyperopt import hp
from hyperopt import fmin
from hyperopt import tpe
from hyperopt import Trials
import matplotlib.pyplot as plt

#Set objective function for hyperopt
def objective_hyperopt(args):
    x, y, z = args
    return x ** 2 + y ** 2 + z ** 2

#Optimized with hyperopt
def hyperopt_exe():
    #Search space settings
    space = [
        hp.uniform('x', -100, 100),
        hp.uniform('y', -100, 100),
        hp.uniform('z', -100, 100)
    ]

    #An object for recording the state of the search
    trials = Trials()

    #Start exploration
    best = fmin(objective_hyperopt, space, algo=tpe.suggest, max_evals=500, trials=trials)
    #Output the result
    print(best)

    epoches = []
    values = []
    best = 100000
    #Examine the search process
    for i, n in zip(trials.trials, range(500)):
        if best > i['result']['loss']:
            best = i['result']['loss']
        epoches.append(n+1)
        values.append(best)
        vals = i['misc']['vals']
        result = i['result']['loss']
        print('vals:', vals, 'result:', result)

    #Draw graph
    plt.plot(epoches, values, color="red")
    plt.title("hyperopt")
    plt.xlabel("trial")
    plt.ylabel("value")
    plt.show()

if __name__ == '__main__':
    hyperopt_exe()

result

The figure of the result of this experiment is as follows. It has converged at an early stage. hyperopt.png

Reference site

Function optimization using Hyperopt Python: Select hyperparameters of machine learning model with Hyperopt

Recommended Posts

Try function optimization using Hyperopt
Try function optimization with Optuna
Try using Tkinter
Try using docker-py
Try using cookiecutter
Try using PDFMiner
Try using scanpy's data integration function (sc.tl.ingest)
Try using geopandas
Try using Selenium
Try using scipy
Try using pandas.DataFrame
Try using django-swiftbrowser
Try using matplotlib
Try using tf.metrics
Try using platypus, a multipurpose optimization library
Try using PyODE
Try to solve the function minimization problem using particle swarm optimization
Function fitting (using Keras)
Try using virtualenv (virtualenvwrapper)
[Azure] Try using Azure Functions
Try using virtualenv now
Try using W & B
Try using Django templates.html
[Kaggle] Try using LGBM
Try using Python's feedparser.
Try using Python's Tkinter
Try using Tweepy [Python2.7]
Try using Pytorch's collate_fn
Try using PythonTex with Texpad.
[Python] Try using Tkinter's canvas
Try using Jupyter's Docker image
Try using scikit-learn (1) --K-means clustering
Try using matplotlib with PyCharm
Try using Azure Logic Apps
Try using Kubernetes Client -Python-
[Kaggle] Try using xg boost
Try using the Twitter API
Try using OpenCV on Windows
Try using Jupyter Notebook dynamically
[Optimization problem] Optuna vs Hyperopt
Try creating a CRUD function
Try using AWS SageMaker Studio
Try tweeting automatically using Selenium.
Try using SQLAlchemy + MySQL (Part 1)
Try using the Twitter API
Notes on optimization using Pytorch
Try using SQLAlchemy + MySQL (Part 2)
Try using Django's template feature
Try using the PeeringDB 2.0 API
Try using Pelican's draft feature
Try using pytest-Overview and Samples-
Try running a function written in Python using Fn Project
Try encryption / decryption using OpenSSL key with Python3 pow function
Try using folium with anaconda
Try using Janus gateway's Admin API
Try using Spyder included in Anaconda
Solve multivariable optimization problems using sagemath
Try using design patterns (exporter edition)
Try using Pillow on iPython (Part 1)
Try using Pillow on iPython (Part 2)
Try using Pleasant's API (python / FastAPI)