How to set optuna (how to write search space)

Make a note of how to set hyperparameters in Optuna.

How to use Optuna, a framework for optimizing hyperparameters. It was created by Preferred Networks.

Official tutorial https://optuna.readthedocs.io/en/stable/tutorial/first.html API reference https://optuna.readthedocs.io/en/stable/reference/index.html

Define the objective function and search space. Next, build a study instance and perform optimization with the ʻoptimize` method.

import optuna

#Definition of objective function
def objective(trial):
    #Describe the search space here


    return ***Objective function**Can be maximized or minimized***

The exploration space is Set with trial.suggest_. Pass it as an integer, category, etc. as follows.

In suggest_categorical (name, choices), in the example, select 3 for the parameter'kernel'. suggest_int (name, low, high, step = 1, log = False) gives the value of an integer parameter. suggest_uniform (name, low, high) gives a linear, continuous value between low and high suggest_loguniform (name, low, high) gives continuous values in logarithmic intervals. suggest_float (name: str, low: float, high: float, *, step: Optional [float] = None) will contain the value of the floating point parameter. With suggest_discrete_uniform (name, low, high, q), the value is sampled from the range low, high, and the discretization number is given in step q.

def objective(trial):
#How to give a search space
    # Categorical parameter
    kernel = trial.suggest_categorical('kernel', ['linear', 'poly', 'rbf'])

    # Int parameter
    num_layers = trial.suggest_int('num_layers', 1, 3)

    # Uniform parameter
    dropout_rate = trial.suggest_uniform('dropout_rate', 0.0, 1.0)

    # Loguniform parameter
    learning_rate = trial.suggest_loguniform('learning_rate', 1e-5, 1e-2)

    # Discrete-uniform parameter
    drop_path_rate = trial.suggest_discrete_uniform('drop_path_rate', 0.0, 1.0, 0.1)
#Automatic optimization of hyperparameters
study = optuna.create_study()
#n_Sets the number of iterations to search in trials.
study.optimize(objective, n_trials = 100)

Recommended Posts

How to set optuna (how to write search space)
How to set up SVM using Optuna
How to set xg boost using Optuna
How to set up Random forest using Optuna
How to set up Random forest using Optuna
XPath Basics (2) -How to write XPath
How to write a Python class
How to write soberly in pandas
Flask reuse How to write html
How to write Docker base image
How to use bing search api
How to write Django1.9 environment-independent wsgi.py
Notes on how to write requirements.txt
Qiita (1) How to write a code name
How to set the server time to Japanese time
How to write Python document comments (Docstrings)
How to search Google Drive with Google Colaboratory
How to write this process in Perl?
How to write Ruby to_s in Python
Summary of how to write AWS Lambda
How to write pydoc and multi-line comments
How to write urlfetch unittest on GAE / P
How to set browser location in Headless Chrome
How to set Django DB to mongodb visual studio 2019
Answer to "Offline real-time how to write F02 problem"
How to write regular expression patterns in Linux
How to add a Python module search path
How to write a ShellScript Bash for statement
Answer to "Offline Real-time How to Write F01 Problem"
How to set up a local development server
How to search HTML data using Beautiful Soup
Answer to "Offline Real-time How to Write E13 Problem"
How to write async and await in Vue.js
How to write a named tuple document in 2020
[Go] How to write or call a function
How to write a ShellScript bash case statement
How to set CPU affinity for process threads
How to set up public key authentication in ssh
20th Offline Real-time How to Write Problems in Python
How to write a GUI using the maya command
How to set up a Python environment using pyenv
How to write string concatenation in multiple lines in Python
How to run python in virtual space (for MacOS)
How to write a list / dictionary type of Python3
How to set layer on Lambda using AWS SAM
[Python] How to write a docstring that conforms to PEP8
[Blender] How to dynamically set the selection of EnumProperty
[Introduction to Udemy Python3 + Application] 30. How to use the set
How to set up and compile your Cython environment
How to write faster when using numpy like deque
Compare how to write processing for lists by language
[Introduction to Python] How to write repetitive statements using for statements
[AWS] How to expand disk space on Amazon linux
How to use xml.etree.ElementTree
How to use Python-shell
How to use tf.data
How to use virtualenv
Scraping 2 How to scrape
How to use Seaboan
How to use image-match
How to use shogun