Optuna is a library that optimizes hyperparameters by Bayesian optimization. This article is a note of an error I encountered when trying to specify different distributions for the same parameters using Optuna.
The reason why I wanted to do this is like max_features in scikit-learn's RandomForest (https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html). For parameters that have three types of specification methods, int type, float type, and category, I thought that it would be easier without thinking if I had Optuna optimize them all together, including which method to specify. is there.
I thought that it would be better to suggest in two steps, referring to "Branches and Loops" at https://optuna.readthedocs.io/en/latest/tutorial/configurations.html.
First, ask for suggestions by suggest_categorical, which is one of the three categories of float, int, and category.
Next, depending on the result of the proposal, try changing the suggest to be called as follows.
--In the case of float, suggest_uniform, suggestenst_loguniform, etc. --suggest_int for int --In the case of category, suggest_categorical
The following error appears during Trial and it ends.
cannot set different distribution kind to the same parameter name.
Bayesian optimization presupposes a distribution of parameters, so it would be inconvenient to give different distributions to the same parameters. So, in conclusion, it seems that there is no choice but to perform as many parameter tuning processes as there are types of specification methods.
Recommended Posts