How to set up Random forest using Optuna

How to set up Random forest using Optuna

Fix 200704 If you think about it, max_depth, n_estimators etc. should have been given as discrete valuessuggest_discrete_uniform (name, low, high, q), so I modified the script. If max_depth, n_estimators, max_leaf_nodes are discrete values suggest_discrete_uniform, the type will be float, so change it to ʻint (suggest_discrete_uniform ()) and change the type to an integer type. I'm handing it over. Before correction Since I wrote how to use ʻOptuna last time, I will describe the individual setting method from now on. .. There are various arguments that can be passed in Randomforest, but I set all the main ones in ʻOptuna. I was wondering whether to pass max_depth, n_estimators` as integers or numbers as categories, but this time I passed them as integers.

def objective(trial):
    criterion = trial.suggest_categorical('criterion', ['mse', 'mae'])
    bootstrap = trial.suggest_categorical('bootstrap',['True','False'])
    max_depth = int(trial.suggest_discrete_uniform('max_depth', 10, 1000,10))
    max_features = trial.suggest_categorical('max_features', ['auto', 'sqrt','log2'])
    max_leaf_nodes = int(trial.suggest_discrete_uniform('max_leaf_nodes', 10, 1000,10))
    n_estimators =  int(trial.suggest_discrete_uniform('n_estimators', 10, 1000,10))
 
    
    regr = RandomForestRegressor(bootstrap = bootstrap, criterion = criterion,
                                 max_depth = max_depth, max_features = max_features,
                                 max_leaf_nodes = max_leaf_nodes,n_estimators = n_estimators,n_jobs=2)
 
    score = cross_val_score(regr, X_train, y_train, cv=3, scoring="r2")
    r2_mean = score.mean()
    print(r2_mean)
 
    return r2_mean
#Learn with optuna
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=1000)
 
#Fits tuned hyperparameters
optimised_rf = RandomForestRegressor(bootstrap = study.best_params['bootstrap'], criterion = study.best_params['criterion'],
                                     max_depth = int(study.best_params['max_depth']), max_features = study.best_params['max_features'],
                                     max_leaf_nodes = int(study.best_params['max_leaf_nodes']),n_estimators = int(study.best_params['n_estimators']),
                                     n_jobs=2)
 
optimised_rf.fit(X_train ,y_train)

I used this to tune hyperparameters using a Boston dataset. It fits nicely.

randomforest_Figure_1.png

Recommended Posts

How to set up Random forest using Optuna
How to set up Random forest using Optuna
How to set up SVM using Optuna
How to set xg boost using Optuna
How to set optuna (how to write search space)
How to set up a local development server
How to set layer on Lambda using AWS SAM
How to set up and compile your Cython environment
How to import Python library set up in EFS to Lambda
Day 67 [Introduction to Kaggle] Have you tried using Random Forest?
Survey log of how to optimize LightGBM hyperparameters using Optuna
How to speed up Python calculations
How to set up and use OMC Log Analytics --Linux version -
Set up a node to do MNIST on ROS using Tensorflow
How to set up the development environment of ev3dev [Windows version]
Random Forest (2)
[Blender] How to set shape_key with script
Random Forest
[Python] How to display random numbers (random module)
How to draw a graph using Matplotlib
How to set the server time to Japanese time
How to install a package using a repository
How to set up a jupyter notebook on ssh destination (AWS EC2)
Disease classification in Random Forest using Python
How to download youtube videos using pytube3
Summary of how to set up major Python Lint (pep8, pylint, flake8)
How to display Map using Google Map API (Android)
How to code a drone using image recognition
How to speed up scikit-learn like conda Numpy
How to set Django DB to mongodb visual studio 2019
How to deal with SessionNotCreatedException when using Selenium
How to get article data using Qiita API
How to search HTML data using Beautiful Soup
How to upload to a shared drive using pydrive
How to uninstall a module installed using setup.py
How to set CPU affinity for process threads
How to create random numbers with NumPy's random module
How to set up a Google Colab environment with Coursera's advanced machine learning courses
How to set up a VPN gateway to establish a connection between Alibaba Cloud and AWS
I want to set up a mock server for python-flask in seconds using swagger-codegen.
[Rails] How to get location information using Geolocation API
How to read dynamically generated table definitions using SQLAlchemy
The easiest way to set up Last-Modified in Flask
How to write a GUI using the maya command
How to scrape horse racing data using pandas read_html
How to auto-submit Microsoft Forms using python (Mac version)
Set up a file server on Ubuntu 20.04 using Samba
How to hold a hands-on seminar using Jupyter using docker
How to right click using keyboard input in RPA?
How to make a Python package using VS Code
[Blender] How to dynamically set the selection of EnumProperty
Set PATH equivalent to "sudo su-" using Ansible environment
How to exit when using Python in Terminal (Mac)
[Introduction to Udemy Python3 + Application] 30. How to use the set
How to play Cyberpunk 2077 on Linux/Ubuntu 20.04 using AMD GPU
How to analyze with Google Colaboratory using Kaggle API
How to retrieve multiple arrays using slice in python.
[Introduction to Python] How to stop the loop using break?
How to execute a command using subprocess in Python
How to write faster when using numpy like deque
[Introduction to Python] How to write repetitive statements using for statements