I'm a transcendental person, so I'll leave it as a memorandum. If you make a mistake, please kindly point it out because it's mental tofu A memorandum that makes the code of the reference site easy to understand The environment is azureml and I'm turning optuna to look for high para
--num_boost_round is the number of gradient boosting iterations --Early_stopping ends the round when the prediction accuracy does not improve the specified number of times for validation. --Callback is a debug-like feature built into XGBoost (ambiguous) --Reference https://xgboost.readthedocs.io/en/latest/python/python_api.html#callback-api
Minimal implementation
def return_callback():
def print_num_boost_round(env):
iteration = env.iteration
msg = '\t'.join([str(x) for x in env.evaluation_result_list])
print(iteration, msg)
As a result
0 ('validation_0-mae', 2657.650391)
1 ('validation_0-mae', 2657.609375)
0 ('validation_0-mae', 2624.649658)
2 ('validation_0-mae', 2657.425049)
1 ('validation_0-mae', 2624.609131)
You get something like Then change the code to
def return_callback():
def print_num_boost_round(env):
print(env)
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7fa972703208>, cvfolds=None, iteration=0, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2657.623047)])
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7fa972703208>, cvfolds=None, iteration=1, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2657.463379)])
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7f7a8224c208>, cvfolds=None, iteration=0, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2624.622314)])
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7fa972703208>, cvfolds=None, iteration=2, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2657.411377)])
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7f7a8224c208>, cvfolds=None, iteration=1, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2624.467285)])
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7fa972703208>, cvfolds=None, iteration=3, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2657.355957)])
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7f0ced02c208>, cvfolds=None, iteration=0, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2639.834229)])
XGBoostCallbackEnv(model=<xgboost.core.Booster object at 0x7f7a8224c208>, cvfolds=None, iteration=2, begin_iteration=0, end_iteration=100, rank=0, evaluation_result_list=[('validation_0-mae', 2624.416016)])
It turns out that the value of iteration is obtained by env.iteration
Reference (https://kunsen.net/2020/05/02/post-3199/)
param_list['num_boost_round'] = trial.suggest_int("num_boost_round", 100, 500)
First, try turning num_boost_round with the initial value of 100 to 500.
If you turn it as it is
{
'max_depth': 20,
'eta': 0.22613771945050443,
'num_leaves': 2560,
'lambda': 6.0425529841148486e-05,
'alpha': 6.69043393720362e-07,
'num_boost_round': 236,
'colsample_bytree': 0.9727432424922707,
'min_child_weight': 239.6173703091301
}
num_boost_round is 236 (not the same every time because it's Optuna's whim) So what is 236 ... Is it going around 236 times in the first place (By the way, it was 253 when I executed it again) As a result output
0 ('validation_0-mae', 2657.650391)
1 ('validation_0-mae', 2657.609375)
0 ('validation_0-mae', 2624.649658)
2 ('validation_0-mae', 2657.425049)
1 ('validation_0-mae', 2624.609131)
Is output, but iteration only rotates up to 100 as end_iteration shows. Next, I searched for the minimum value (manual) Since 135.56956 was the minimum value, I counted the number of lines where that value appeared. The result is 482
If you look closely, just because the iterations are the same does not mean that the values are the same. It might have been easier to understand if I read the XGBoost paper and had it as prerequisite knowledge ... Is there no choice but to push it now ... ??
Recommended Posts