# TuneSearchCV - bohb Are you using HyperBandForBOHB as a scheduler? **URL:** \ **Category:** Uncategorized\ **Created:** [October 4, 2022, 1:17pm UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782 "2022-10-04T13:17:33Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.ray.io/u/George)\ **Post date:** [October 4, 2022, 1:17pm UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/1 "2022-10-04T13:17:33Z")
Hello , I wanted to ask . According to [https://docs.ray.io/en/latest/tune/api\_docs/sklearn.html](https://docs.ray.io/en/latest/tune/api_docs/sklearn.html) we can choose `search_optimization='bohb'` So, I am using this, in my `TuneSearchCV` and I am receiving: `BOHB Info not detected in result. Are you using HyperBandForBOHB as a scheduler?` So, i am not sure, does it uses the `bohb` or not? Thanks ---
**Author:** ![Yard1](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yard1/32/508_2.png) [@Yard1](https://discuss.ray.io/u/Yard1)\ **Post date:** [October 4, 2022, 9:35pm UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/2 "2022-10-04T21:35:32Z")
Hey @George, can you show how you call `TuneSearchCV`? ---
**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.ray.io/u/George)\ **Post date:** [October 5, 2022, 6:03am UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/3 "2022-10-05T06:03:35Z")
Hi, yes: ```auto for i, (train_idx, val_idx) in enumerate(kfold.split(x_train, y_train)): x_train_, y_train_ = x_train[train_idx, :], y_train[train_idx] x_val_, y_val_ = x_train[val_idx, :], y_train[val_idx] # Initialize classifier xgb_class = xgb.XGBClassifier(objective ='multi:softprob', use_label_encoder=False, eval_metric=["merror", "mlogloss", "auc"] , seed=123, enable_categorical=False) model = TuneSearchCV( xgb_class, param_distributions=xgb_params, n_trials=20, max_iters=10, search_optimization='bohb, early_stopping=True, scoring='f1_micro', n_jobs=4, name='Ray tune', verbose=0, local_dir='./ray_results', ) history = model.fit(x_train_, y_train_, eval_metric=["merror", "mlogloss", "auc"], early_stopping_rounds=20, eval_set=[(x_train_, y_train_), (x_val_, y_val_)]) ``` ---
**Author:** ![Yard1](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yard1/32/508_2.png) [@Yard1](https://discuss.ray.io/u/Yard1)\ **Post date:** [October 5, 2022, 4:27pm UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/4 "2022-10-05T16:27:30Z")
Hey, thanks, I can reproduce the issue. This will be fixed soon. In the mean time, as a workaround, try not passing the `early_stopping` argument. That should make `TuneSearchCV` use the correct scheduler. ---
**Author:** ![Yard1](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yard1/32/508_2.png) [@Yard1](https://discuss.ray.io/u/Yard1)\ **Post date:** [October 5, 2022, 6:20pm UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/5 "2022-10-05T18:20:29Z")
Fixed in [Release tune-sklearn 0.4.4 · ray-project/tune-sklearn · GitHub](https://github.com/ray-project/tune-sklearn/releases/tag/v0.4.4)! ---
**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.ray.io/u/George)\ **Post date:** [October 6, 2022, 6:15am UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/6 "2022-10-06T06:15:17Z")
Nice @Yard1 ! I updated `tune-sklearn` to `0.4.4`. Now, it doesn’t show the message even with `early_stopping=True` But now ( I don’t remember if this message was before, I think it wasn’t) , it shows me a : `UserWarning: tune-sklearn implements incremental learning for xgboost models following this: https://github.com/dmlc/xgboost/issues/1686. This may negatively impact performance. To disable, set `early\_stopping=False`.` Also, it shows me: ```auto - The `callbacks.on_trial_result` operation took 1.253 s, which may be a performance bottleneck. 2022-10-06 09:09:40,163 WARNING util.py:220 -- The `process_trial_result` operation took 1.254 s, which may be a performance bottleneck. 2022-10-06 09:09:40,169 WARNING util.py:220 -- Processing trial results took 1.261 s, which may be a performance bottleneck. Please consider reporting results less frequently to Ray Tune. 2022-10-06 09:09:40,170 WARNING util.py:220 -- The `process_trial_result` operation took 1.262 s, which may be a performance bottleneck. 2022-10-06 09:09:43,151 WARNING util.py:220 -- The `start_trial` operation took 1.314 s, which may be a performance bottleneck. ``` These messages were there before the `tune-sklearn` update also. ---
**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.ray.io/u/George)\ **Post date:** [October 6, 2022, 6:50am UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/7 "2022-10-06T06:50:51Z")
Hmm… I am receiving messages that I haven’t before : ```auto 2022-10-06 09:18:50,164 WARNING util.py:220 -- The `start_trial` operation took 1.321 s, which may be a performance bottleneck. (_Trainable pid=204883) 2022-10-06 09:21:39,959 INFO trainable.py:668 -- Restored on 10.20.0.57 from checkpoint: /tmp/checkpoint_tmp_tpz_yrdk (_Trainable pid=204883) 2022-10-06 09:21:39,960 INFO trainable.py:677 -- Current state after restoring: {'_iteration': 1, '_timesteps_total': None, '_time_total': 430.8777885437012, '_episodes_total': None} 2022-10-06 09:21:40,158 WARNING util.py:220 -- The `start_trial` operation took 1.618 s, which may be a performance bottleneck. 2022-10-06 09:21:41,795 WARNING util.py:220 -- The `start_trial` operation took 1.636 s, which may be a performance bottleneck. (_Trainable pid=209771) /home/ggous/miniconda3/envs/sklearn/lib/python3.9/site-packages/xgboost/compat.py:36: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead. (_Trainable pid=209771) from pandas import MultiIndex, Int64Index (_Trainable pid=209771) 2022-10-06 09:21:48,120 INFO trainable.py:668 -- Restored on 10.20.0.57 from checkpoint: /tmp/checkpoint_tmp_4l38pd6u (_Trainable pid=209771) 2022-10-06 09:21:48,120 INFO trainable.py:677 -- Current state after restoring: {'_iteration': 1, '_timesteps_total': None, '_time_total': 429.09668040275574, '_episodes_total': None} ``` I tried to use : `early_stopping=False` , but I am receiving: `early_stopping must not be False when using BOHB` ---
**Author:** ![Yard1](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yard1/32/508_2.png) [@Yard1](https://discuss.ray.io/u/Yard1)\ **Post date:** [October 6, 2022, 8:30am UTC](https://discuss.ray.io/t/tunesearchcv-bohb-are-you-using-hyperbandforbohb-as-a-scheduler/7782/8 "2022-10-06T08:30:52Z")
The bottleneck messages can be ignored. As the message says, using early stopping with XGBoost may negatively impact performance in some cases, especially for small datasets. BOHB is combined bayesian optimization with early stopping, which means early stopping cannot be turned off. You can choose to use a different search algorithm, eg. optuna or hyperopt, which can work without early stopping.