# Hot to use together ray tune+ xgboost+cross\_val\_score+early stopping

**URL:** <https://discuss.ray.io/t/hot-to-use-together-ray-tune-xgboost-cross-val-score-early-stopping/2930>\
**Category:** Ray Tune\
**Created:** [July 19, 2021, 11:13am UTC](https://discuss.ray.io/t/hot-to-use-together-ray-tune-xgboost-cross-val-score-early-stopping/2930 "2021-07-19T11:13:22Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Peter\_Pirog](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/peter_pirog/32/234_2.png) [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)\
**Post date:** [July 19, 2021, 11:13am UTC](https://discuss.ray.io/t/hot-to-use-together-ray-tune-xgboost-cross-val-score-early-stopping/2930/1 "2021-07-19T11:13:22Z")

</div>

Hi,  
I try to use together ray tune+ xgboost+cross\_val\_score+early stopping and I have the problem how to add early stopping.

I can use:  
ray tune+ xgboost+cross\_val\_score or ray tune+ xgboost +early stopping  
but I don’t know how tu use both cross\_val\_score and early stopping

I found some tutorials:  
[https://docs.ray.io/en/master/tune/tutorials/tune-xgboost.html#early-stopping](https://docs.ray.io/en/master/tune/tutorials/tune-xgboost.html#early-stopping)

I try to use together ray tune+ xgboost+cross\_val\_score+early stopping and I have the problem how to add early stopping.

I can use:  
ray tune+ xgboost+cross\_val\_score or ray tune+ xgboost +early stopping  
but I don’t know how tu use both cross\_val\_score and early stopping in the same ray tuning

I found tutorials:  
[https://docs.ray.io/en/master/tune/tutorials/tune-xgboost.html#early-stopping](https://docs.ray.io/en/master/tune/tutorials/tune-xgboost.html#early-stopping)

[https://stackoverflow.com/questions/43866284/grid-search-and-early-stopping-using-cross-validation-with-xgboost-in-scikit-lea](https://stackoverflow.com/questions/43866284/grid-search-and-early-stopping-using-cross-validation-with-xgboost-in-scikit-lea)

[10. XGBoost with Hyperopt, Optuna, and Ray](https://towardsdatascience.com/beyond-grid-search-hypercharge-hyperparameter-tuning-for-xgboost-7c78f7a2929d)  
and code:

```auto
fit_params = {"early_stopping_rounds":1,
             "eval_set" : [[testX, testY]]}

model = xgb.XGBRegressor()
gridsearch = GridSearchCV(estimator = xgb.XGBRegressor(), 
                          param_grid=param_grid,
                          fit_params=fit_params,                          
                          verbose=1,                          
                          cv=2)

```

I’m not sure how to use “eval\_set” if I make cross validation . I will be grateful for any hints or ideas.

---

<div class="post-metadata">

**Author:** ![Yard1](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/yard1/32/508_2.png) [@Yard1](https://discuss.ray.io/u/Yard1)\
**Post date:** [July 19, 2021, 1:17pm UTC](https://discuss.ray.io/t/hot-to-use-together-ray-tune-xgboost-cross-val-score-early-stopping/2930/2 "2021-07-19T13:17:44Z")

</div>

Hey @Peter_Pirog, have you considered using tune-sklearn? It has automatic support for XGBoost early stopping. You can install it through `pip install tune-sklearn`, and use it in a similar way to the example here: [tune-sklearn/xgbclassifier.py at master · ray-project/tune-sklearn · GitHub](https://github.com/ray-project/tune-sklearn/blob/master/examples/xgbclassifier.py)

It will automatically run both cross validation and early stopping, up to the value of the `n_estimators` parameter of the passed `XGBRegressor`, and you don’t need to pass any `fit_params`.

---

<div class="post-metadata">

**Author:** ![Peter\_Pirog](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/peter_pirog/32/234_2.png) [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)\
**Post date:** [July 19, 2021, 4:41pm UTC](https://discuss.ray.io/t/hot-to-use-together-ray-tune-xgboost-cross-val-score-early-stopping/2930/3 "2021-07-19T16:41:28Z")

</div>

@Yard1 Thank You for the answer. I will try it. 😀  
I will check paremetwers description in [https://docs.ray.io/en/master/tune/api\_docs/sklearn.html?highlight=tune-sklearn#](https://docs.ray.io/en/master/tune/api_docs/sklearn.html?highlight=tune-sklearn#)