I’m trying to perform hyperparameter tuning for an xgboost classifier. I gather there are at least two ways to do this and I’m trying to figure out what exactly they do and how they might differ:
(1) tune.run() with the native xgboost interface (https://docs.ray.io/en/latest/tune/tutorials/tune-xgboost.html)
(2) TuneSearchCV using the sklearn interface to xgboost (https://github.com/ray-project/tune-sklearn/blob/master/examples/xgbclassifier.py).
How exactly does early stopping work in either of those cases? Are there differences between these two implementations?
If, say, I run (2) with a classifier with n_estimator=500 and set early_stopping=True and n_trials=20 then tune will run 20 parameter combinations but not for the full 500 boosting rounds, correct?
But what determines after how many rounds the score will be checked and which trials will be prematurely abandoned? Is there a way for me to see after the fact how each of these trials was handled, i.e. when it was killed and why?