I am confused about the max_iters
parameter in ray.tune.sklearn.TuneGridSearchCV
.
Assume I have e.g. an sklearn.ElasticNet trainable. In that trainable, I define the number of iterations that I allow the respective optimizer to run:
ELN = ElasticNet(max_iter=1000)
Now when I want to tune this trainable via ray.tune.sklearn.TuneGridSearchCV
, I can define a max_iters
which, according to the documentation “Indicates the maximum number of epochs to run for each hyperparameter configuration sampled.”. E.g. assuming some pre-defined parameter space (param
):
ray.tune.sklearn.TuneGridSearchCV(estimator=ELN, param_grid=param, max_iters=100)
Now my question, formulated as an example: Let’s say I set max_iter=1000
and max_iters=100
as in the example above. Does my optimizer run for 100 iterations and the scheduler checks every iteration wether to stop or not, or does it run for 1000 iterations and the scheduler checks every 10 iterations whether to stop a trial or not?
I want to tune an ElasticNet object, such that the training stops if the scoring does not improve for 8 iterations, and I am currently confused how to do that.