Tune-sklearn and early stopping error

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

The script that I’m using-

tune_search = TuneSearchCV(model,
                param_distributions=config,
                n_trials=100,
                early_stopping=True,
                max_iters=10,
                search_optimization="bayesian",
                n_jobs=-1,
                refit=True,
                cv=5,
                verbose=0,
                random_state=42,
                local_dir="./ray_results",
                )

Error that I’m getting -

ValueError: tune-sklearn uses `n_estimators` to warm start, so this parameter can't be set when warm start early stopping.

If I set early_stopping=False, then it automatically sets max_iters=1. I have no idea what’s going on. I’m currently tuning ExtraTreesClassifier from sklearn.

Here is the config -

{
                "n_estimators" : tune.randint(50, 200),
                "min_samples_split" : tune.randint(2, 100),
                "min_samples_leaf" : tune.randint(1, 100),
                "criterion" : tune.choice(["gini", "entropy"]),
                "max_features" : tune.choice(["sqrt", "log2"]),
                #"oob_score" : tune.choice([True, False]),
                "class_weight" : tune.choice([None, "balanced", "balanced_subsample"])
            }
2

For forests/trees, n_estimators indicates the number of trees to build, if I remember correctly.

Early stopping will alter that number (because it will determine whether to build another tree given current performance). Thus, you shouldn’t set n_estimators when using early_stopping.

max_iters specifies how many times tune-sklearn will be given the decision to start/stop training a model. Thus, if you have early_stopping=False, you should set max_iters=1 (let sklearn fit the entire estimator).

1 Like