TuneSearchCV with hyperopt and Ensemble classifiers

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

Looks like there is something going on with TuneSearchCV and Ensemble classifiers. I’m getting this error -

Traceback (most recent call last):
  File "Tune_models_copy.py", line 106, in <module>
    hyperopt_tune_search.fit(X_train, Y_train)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/tune_sklearn/tune_basesearch.py", line 664, in fit
    result = self._fit(X, y, groups, **fit_params)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/tune_sklearn/tune_basesearch.py", line 565, in _fit
    analysis = self._tune_run(config, resources_per_trial)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/tune_sklearn/tune_search.py", line 715, in _tune_run
    analysis = tune.run(trainable, **run_args)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/tune.py", line 345, in run
    if config and not search_alg.set_search_properties(metric, mode, config):
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/suggest/search_generator.py", line 53, in set_search_properties
    return self.searcher.set_search_properties(metric, mode, config)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/suggest/hyperopt.py", line 258, in set_search_properties
    self._setup_hyperopt()
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/suggest/hyperopt.py", line 200, in _setup_hyperopt
    self.domain = hpo.Domain(lambda spc: spc, self._space)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/base.py", line 835, in __init__
    self.expr = pyll.as_apply(expr)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/base.py", line 220, in as_apply
    named_args = [(k, as_apply(v)) for (k, v) in items]
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/base.py", line 220, in <listcomp>
    named_args = [(k, as_apply(v)) for (k, v) in items]
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/base.py", line 212, in as_apply
    rval = Apply("pos_args", [as_apply(a) for a in obj], {}, None)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/base.py", line 212, in <listcomp>
    rval = Apply("pos_args", [as_apply(a) for a in obj], {}, None)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/base.py", line 226, in as_apply
    rval = Literal(obj)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/base.py", line 543, in __init__
    o_len = len(obj)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/sklearn/ensemble/_base.py", line 164, in __len__
    return len(self.estimators_)
AttributeError: 'RandomForestClassifier' object has no attribute 'estimators_'

The same error appears with other Ensemble classifiers (like ExtraTrees, GradientBoost) as well. Below is my script -

hyperopt_tune_search = TuneSearchCV(RandomForestClassifier(n_jobs=-1),
                param_distributions={
                "n_estimators" : tune.randint(10, 200),
                "min_samples_split" : tune.randint(1, 10),
                "min_samples_leaf" : tune.randint(2, 10),
                "criterion" : tune.choice(["gini", "entropy"]),
                "max_features" : tune.choice(["sqrt", "log2"]),
                "class_weight" : tune.choice(["balanced", "balanced_subsample"]),
                "oob_score" : tune.choice([True, False])
                            },
                #n_trials=5,
                #early_stopping=True, # uses Async HyperBand if set to True
                max_iters=100,
                search_optimization="hyperopt",
                n_jobs=-1,
                #refit=True,
                #cv=5,
                verbose=1,
                random_state=42,
                local_dir="./ray_results",
                )
        hyperopt_tune_search.fit(X_train, Y_train)
2

Can you try installing this from master? I think this should be fixed?

3

Is this the correct git page? I don’t see any new commits. I’ve tried installing it again and still see the error.

4

Ah ok - this is probably the same issue as [bug] TuneSearchCV with hyperopt raise an error with sklearn.RandomForestClassifier · Issue #186 · ray-project/tune-sklearn · GitHub. Maybe try a different search algorithm for now?

5

ok Thanks! Glad to see that it is a known issue and you all are working on it.

Do you have any suggestions for a search algorithm that works good with Integer, Real and Categorical parameters?