TuneSearchCV with hyperopt and Ensemble classifiers

Looks like there is something going on with TuneSearchCV and Ensemble classifiers. I’m getting this error -

Traceback (most recent call last):
  File "", line 106, in <module>, Y_train)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/tune_sklearn/", line 664, in fit
    result = self._fit(X, y, groups, **fit_params)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/tune_sklearn/", line 565, in _fit
    analysis = self._tune_run(config, resources_per_trial)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/tune_sklearn/", line 715, in _tune_run
    analysis =, **run_args)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/", line 345, in run
    if config and not search_alg.set_search_properties(metric, mode, config):
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/suggest/", line 53, in set_search_properties
    return self.searcher.set_search_properties(metric, mode, config)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/suggest/", line 258, in set_search_properties
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/ray/tune/suggest/", line 200, in _setup_hyperopt
    self.domain = hpo.Domain(lambda spc: spc, self._space)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/", line 835, in __init__
    self.expr = pyll.as_apply(expr)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/", line 220, in as_apply
    named_args = [(k, as_apply(v)) for (k, v) in items]
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/", line 220, in <listcomp>
    named_args = [(k, as_apply(v)) for (k, v) in items]
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/", line 212, in as_apply
    rval = Apply("pos_args", [as_apply(a) for a in obj], {}, None)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/", line 212, in <listcomp>
    rval = Apply("pos_args", [as_apply(a) for a in obj], {}, None)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/", line 226, in as_apply
    rval = Literal(obj)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/hyperopt/pyll/", line 543, in __init__
    o_len = len(obj)
  File "/home/tmamidi/.conda/envs/training/lib/python3.8/site-packages/sklearn/ensemble/", line 164, in __len__
    return len(self.estimators_)
AttributeError: 'RandomForestClassifier' object has no attribute 'estimators_'

The same error appears with other Ensemble classifiers (like ExtraTrees, GradientBoost) as well. Below is my script -

hyperopt_tune_search = TuneSearchCV(RandomForestClassifier(n_jobs=-1),
                "n_estimators" : tune.randint(10, 200),
                "min_samples_split" : tune.randint(1, 10),
                "min_samples_leaf" : tune.randint(2, 10),
                "criterion" : tune.choice(["gini", "entropy"]),
                "max_features" : tune.choice(["sqrt", "log2"]),
                "class_weight" : tune.choice(["balanced", "balanced_subsample"]),
                "oob_score" : tune.choice([True, False])
                #early_stopping=True, # uses Async HyperBand if set to True
                ), Y_train)

Can you try installing this from master? I think this should be fixed?

Is this the correct git page? I don’t see any new commits. I’ve tried installing it again and still see the error.

Ah ok - this is probably the same issue as [bug] TuneSearchCV with hyperopt raise an error with sklearn.RandomForestClassifier · Issue #186 · ray-project/tune-sklearn · GitHub. Maybe try a different search algorithm for now?

ok Thanks! Glad to see that it is a known issue and you all are working on it.

Do you have any suggestions for a search algorithm that works good with Integer, Real and Categorical parameters?