How to use non-scalars for points_to_evaluate in Tune / Hyperopt?

Ray Libraries (Data, Train, Tune, Serve)
1

I have a hyperparameter tuple and I would like to supply hyperopt off with a good starting choice like so:

from ray.tune.search.hyperopt import HyperOptSearch
search_space = {"num_hidden": tune.choice([ (250, 250), (500, 500)])}
current_best_params = [{'num_hidden': (250, 250),}]

hyperopt_search = HyperOptSearch(
    metric="mean_loss", mode="min",
    points_to_evaluate=current_best_params)

However, when fitting the tuner, it rejects my search with this error which I am unclear about debugging:

Did not find category with value `(250, 250)` in hyperopt parameter `num_hidden`. Please make sure the specified category is valid.

This search space will work. The tuple is ok, but anything more than a singular value breaks it:

search_space = {"num_hidden": tune.choice([ (1), (200, 500)])}
current_best_params = [{'num_hidden': (1),}]

It would be nice to send tuples as choices instead of setting up conditional hyperparameter searches that selects the second part of the tuple given what the first is.

2

@kai do you know if this is expected to be supported? Is it parsing it as nested values instead?

3

It looks like hyperopt is parsing the tuple as nested values: In the example above it looks like it gets parsed as

[<hyperopt.pyll.base.Apply object at 0x2ac363160>, <hyperopt.pyll.base.Apply object at 0x2ac3631f0>]

We need to convert the categories into numerical values, but it seems we haven’t covered the case of tuples.

Can you file an issue for this @localh? Also, we probably won’t be able to get to this soon. If you’d be up to contribute a fix, that would be very much appreciated. I’m happy to shepherd the PR and assist you if needed.

4

Sure will do on filing an issue; I’ll also take a look at a PR!