Ray + Optuna + Grid Search + Pruner

Daraan · July 3, 2025, 12:34pm

I’ve written a lightweight custom Optuna searcher that allows to use ray.tune.grid_search and pruners from optuna with it to early terminate trials like optuna would do.

You can find it in my ray_utilites project on GitHub:

The nearly standalone file (expect one constant) can can be found at:

github.com/Daraan/ray_utilities

ray_utilities/setup/optuna_setup.py

main

"""
See Also:
    https://docs.ray.io/en/latest/tune/examples/optuna_example.html


.. code-block:: python

    param_space = {
        "x": tune.grid_search([1, 2, 3]),  # not compatible with OptunaSearch
        "y": tune.choice([1, 2, 3]),
    }
    search, stopper = create_search_algo(
        "MyStudy",
        hparams=param_space,
        metric=EVAL_METRIC_RETURN_MEAN,  # flattened key
        mode="max",
        storage=None,
        seed=None,  # required
        pruner=None,  # pass your own Pruner or True for MedianPruner
        # Accepts all other kwargs of OptunaSearch

This file has been truncated. show original

The main trick to use pruners is to use the Searcher as a Stopper as well.

christina · July 3, 2025, 8:37pm

Really cool project! I starred it Keep up the great work and ty for sharing!

Topic		Replies	Views
Use optuna terminators for early stopping in optuna.OptunaSearch?	0	373	March 7, 2024
Specify trial resources when using search algorithm to tune hyper-parameters RLlib	2	501	September 7, 2022
Specify trial resources when using Optuna search algorithm to tune hyper-parameters	4	760	October 3, 2022
Conditional search space examples with Ray Tune Ray Tune	0	645	August 31, 2021
Using Optuna hyperparameter importance on Ray Tune experiment with Optuna as search algorithm Ray Tune	1	687	June 6, 2022

Ray + Optuna + Grid Search + Pruner

Related topics