Nested Tuners problem

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi, Team! I’m trying to use embedded tuners (one is inside of another one’s trainable func):

import pandas as pd
import sys, os, time
import numpy as np
import ray
from ray import tune
from ray.tune.registry import register_env

from ray.rllib.algorithms.dqn.dqn import DQNConfig
from ray import air, tune
from ray.air.config import RunConfig, ScalingConfig, CheckpointConfig
from ray.rllib.utils import check_env
from ray.rllib.algorithms.algorithm import Algorithm
from icecream import ic
from import BayesOptSearch
from import OptunaSearch
import random

def test_embedded_tuners():
    num_cpus = 2
    local_mode = True
    framework = 'torch'
    evaluation_parallel_to_training = True
    ray.init(num_cpus=num_cpus or None, local_mode=local_mode)

    config = {
                "num_iterations": 10,
                "h1": tune.uniform(0, 3),
                "h2": tune.uniform(2, 7)

    # algo = BayesOptSearch(random_search_steps=7)
    algo = OptunaSearch(metric="score", mode="min")

    def objective(config):
        def inner_objective(config):
            return {"score": random.random()}

        conf = {"id": tune.grid_search(np.arange(0,10))}
        tuner = tune.Tuner(
        results =
        df = results.get_dataframe(filter_metric="score", filter_mode="min")
        return {"score":df["score"].mean()}

    tuner = tune.Tuner(

    results =
    df = results.get_dataframe(filter_metric="score", filter_mode="min")
    return df

if __name__ == "__main__":

and I’m getting this error(real script output is bigger and I can provide it if neccessary):

[2023-11-26 20:54:23,371 E 225828 226020] Pushed Error with JobID: 01000000 of type: task with message: ray::ImplicitFunc.__init__() (pid=225828, ip=, actor_id=f030c6758e7c50f92449196b01000000, repr=inner_objective)
  File "/home/happycosmonaut/.cache/pypoetry/virtualenvs/tensortrade-VXBAofv_-py3.10/lib/python3.10/site-packages/ray/tune/trainable/", line 161, in __init__
  File "/home/happycosmonaut/.cache/pypoetry/virtualenvs/tensortrade-VXBAofv_-py3.10/lib/python3.10/site-packages/ray/tune/trainable/", line 114, in setup
  File "/home/happycosmonaut/.cache/pypoetry/virtualenvs/tensortrade-VXBAofv_-py3.10/lib/python3.10/site-packages/ray/train/_internal/", line 559, in init_session
    raise ValueError(
ValueError: A Train session is already in use. Do not call `init_session()` manually. at time: 1.70101e+09

Can anybody help me with this?

Hey there, nested Tuners are not supported in Ray Tune today. Is it possible to model your problem space in a single layer?

Hi! I’d like to implement this somehow

def hpo(config):
    def trainable(config):
        test_metrics = []
        for i in range(config["num_of_samples"]):     
            # train on train_env
                # I'd like to use stoppers here, 
                # ExperimentPlateauStopper for example

            # test on test_env and get test_metric

        return np.mean(test_metrics)

    hpo_tuner = tune.Tuner(trainable, ...)
    results =

Can you advise anything?

It might be easiest to just implement the stopping logic as a utility that can be used in the loop directly without Tune,.

There are many ready made stoppers and schedulers in Ray Tune. Maybe there are some conventional ways to use them for my needs without Tune?
Or I should implement my own stoppers and schedulers from scratch?

There’s no conventional way, but it should be a fairly minimal change to implement your own with the same logic.

1 Like