Nested Tuners problem

Ray Tune
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi, Team! I’m trying to use embedded tuners (one is inside of another one’s trainable func):

import pandas as pd
import sys, os, time
sys.path.append(os.getcwd())
import numpy as np
import ray
from ray import tune
from ray.tune.registry import register_env

from ray.rllib.algorithms.dqn.dqn import DQNConfig
from ray import air, tune
from ray.air.config import RunConfig, ScalingConfig, CheckpointConfig
from ray.rllib.utils import check_env
from ray.rllib.algorithms.algorithm import Algorithm
from icecream import ic
from ray.tune.search.bayesopt import BayesOptSearch
from ray.tune.search.optuna import OptunaSearch
import random

def test_embedded_tuners():
    ray.shutdown()
    num_cpus = 2
    local_mode = True
    framework = 'torch'
    evaluation_parallel_to_training = True
    ray.init(num_cpus=num_cpus or None, local_mode=local_mode)

    config = {
                "num_iterations": 10,
                "h1": tune.uniform(0, 3),
                "h2": tune.uniform(2, 7)
             }

    # algo = BayesOptSearch(random_search_steps=7)
    algo = OptunaSearch(metric="score", mode="min")

    def objective(config):
        def inner_objective(config):
            return {"score": random.random()}

        conf = {"id": tune.grid_search(np.arange(0,10))}
        tuner = tune.Tuner(
            inner_objective,
            param_space=conf,
        )
        results = tuner.fit()
        df = results.get_dataframe(filter_metric="score", filter_mode="min")
        return {"score":df["score"].mean()}

    tuner = tune.Tuner(
        objective,
        tune_config=tune.TuneConfig(
            metric="score",
            mode="max",
            search_alg=algo,
            num_samples=7
        ),
        param_space=config,
    )

    results = tuner.fit()
    df = results.get_dataframe(filter_metric="score", filter_mode="min")
    return df


if __name__ == "__main__":
    test_embedded_tuners()

and I’m getting this error(real script output is bigger and I can provide it if neccessary):

[2023-11-26 20:54:23,371 E 225828 226020] core_worker.cc:1727: Pushed Error with JobID: 01000000 of type: task with message: ray::ImplicitFunc.__init__() (pid=225828, ip=192.168.1.65, actor_id=f030c6758e7c50f92449196b01000000, repr=inner_objective)
  File "/home/happycosmonaut/.cache/pypoetry/virtualenvs/tensortrade-VXBAofv_-py3.10/lib/python3.10/site-packages/ray/tune/trainable/trainable.py", line 161, in __init__
    self.setup(copy.deepcopy(self.config))
  File "/home/happycosmonaut/.cache/pypoetry/virtualenvs/tensortrade-VXBAofv_-py3.10/lib/python3.10/site-packages/ray/tune/trainable/function_trainable.py", line 114, in setup
    init_session(
  File "/home/happycosmonaut/.cache/pypoetry/virtualenvs/tensortrade-VXBAofv_-py3.10/lib/python3.10/site-packages/ray/train/_internal/session.py", line 559, in init_session
    raise ValueError(
ValueError: A Train session is already in use. Do not call `init_session()` manually. at time: 1.70101e+09

Can anybody help me with this?

2

Hey there, nested Tuners are not supported in Ray Tune today. Is it possible to model your problem space in a single layer?

3

Hi! I’d like to implement this somehow

def hpo(config):
    
    def trainable(config):
        test_metrics = []
        for i in range(config["num_of_samples"]):     
            # train on train_env
                # I'd like to use stoppers here, 
                # ExperimentPlateauStopper for example

            # test on test_env and get test_metric
            test_metrics.append(test_metric)

        return np.mean(test_metrics)


    hpo_tuner = tune.Tuner(trainable, ...)
    results = hpo_tuner.fit()

Can you advise anything?

4

It might be easiest to just implement the stopping logic as a utility that can be used in the loop directly without Tune,.

5

There are many ready made stoppers and schedulers in Ray Tune. Maybe there are some conventional ways to use them for my needs without Tune?
Or I should implement my own stoppers and schedulers from scratch?

6

There’s no conventional way, but it should be a fairly minimal change to implement your own with the same logic.

1 Like