How to parallelize training multiple models

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

I’m working to learn RLlib / Tune / Ray and I could use advice on how to approach my use case.

I currently train a model using Tune.run. Now, I’d like to train 40 to 50 models for different settings using my custom gym environment.

My current approach is training the models one at a time by invoking Tune.run in a loop, changing the parameter setting for each training run.

Is there an approach where I could parallelize the training of models ?

Regards,
Rajesh

1 Like
2

@rliaw @kai could you provide a pointer here?

3

Can you set the parameter settings as a Tune hyperparameter space?

https://docs.ray.io/en/master/tune/api_docs/suggestion.html#tune-basicvariant

1 Like
4

In working to use this suggestion, I’d like to pass in parameters to my custom environment. When I try this, ( see code at end ), I end up with a TypeError.

My question is: What is the correct way to pass information to the custom environment?

Regards,
Rajesh

My call for tune.run():

analysis = tune.run(
    "PPO",
    name="{}_{}_ray_{}_tf_{}".format(
        timelog, "PPO", ray.__version__, tf.__version__),
    num_samples=num_samples,
    metric="episode_reward_mean",
    checkpoint_freq=1,
    checkpoint_score_attr="episode_reward_mean",
    mode="max",
    verbose=1,
    stop={"training_iteration": 50},
    search_alg=BasicVariantGenerator(points_to_evaluate=[
        {"env_config": {2, 11, 0.0, 0.3, 0.7, 30}},

    ]),
    config=config)

The error:

ray.exceptions.RayTaskError(TypeError): ray::RolloutWorker.foreach_policy() (pid=18184, ip=10.0.1.2)
  File "python/ray/_raylet.pyx", line 439, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 473, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 476, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 480, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 432, in ray._raylet.execute_task.function_executor
  File "/home/opc/mlenv/lib64/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 347, in __init__
    env_config or {}, worker_index, num_workers=num_workers)
  File "/home/opc/mlenv/lib64/python3.6/site-packages/ray/rllib/env/env_context.py", line 31, in __init__
    dict.__init__(self, env_config)
TypeError: cannot convert dictionary update sequence element #0 to a sequence
1 Like
5

Hmm, could you show me the example of what you’re trying to do without the BasicVariantGenerator?