# Running RLlib with Ray Tune on GCP

**URL:** <https://discuss.ray.io/t/running-rllib-with-ray-tune-on-gcp/3790>\
**Category:** RLlib\
**Created:** [October 12, 2021, 2:41pm UTC](https://discuss.ray.io/t/running-rllib-with-ray-tune-on-gcp/3790 "2021-10-12T14:41:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [October 12, 2021, 2:41pm UTC](https://discuss.ray.io/t/running-rllib-with-ray-tune-on-gcp/3790/1 "2021-10-12T14:41:35Z")

</div>

Hi fellows,

I want to understand how RLlib can be used in larger scaled experiments and therefore try to get an overview of how to run an experiment with RLlib in the cloud - more precisely on GCP.

I found a helpful introduction to deploying a Ray cluster in the cloud under [Launching Cloud Clusters](https://docs.ray.io/en/latest/cluster/cloud.html) in the docs. I set up a new project on GCP and ran the example code and that works fine. Now, I want to understand how to setup larger experiments with RLlib on GCP. And here I am a little stuck as I do not understand, how to best organize my code, so I hope to get here some best practice guidelines from experienced RLlib/Ray Tune users (@kai, @mannyv, @rliaw 🦊).

* * *

**RLlib’s DQN agent example**  
Let us for an example use vanilla DQN 👾 together with Ray Tune running an experiment with two different learning rates.

I set up my cluster with `ray up -y cluster.yaml`. What now?

1. How can I submit a tuning job with the _DQN_ agent on an environment writing out into _GCS_?
2. Do you also use a `.yaml` file for the _Trainer configuration_ (I saw such in [this file](https://github.com/ray-project/ray/blob/master/rllib/tuned_examples/dqn/atari-dqn.yaml) from @sven1977)?
3. Does any one have a full example from which to learn (i.e. cluster.yaml, config.yaml, etc.)?

* * *

**Custom example**  
Now a more custom example. Let us assume the code is distributed among several files:

```auto
-- my code 
  \__ 
      |__ my_env.py (containing the environment definition) 
      |__ my_policy.py (containing the policy definition)
      |__ my_utilities.py (containing utility functions)
      |__ main.py (main script that can be executed) 

```

1. How can the code be sent to the head node on the GCP cluster to be executed? Do I need to use something like:

```auto
ray rsync_up cluster.yaml 'cluster/my_code' 'local/my_code'

```

and then

```auto
ray exec cluster.yaml 'python main.py'?

```

Many thanks to everyone who tries to help here. I just want to dive deeper and see how I could really use RLlib in projects 🕵️‍♂️ . I also hope to produce a starting point for everyone who wants to run RLlib on GCP 🙌 .

---

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [October 28, 2021, 1:14pm UTC](https://discuss.ray.io/t/running-rllib-with-ray-tune-on-gcp/3790/2 "2021-10-28T13:14:42Z")

</div>

Alright, in the last weeks I found out how to set up an experiment using DQN on GCP:

**RLlib DQN example**  
Using the [`example-full.yaml`](https://github.com/ray-project/ray/blob/master/python/ray/autoscaler/aws/example-full.yaml) I had to make the following changes:

1. See issue [#3858](https://discuss.ray.io/t/tune-cannot-sync-to-gcs/3858) in #ray-clusters, where I also posted the solution.
2. About the `.yaml` file for the _Trainer configuration_ I have no news yet … less priority.
3. So the full example is not there because of 2. missing, but the example in issue [#3858](https://discuss.ray.io/t/tune-cannot-sync-to-gcs/3858) should run for anyone who wants to try this out.

At this point many thanks to the @asawari for this awesome work and for providing so many examples: Setting up the cluster and running the scripts runs amazingly smoothly!!

**Custom example**  
The custom example runs similarly and in the way I expected in above:

1. The code is send to the head node by using `ray rsync-up` as shown above uploading all necessary files to the cluster.
2. To run the `main.py` I used `ray exec` as shown above and the code ran errorless.

Hope this helps others, who stand at the same point in their projects.