RNN support + RAM usage for RL algorithms

qtomcat · January 16, 2023, 3:52pm

Dear developers,

I am AI researcher and very new to Ray, and it looks to me as a very diverse and powerful ML package. I’m looking for a working example of RL algorithm with support of RNN. Let’s say PPO+LSTM that solves Cart Pole, or anything like that. Will appreciate your help a lot.

And also, please, explain why Ray instances take so much RAM on my PC (Windows 11, Anaconda, no CUDA). Is it possible to reduce the memory usage in Ray instances?

mannyv · January 17, 2023, 1:32am

Hi @qtomcat,

Here is an example with PPO / IMPALA in a stateless version of the cartpole environment using an LSTM.

github.com

ray-project/ray/blob/master/rllib/examples/cartpole_lstm.py

import argparse
import os

from ray.rllib.examples.env.stateless_cartpole import StatelessCartPole
from ray.rllib.utils.test_utils import check_learning_achieved

parser = argparse.ArgumentParser()
parser.add_argument(
    "--run", type=str, default="PPO", help="The RLlib-registered algorithm to use."
)
parser.add_argument("--num-cpus", type=int, default=0)
parser.add_argument(
    "--framework",
    choices=["tf", "tf2", "torch"],
    default="tf",
    help="The DL framework specifier.",
)
parser.add_argument("--eager-tracing", action="store_true")
parser.add_argument("--use-prev-action", action="store_true")
parser.add_argument("--use-prev-reward", action="store_true")

This file has been truncated. show original

qtomcat · January 17, 2023, 2:20pm

many thanks, I will try it

Topic		Replies	Views
Custom LSTM model doesn't perform well RLlib	3	542	January 13, 2023
A little help for a novice RLlib	1	385	October 26, 2022
Yet another question on RNN sequencing RLlib	7	797	January 4, 2022
Is LSTM available in PPO's RL Module RLlib	0	13	December 26, 2024
[Rllib] compute_single_action() with an LSTM-PPO trainer fails RLlib	1	952	February 3, 2023

RNN support + RAM usage for RL algorithms

Related topics