Hello there, I made a custom environment (with Gym API) and I was able to use Rllib for training agents in this environment. I wrote a rule-based "expert " that does not utilize a neural network to sample its actions. I wish to sample trajectories from this “expert” then warm-start my RL agents usi…

Hi @mickelliu and welcome, so to sample data from the expert interactions with your environment I would write a custom policy as shown in the documentation . Then to produce an offline dataset for training you need to use the offline dataset API . Basically, you create from your Custom Policy a Trai…

Thanks, @Lars_Simon_Zehnder for your quick response. Yestersday I figured out a way to do it by defining a custom trainable and policy then manually roll out the workers and run them with tune.run(). But it looks that the trainer template is much cleaner to write.

Hey @mickelliu , you can then also take a look at the BC/MARWIL test cases here: ray.rllib.agents.marwil.tests.test_[bc|marwil].py or at this example script: ray.rllib.examples.offline_rl.py that show how you hook up your dataset and use it to do offline training.

hello @Lars_Simon_Zehnder ! I am also trying to write a rule based policy for data collection purposes, but from ray.rllib.agents.trainer_template import build_traine seems to be deprecated in ray 2.0.0. Do you happen to know what has replaced it? I am trying to register it as an algoirhtm but ther…

Hi @fksvensson , I had to search a little and roll back to ray 1.6.0 to find the trainer_template.py :smiley: . In Ray 2.0.0 this has been changed to class inheritance. You can (and you should) directly inherit from the Algorithm class which is a tune Trainable. In there you can define your own train…

Sample Rule-Based Expert Demonstrations in Rllib

RLlib

Lars_Simon_Zehnder July 29, 2021, 6:03pm 3

Also, maybe this thread helps you.

Topic		Replies	Views
Implementing Jump Start Reinforcement Learning in RLLib RLlib	8	1241	May 27, 2022
Offline data with self made dataset RLlib	1	296	June 7, 2023
[RLlib] Make it easier to play trained policies RLlib	2	799	June 3, 2021
Pre-train a model with baseline policy Offline RL	0	199	February 6, 2024
Behavior Cloning through custom env RLlib	4	551	August 13, 2021

Sample Rule-Based Expert Demonstrations in Rllib

Related topics