Hello, I was wanting to try something out, I want to “Jump-Start” my agent as seen here: [2204.02372] Jump-Start Reinforcement Learning The way they go about it is, some of the time they inject their own actions from another source other then the model. Therefore you would store the action you in…

It seems something about it was talked about here, but it doesn’t look like anything came from it. [image] Implementing Jump Start Reinforcement Learning in RLLib RLlib Hi @manjrekarom , Not sure if this would help but maybe you can combine these concepts to achieve Jump-s…

Hi @Samuel_Fipps , Welcome to the forum. This is how I would recommend designing jump-start in rllib. I would create a custom environment that wraps the underlying environment and has the know transitions for the guided policy. It also keeps track of which h is being used from H and when it shou…

I might do something a little easier for a proof of concept for now. I think I can change the forward method and change the direct output of the model to make sure that the outputs are correctly updated throughout the code. So basically if I want all my actions to be 0.4 I can do this: @overri…

Hi @Samuel_Fipps , If you want to check if the mechanics work out that should work but you are going to run into two issues with trying to actually learn anything. The first is this paper assumes that the jump start trajectories H are useful trajectories. Having the guide policy produce the same ac…

Yes sorry, I meant I was going to implement the “Guided assistant”, and the proper mechanism to switch between training and eval mode in the forward() method. I was still trying to figure out where I wanted to do this at. I would just be able to override the forward instead of having to make a whol…

@Samuel_Fipps , I did not mean to offend. We get a variety of skill levels and experiences here so I try not to assume anything other than what is in the post. Good luck and yes please share the results if you get some.

Oh no your good, you didn’t offend. I should have some results by the end of week to update this.

I have some results! Here is the code I used: class CustomModel(FullyConnectedNetwork): def __init__(self, obs_space, action_space, num_outputs, model_config, name): super().__init__(obs_space, action_space, num_outputs, model_config, name) self.total_training_steps = 500_000 …

The Jumpstarting results. (I had to make these 2 post since I can’t upload 2 pictures on one post) [jump_starting_with_no_pid]

It looks like the jump-start has helped in that you have a higher win rate and the variance of the win rate appears much lower. Any idea why you the performance degrades over training in both cases? I feel like a broken record because I point a lot of people in this direction. But there can be some…

Jump-Start Reinforcement Learning

RLlib

Samuel_Fipps October 12, 2024, 2:08am 2

It seems something about it was talked about here, but it doesn’t look like anything came from it.

Topic		Replies	Views
PPO nan in actor logits RLlib	7	1024	October 1, 2024
Error: nan Tensors in PyTorch with Ray RLlib for MARL RLlib	12	1368	August 10, 2024
ValueError: Expected parameter logits (...) to satisfy the constraint IndependentConstraint(Real(), 1) RLlib	38	9613	October 14, 2024
TrajectoryTracking with RLLIB RLlib	14	1472	November 17, 2021
Unable to replicate original PPO performance RLlib	0	229	May 10, 2024

Jump-Start Reinforcement Learning

Related topics