Hi everyone, we’re trying to use the rllib implementation of IMPALA with a custom model and a custom environment. Currently, we’re facing strange errors when using a custom model (simple Actor Critic with FC layers). Running the code below causes a ValueError: Shape (49, 4, 1) must have rank 2 insi…

Hi @thomasbbrunner , The issue is in the value_function. You need to squeeze out the trailing singleton dimension. You can fix it by following the example from here: ray/fcnet.py at 3f89f35e5269c8a9391fb98a535cde7ffd6bcd9d · ray-project/ray · GitHub def value_function(self): return tf.r…

How to use custom model with IMPALA trainer (first time using rllib)

RLlib

mannyv September 8, 2021, 1:47pm 3

Hi @thomasbbrunner,

Welcome to the forum. Do you have a stack trace with the error you are getting?

Custom Impala model

Topic		Replies	Views
Example of custom tensorflow or pytorch neural network RLlib	5	662	December 10, 2021
Custom Impala model Configure Algorithm, Training, Evaluation, Scaling	1	133	June 4, 2024
What happens when you pass a custom model to an actor-critic method RLlib	1	294	March 16, 2022
RLLIB Custom_keras_model RLlib	1	729	July 24, 2022
Question about how to use custom models RLlib	3	1137	September 28, 2021

How to use custom model with IMPALA trainer (first time using rllib)

Related topics