Algo.train() calls env.step() with empty action object

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I made a custom MultiAgentEnv class, and when I print action_dict at the beginning of my step(self, action_dict) method, it’s just an empty object ({}). Shouldn’t algo.train() sample actions from self.action_space and use that as the argument?

I ran GuessTheNumberGame from rllib examples and it’s giving me the same error. I’m very new to Ray/RLlib and not sure what I’m missing.

I have ray 2.5.1, and below is my algo:

algo = (
    PPOConfig()
    .environment(GuessTheNumberGame)
    .build()
)
result = algo.train()

Sorry if this is a silly misunderstanding, and thanks for the help!

2

Need to check your custom environment’s step() function. What are you returning there?