About compute_single_action after training atari breakout

[BlackSparrow-43]

I am a newbie in the field of RL and Rllib. Right now just exploring rllib, training atari ‘breakout’. The difficulty I face is that the making the trained agent to play the game.

The First Problem is that the Agent.compute_single_action(obs) doesn’t automatically preprocess the (1, 210, 160, 3) atari.

So, because of that I tried to use rllibs built-in preprocessor to do so.But it converts the input format
(1, 210, 160, 3) to (1, 84, 84, 3) but the expected input is (?, 84, 84, 4).

So, how do i framestack the frame from 3 to 4 in rllib.

prep = get_preprocessor(env.observation_space)(env.observation_space)
obs = prep.transform(observation)

And is there any other method to solve this?

Ray version==2.0.1

[kourosh]

Hi @BlackSparrow-43,

compute_single_action does not do the processing for you but compute_actions() does. See ray/algorithm.py at master · ray-project/ray · GitHub. This is from master, so if you see any discrepancies switch to the latest version. At some point in the future we will only have one entry point so that people don’t get confused which one to use.