I tried to setup Ape-X with SpaceInvaders Environment, following the tuned example.
However i cannot see any improvement in the learning curve.
(ignore label in screenshot, interval is 100.000 steps)
The reported reward is about 200-300, not improving over 10 mil timesteps.
Im running on ryzen 5800X with RTX 3060, cpu load is about 85% but gpu load is only 14%.
Only change in config is setting replay size to 250k, because only 16gb ram available.
Would be thankfull for any advice.
Here is my config:
# Works for both torch and tf.
compress_observations: true # decrease size of observations