Action space Discrete is not supported for DQN

PeterF · September 28, 2024, 7:02pm

How severe does this issue affect your experience of using Ray?

Medium: It contributes to significant difficulty in completing my task, but I can work around it.

Hello everyone,
I learning RLlib, and I am trying to replicate the “tutorial” contained in the book “learning ray flexible distributed python for Machine Learning” concerning a maze environment for a multi-agent application.

Before starting the training step, I get the following error message: UnsupportedSpaceException: Action space Discrete(4) is not supported for DQN.

But DQN should support discrete spaces.

Can anyone help me?

Thanks in advance.

MCW_Lad · September 24, 2025, 6:04am

github.com/ray-project/ray

rllib/tuned_examples/dqn/cartpole_dqn.py

be3e513dd


      
          from ray.rllib.algorithms.dqn import DQNConfig
          from ray.rllib.core.rl_module.default_model_config import DefaultModelConfig
          from ray.rllib.utils.test_utils import add_rllib_example_script_args
          
          parser = add_rllib_example_script_args(
              default_reward=450.0,
              default_timesteps=200000,
          )
          # Use `parser` to add your own custom command line options to this script
          # and (if needed) use their values to set up `config` below.
          args = parser.parse_args()
          
          config = (
              DQNConfig()

This example shows CartPole-v1, which has a Discrete action space, running on a DQN. It might be that you’ve got a custom model config, or something of the kind, that’s not getting fed in properly. If you’ve got a replication script, I could take a look.

Edit: Whoops, this is from September of the wrong year.

Topic		Replies	Views
Action space compatibility with algorithms RLlib	1	568	May 23, 2023
Unsupported Action Space Exception (Dict with a DQN) RLlib	3	492	June 24, 2024
[RLlib] Why some algorithms do not suppport multiagent or discrete/continuous action space? RLlib	1	498	January 25, 2021
Discrete tuple action space for simple Q RLlib	4	1308	October 14, 2021
Rainbow/DQN with MultiDiscrete Action Spaces RLlib	2	2457	May 24, 2021

Action space Discrete is not supported for DQN

Related topics