Ray Spaces Support

Gonzalo_Diaz · July 12, 2025, 5:15pm

**1. Severity of the issue: **
Medium: Significantly affects my productivity but can find a workaround.

Hi community, I’ve been building a tool to use RLlib with Unreal Engine, but right now I’m facing an issue that I can’t find an answer. I have tried to implement a utility to use the six types of spaces of rllib, for obs and actions, however I have realized that I can only use Box and Discrete. Any attempt to use Multibinary, or Multidiscret leads me to any of this errors:

RuntimeError: Running the example script resulted in one or more errors! [NotImplementedError('Unsupported action space: MultiBinary(8)')]

or

RuntimeError: Running the example script resulted in one or more errors! [ValueError('No default encoder config for obs space=MultiDiscrete([30 30 30 30 30 30 30]), lstm=False found.')]

Until now I haven’t tried to use Dict or Tuple because they are a little more complex to implement, but I think I will find similar problem there.

More explicitly, I am using my own implementation of tcp_client_env_runner. It works when I use Box or Discrete either in obs or actions, but not with other spaces. My question is, does RLLib support this other spaces for actions and obs? if not, why are they included? or are they just mentioned because they belong to Gymnasium but not to RLLib?

You can find my implementation of multiagent here in case is useful:

github.com/Nullspace-Colombia/unray-server

server/unray_bridge/server/tcp_client_multiagent_env_runner.py

f

import base64
from collections import defaultdict
import gzip
import json
import pathlib
import socket
import tempfile
import threading
import time
from typing import Collection, DefaultDict, List, Optional, Union

import gymnasium as gym
import numpy as np
import onnxruntime

from ray.rllib.core import (
    Columns,
    COMPONENT_RL_MODULE,
    DEFAULT_AGENT_ID,
    DEFAULT_MODULE_ID,

This file has been truncated. show original

Thanks!

PhilippWillms · July 15, 2025, 6:45pm

Hi, reading the error message, this seems to be one of those various encoder topics in new RLlib API stack. Have you already created an issue for that on GitHub?

Gonzalo_Diaz · July 15, 2025, 7:22pm

No, not yet, however I found out that for obs it seems you can only use Box, any attempt to use the other spaces raises the error of encoder, I haven’t tried with Tuple and Dict though. For actions I found out that you can use Discrete, Box and Multidiscrete, but Multibinary raises the error of Unsupported action space. Didn’t try with Dict or Tuple.

I leave it that way by now, because I can handle the use cases with this, but for future implementation, i.e Dict for Camera sensors, I will need to explore if it is supported.

Topic		Replies	Views
RLlib and gym.space RLlib	4	736	November 14, 2021
Is mixed action spaces supported? RLlib	10	1406	February 23, 2023
Does RLlib algorithm support both discrete and continuous action spaces simultaneously? RLlib	7	1651	February 22, 2023
Using MultiBinary as an observation space RLlib	1	431	September 29, 2022
Using Dict observation space with custom RLModule RLlib	7	394	January 6, 2025

Ray Spaces Support

Related topics