Failed to use Simplex space with shape=(1,3) and uniform concentration

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi, I’m using the simplex space defined in RRLib as the action space for an environment. I want an action space that contains a single point with three coordinates whose values are between [0,1] and sum to 1. The Simplex constructor takes shape and concentration parameters:

  • I set shape to (1,3) (1 independent 3d Dirichlet)
  • I set concentration to np.array([1, 1, 1]) (uniform).

The problem is that I cannot initialize this environment because it throws an AssertionError exception from the Simplex constructor.

This is the main_ray.py file to reproduce the problem. I am using Python 3.10.14 and Ray 2.23.0:

import numpy as np

import gymnasium as gym

from ray.rllib.utils.spaces.simplex import Simplex

class SimplexTestEnv(gym.Env):
    def __init__(self, env_config):
        self.action_space = Simplex(shape=(1,3), concentration=np.array([1, 1, 1]))
        self.observation_space = gym.spaces.Box(shape=(1,), low=-1, high=1)

    def reset(self):
        return np.zeros(1)

    def step(self, action):
        return np.zeros(1), 0, False, False, info


def main():
    env = SimplexTestEnv({})

    result = env.step(np.zeros(1))

if __name__ == "__main__":
    main()

When the file is executed with python main_ray.py, this is the result:

(.env2.23) emanuele@fedora-t4:~/ray/test-simplex$ python main_ray.py 
Traceback (most recent call last):
  File "/home/emanuele/ray/test-simplex/main_ray.py", line 25, in <module>
    main()
  File "/home/emanuele/ray/test-simplex/main_ray.py", line 20, in main
    env = SimplexTestEnv({})
  File "/home/emanuele/ray/test-simplex/main_ray.py", line 9, in __init__
    self.action_space = Simplex(shape=(1,3), concentration=np.array([1, 1, 1]))
  File "/home/emanuele/ray/.env2.23/lib64/python3.10/site-packages/ray/rllib/utils/spaces/simplex.py", line 32, in __init__
    concentration.shape == shape[:-1]
AssertionError: (3,) vs (1,)
(.env2.23) emanuele@fedora-t4:~/ray/test-simplex$

The problem is inside the ray/rllib/utils/spaces/simplex.py file in Ray. See the constructor:

class Simplex(gym.Space):
    """Represents a d - 1 dimensional Simplex in R^d.
    [...]
    """

    def __init__(self, shape, concentration=None, dtype=np.float32):
        assert type(shape) in [tuple, list]

        super().__init__(shape, dtype)
        self.dim = self.shape[-1]

        if concentration is not None:
            assert (
                concentration.shape == shape[:-1]
            ), f"{concentration.shape} vs {shape[:-1]}"
            self.concentration = concentration
        else:
            self.concentration = np.array([1] * self.dim)

Why is there an assertion concentration.shape == shape[:-1]? Why is there [:-1] in the assertion? For shape=(1,3) and concentration=np.array([1,1,1]):

  • shape[:-1] is (1,)
  • concentration.shape is (3,)

How to solve the problem? Is it a Ray problem or my problem?

2

Hi, just an update: I opened an issue on Github the month after this thread and it turns out that there was a bug that has been fixed in Ray 2.38.