StatelessPendulum not working

mg64ve · December 9, 2021, 2:01pm

I am just testing the following environment:

ray-project/ray/blob/8a72824c63dbae9d8d189ce9c553c2e8c4299494/rllib/examples/env/stateless_cartpole.py

from gym.spaces import Box
import numpy as np

from gym.envs.classic_control import CartPoleEnv


class StatelessCartPole(CartPoleEnv):
    """Partially observable variant of the CartPole gym environment.

    https://github.com/openai/gym/blob/master/gym/envs/classic_control/
    cartpole.py

    We delete the x- and angular velocity components of the state, so that it
    can only be solved by a memory enhanced model (policy).
    """

    def __init__(self, config=None):
        super().__init__()

        # Fix our observation-space (remove 2 velocity components).

This file has been truncated. show original

For this reason I have just made the following notebook:

github.com

mg64ve/ray_tests/blob/main/StatelessPendulum/StatelessPendulum-LSTMwrapper.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "feb1dd3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gym.spaces import Box\n",
    "import numpy as np\n",
    "\n",
    "from gym.envs.classic_control import PendulumEnv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1c48d553",
   "metadata": {},

This file has been truncated. show original

and I am getting null value as reward.
It seems only a list of actions can be used.
Similar notebook with SatelessCartPole is working perfectly:

github.com

mg64ve/ray_tests/blob/main/StatelessPendulum/StatelessCartpole-LSTMwrapper.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "feb1dd3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from gym.spaces import Box\n",
    "import numpy as np\n",
    "\n",
    "from gym.envs.classic_control import CartPoleEnv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1c48d553",
   "metadata": {},

This file has been truncated. show original

What could be the reason?

Topic		Replies	Views
Manual state-in inference does not work with MultiDiscrete environment RLlib	4	290	November 23, 2021
Yet another question on RNN sequencing RLlib	7	728	January 4, 2022
Running a random policy with RLlib RLlib	1	733	February 25, 2022
Registering Custom Environment for `CartPole-v1` with RLlib and Running via Command Line RLlib	8	1120	April 14, 2023
Trials did not complete RLlib	3	2146	April 14, 2023

StatelessPendulum not working

Related Topics