Preprocessor error on batches of observations

mannyv · February 4, 2023, 4:20am

It is hard for us to help without seeing the majority of the logic. If you can share that we could have a better idea of what the issue might be.

Is there a reason you are not just using the FCNet as is rather than only using the _hidden_layers and then adding your own logits layer on top?

The way I would approach this is in two parts.

Verify the environment is meeting rllibs expectations.
For this I would use one of the built in models with the custom environment. This will obviously not have the action masking but if it ran cleanly you would know your environment is running as expected.
I would use the custom model with rllibs randomenv. This environment allows you to specify an observation and action space and then generates random values that match the space you provided. Using this environment would allow you to determine if there are issues with the custom model.

github.com

ray-project/ray/blob/ray-2.2.0/rllib/examples/env/random_env.py

import copy
import gym
from gym.spaces import Discrete, Tuple
import numpy as np

from ray.rllib.examples.env.multi_agent import make_multi_agent


class RandomEnv(gym.Env):
    """A randomly acting environment.

    Can be instantiated with arbitrary action-, observation-, and reward
    spaces. Observations and rewards are generated by simply sampling from the
    observation/reward spaces. The probability of a `done=True` after each
    action can be configured, as well as the max episode length.
    """

    def __init__(self, config=None):
        config = config or {}

This file has been truncated. show original

Topic		Replies	Views
`training_step()` fails with custom environment RLlib	1	293	August 1, 2023
Undestanding the expected output shapes of a Recurrent model with Dict Action Space Configure Algorithm, Training, Evaluation, Scaling	2	287	January 15, 2024
Error with custom environments RLlib	1	52	October 14, 2024
Clear documentation on config dicts? RLlib	1	213	December 15, 2022
Questions and Confusion: Getting started with RLlib Configure Algorithm, Training, Evaluation, Scaling	0	42	February 19, 2025

Preprocessor error on batches of observations

Related topics