Trying to integrate gym environments with RLLIB


I am trying to integrate a simple gym environment with RLLIB and at the moment it looks like this:

import gym
import gym_anytrading
import ray
import pandas as pd
from ray.rllib.algorithms.dqn.dqn import DQNConfig

ray.init()  # Initializes Ray

# Load custom data
df = pd.read_csv('AAPL.CO.csv')
df = df.drop(columns=['Adj Close'])
df = df[['Date', 'Open', 'High', 'Low', 'Close', 'Volume']]
df['Date'] = pd.to_datetime(df['Date'])
df.set_index('Date', inplace=True)

# Gym environment
gym_env = gym.make('stocks-v0', df=df, frame_bound=(5,100), window_size=5)

# Ray config
config = DQNConfig()
replay_config = config.replay_buffer_config.update( 
        "capacity": 60000,
        "prioritized_replay_alpha": 0.5,
        "prioritized_replay_beta": 0.5,
        "prioritized_replay_eps": 3e-6,
config =
config = config.resources(num_gpus=1)  
config = config.rollouts(num_rollout_workers=3)  
config = config.environment(gym_env)

explore_config = config.exploration_config.update( 
        "initial_epsilon": 1.5,
        "final_epsilon": 0.01,
        "epsilone_timesteps": 5000,
)[[1, 1e-3], [500, 5e-3]])\ 

algo =  

The error I get however, seems to be quite unique:

self.exploration_config[“type”] == “ParameterNoise”: Typerror: ‘NoneType’ object is not subscriptable.

Does anyone know something?

Could it be a typo - “epsilone_timesteps” probably should read “epsilon_timesteps” in your explore_config.

I wonder if you got the extra “e” from here, but SimpleQConfig, as DQNConfig’s parent class, has “epsilon_timesteps” here" and here.

Bumped into it earlier - see if it helps.

I think that you need to add "type": str into the update of explore_config.