# Trying to integrate gym environments with RLLIB
**URL:** https://discuss.ray.io/t/trying-to-integrate-gym-environments-with-rllib/11667
**Category:** RLlib
**Created:** [August 3, 2023, 3:02pm UTC](https://discuss.ray.io/t/trying-to-integrate-gym-environments-with-rllib/11667 "2023-08-03T15:02:40Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@Villadsg](https://discuss.ray.io/u/Villadsg)
#### Post date: [August 3, 2023, 3:02pm UTC](https://discuss.ray.io/t/trying-to-integrate-gym-environments-with-rllib/11667/1 "2023-08-03T15:02:40Z")
Hi,
I am trying to integrate a simple gym environment with RLLIB and at the moment it looks like this:
```auto
import gym
import gym_anytrading
import ray
import pandas as pd
from ray.rllib.algorithms.dqn.dqn import DQNConfig
ray.init() # Initializes Ray
# Load custom data
df = pd.read_csv('AAPL.CO.csv')
df = df.drop(columns=['Adj Close'])
df = df[['Date', 'Open', 'High', 'Low', 'Close', 'Volume']]
df['Date'] = pd.to_datetime(df['Date'])
df.set_index('Date', inplace=True)
# Gym environment
gym_env = gym.make('stocks-v0', df=df, frame_bound=(5,100), window_size=5)
# Ray config
config = DQNConfig()
print(config.replay_buffer_config)
replay_config = config.replay_buffer_config.update(
{
"capacity": 60000,
"prioritized_replay_alpha": 0.5,
"prioritized_replay_beta": 0.5,
"prioritized_replay_eps": 3e-6,
}
)
config = config.training(replay_buffer_config=replay_config)
config = config.resources(num_gpus=1)
config = config.rollouts(num_rollout_workers=3)
config = config.environment(gym_env)
print(config.exploration_config)
explore_config = config.exploration_config.update(
{
"initial_epsilon": 1.5,
"final_epsilon": 0.01,
"epsilone_timesteps": 5000,
}
)
config.training(lr_schedule=[[1, 1e-3], [500, 5e-3]])\
.exploration(exploration_config=explore_config)
algo = config.build()
algo.train()
```
The error I get however, seems to be quite unique:
> self.exploration\_config[“type”] == “ParameterNoise”: Typerror: ‘NoneType’ object is not subscriptable.
Does anyone know something?
---
### Author:  [@Teenforever](https://discuss.ray.io/u/Teenforever)
#### Post date: [August 9, 2023, 3:52am UTC](https://discuss.ray.io/t/trying-to-integrate-gym-environments-with-rllib/11667/2 "2023-08-09T03:52:56Z")
Could it be a typo - “epsilone\_timesteps” probably should read “epsilon\_timesteps” in your explore\_config.
I wonder if you got the extra “e” from [here](https://github.com/ray-project/ray/blob/8afcf9b25ed334dd80666147ff03bae75bdb9cb9/rllib/algorithms/dqn/dqn.py#L95), but SimpleQConfig, as DQNConfig’s parent class, has “epsilon\_timesteps” [here](https://github.com/ray-project/ray/blob/8afcf9b25ed334dd80666147ff03bae75bdb9cb9/rllib/algorithms/simple_q/simple_q.py#L87)" and [here](https://github.com/ray-project/ray/blob/8afcf9b25ed334dd80666147ff03bae75bdb9cb9/rllib/algorithms/simple_q/simple_q.py#L148).
Bumped into it earlier - see if it helps.
---
### Author:  [@hermmanhender](https://discuss.ray.io/u/hermmanhender)
#### Post date: [January 9, 2024, 1:25pm UTC](https://discuss.ray.io/t/trying-to-integrate-gym-environments-with-rllib/11667/3 "2024-01-09T13:25:23Z")
I think that you need to add `"type": str` into the update of `explore_config`.