# Issue with custom environment

**URL:** <https://discuss.ray.io/t/issue-with-custom-environment/1840>\
**Category:** RLlib\
**Created:** [April 22, 2021, 10:04am UTC](https://discuss.ray.io/t/issue-with-custom-environment/1840 "2021-04-22T10:04:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![PatrickSampaioUSP](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/patricksampaiousp/32/870_2.png) [@PatrickSampaioUSP](https://discuss.ray.io/u/PatrickSampaioUSP)\
**Post date:** [April 22, 2021, 10:04am UTC](https://discuss.ray.io/t/issue-with-custom-environment/1840/1 "2021-04-22T10:04:00Z")

</div>

Hi, I’m using a custom environment, I was using `stable-baslines3` for training but I’m trying to migrate to RLlib for scaling purposes. However after several training iterations, tuning hyperparams my model never converged to a result nearly as good as in the `stabe-baselines3` and I tried to run a episode with a trained file to see what is going on. My agent is always using the maximum actions like:

array([-1., 1., 1., 1., -1.], dtype=float32)

(Actions normalized between -1 and 1)

However my environment does not have a bug because I’m running OK in the `stable-baselines3` and I don’t think there is a issue with my RLlib config, therefore I don’t know why my agent is bugging in those extreme actions.

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [April 27, 2021, 9:51am UTC](https://discuss.ray.io/t/issue-with-custom-environment/1840/2 "2021-04-27T09:51:02Z")

</div>

Hey @PatrickSampaioUSP , thanks for posting this issue. Would you be able to share your environment and config so we can take a look? It’s impossible to figure out why RLlib would not learn otherwise. Thanks.

---

<div class="post-metadata">

**Author:** ![mjlbach](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mjlbach/32/1294_2.png) [@mjlbach](https://discuss.ray.io/u/mjlbach)\
**Post date:** [July 27, 2021, 12:09am UTC](https://discuss.ray.io/t/issue-with-custom-environment/1840/3 "2021-07-27T00:09:53Z")

</div>

I have a reproduction in [Issues reproducing stable-baselines3 PPO performance with rllib](https://discuss.ray.io/t/issues-reproducing-stable-baselines3-ppo-performance-with-rllib/3028)