Model is predicting less than gym.spaces.Box - low values and throwing error

RLlib
1

I am new to ray - reinforcement learning. I am trying to build a ray-ppo algorithm which will predict cloud hardware capacity required for an org in terms of compute, memory. My reward is a simple calculation based on how much the prediction is off from utilization threshold provided by user (For example if the user provides that he wants the compute utilization to be 80%, my reward will be the difference between optimum value and predicted value).

The challenge I am facing is the model is predicting less than gym.spaces.Box - low values after few iterations and throwing error and training getting stopped.

Please find the below error which I’m getting:
ValueError: (‘Observation ({}) outside given space ({})!’, array([ 1.89, 24.12]), Box(1.0, 21186.44, (2,), float64))

2

Hi Vimala,
How are you sure that the output of your model subceeds the lower limit?
If you are not: As you might know, training of RL models is often delicate and decisions such as choosing a high learning rate or a small batch size for a gradient based algorithm s.a. PPO may result in divergence of the optimization.
I have encountered your error message multiple times in this context.
Many parameters can hinder learning and result in your error message, but if I was you, I would try a lower learning rate and/or a larger batch size first.
Let me know if that solves your issue or if you have solved it already!