Getting Exception - RayActorError The actor died because of an error raised in its creation task

RLlib
1

I am using openAI Gym and define a custom Environment as follows
class StockMarketEnv(gym.Env):
“”“Custom Evnvironment with gym interfaces “””
metadata = {‘render.modes’:[‘human’]}
MAX_SHARE_PRICE = 10
MAX_NUM_SHARES = 10
MAX_ACCOUNT_BALANCE = 100

"""Action and Observation Spaces """
def __init__(self,config: EnvContext):
    super(StockMarketEnv,self).__init__()
    self.df =config["df"]
    self.reward_range = (0,MAX_ACCOUNT_BALANCE)
    
    # Actions of the format Buy x%, Sell x%, Hold, etc.
    self.action_space = spaces.Box(low=np.array([0,0]), 
                                   high = np.array([3,1]), dtype=np.float16)
    
    # Prices contains the OHCL values for the last five prices
    self.observation_space = spaces.Box(
      low=0, high=1, shape=(6, 6), dtype=np.float16)

def reset(self):
INITIAL_ACCOUNT_BALANCE = 0
# Reset the state of the environment to an initial state
print(‘Starting with Reset’)
self.balance = INITIAL_ACCOUNT_BALANCE
self.net_worth = INITIAL_ACCOUNT_BALANCE
self.max_net_worth = INITIAL_ACCOUNT_BALANCE
self.shares_held = 0
self.cost_basis = 0
self.total_shares_sold = 0
self.total_sales_value = 0
print(‘Inside reset reached here 1 …’)

    # Set the current step to a random point within the data frame
    self.current_step = random.randint(0, len(self.df.loc[:, 'Open'].values) - 6) 
    print('inside reset self.current_step' , self.current_step)
    return self._next_observation()

Similarly the rest of the methods are defined inside this class

When running the following piece of code
ray.shutdown()
MAX_ACCOUNT_BALANCE = 1000
INITIAL_ACCOUNT_BALANCE =0
dummydata = [[1,2,3,4,5],[2,3,4,5,6],[2,3,4,5,9],[2,3,4,5,10],[2,3,4,5,11],[2,3,4,5,19],[2,3,4,15,18],[2,3,4,15,180]]
stockDF = pd.DataFrame(dummydata,columns = [‘Open’,‘High’,‘Low’,‘Close’,‘Volume’])
print(“stockDF–>”,stockDF)
ray.init(ignore_reinit_error=True)
config = {
“env” : StockMarketEnv,
“env_config”:{
“df”:stockDF,
“num_workers”:1,
“num_gpus”:0
}
}
ppo_config = ppo.DEFAULT_CONFIG.copy()
ppo_config.update(config)

trainer = ppo.PPOTrainer(env=StockMarketEnv,config=ppo_config)

It gives a error
RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=260, ip=172.27.43.203)
File “/home/pdas/smartInvestor/rnd_env1/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py”, line 459, in init
_validate_env(self.env, env_context=self.env_context)
File “/home/pdas/smartInvestor/rnd_env1/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py”, line 1522, in _validate_env
dummy_obs = env.reset()
File “/home/pdas/smartInvestor/rnd_env1/lib/python3.8/site-packages/gym/core.py”, line 74, in reset
raise NotImplementedError
NotImplementedError

Any suggestion ?

2

Hey @PRADIP_DAS, hmmm it seems like your custom env is not being properly set in rllib. @gjoliver @sven1977 @avnishn any ideas here?

Also @PRADIP_DAS, do you think you can fix the formatting for your code? It’s difficult to read as is.

1 Like
3

Hi @PRADIP_DAS ,

it appears to me that the abstract method from gym.Env, reset() is not properly defined. This is why you get a NotImplementedError when executing.

As the code shows only a part of the environment definition, I can only guess: you might have not defined the internal method _next_observation() that should provide the next observation and therefore the reset() method does not return the observation as defined in the observation_space attribute of your environment.
Another possibility is that you defined the _next_observation() method, but it does not return a proper observation as defined by the observation_space attribute.

As mentioned by @amogkam your code is really difficult to read. It would help the readers and in turn yourself to reformat it.

Hope this helps

1 Like