# Parallel workers compute action

**URL:** <https://discuss.ray.io/t/parallel-workers-compute-action/2421>\
**Category:** RLlib\
**Created:** [June 6, 2021, 2:30pm UTC](https://discuss.ray.io/t/parallel-workers-compute-action/2421 "2021-06-06T14:30:33Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vishalrangras](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/vishalrangras/32/976_2.png) [@vishalrangras](https://discuss.ray.io/u/vishalrangras)\
**Post date:** [June 6, 2021, 2:30pm UTC](https://discuss.ray.io/t/parallel-workers-compute-action/2421/1 "2021-06-06T14:30:33Z")

</div>

I am using the logic of computing action as described in documentation (screenshot below), for a custom evaluation of policy. Basically, I want to evaluate various policies trained on different algorithms and hyperparameters, for my domain specific metrics. I don’t want to use custom callback and evaluation during or at the end of training. I want to evaluate the policies later using compute action approach shown below.

I want to do parallel rollout of environment on multiple cores while evaluating single policy for around 50 episodes. I am passing env\_config while creating the environment but still getting the exception “ **AttributeError: ‘dict’ object has no attribute ‘worker\_index’** ”. I believe the env\_context meta-data is not been created like it gets created while training the policy on multiple cores.

How can I use multiple cores to evaluate/rollout the policy in parallel environments using the following code snippet?

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/7/7c32e0c4dfc5ea751224f1634543fd1a049e0194.png)

---

<div class="post-metadata">

**Author:** ![michaelzhiluo](https://avatars.discourse-cdn.com/v4/letter/m/41988e/32.png) [@michaelzhiluo](https://discuss.ray.io/u/michaelzhiluo)\
**Post date:** [June 7, 2021, 7:59pm UTC](https://discuss.ray.io/t/parallel-workers-compute-action/2421/2 "2021-06-07T19:59:12Z")

</div>

You can do custom evaluation with the `evaluation_config: {}` hyperparameter in `trainer.py`: [ray.rllib.agents.trainer — Ray 0.7.1 documentation](https://docs.ray.io/en/releases-0.7.1/_modules/ray/rllib/agents/trainer.html).

In addition, there are other hyperparameters such `evaluation_num_episodes` and `evaluation_num_workers`, which I think is particularly relevant in your case.

---

<div class="post-metadata">

**Author:** ![vishalrangras](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/vishalrangras/32/976_2.png) [@vishalrangras](https://discuss.ray.io/u/vishalrangras)\
**Post date:** [June 8, 2021, 4:23am UTC](https://discuss.ray.io/t/parallel-workers-compute-action/2421/3 "2021-06-08T04:23:24Z")

</div>

Thanks for your reply. However, I get “This page does not exist yet” error when I click on the doc. Should this hyperparameter work without using ray tune, as I described in my question? Also, how do I pass evaluation\_num\_workers param? In the env\_config dictionary?

I thought the evaluation\_num\_workers and evaluation\_num\_episodes are only used when we want to evaluate the policy after a batch or at the end of training process. As I mentioned, my requirement is to evaluate policy at later point in time by restoring the checkpoint manually. Will these hyperparameters work in such situation? Please share whatever insights you can.

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [June 11, 2021, 10:04am UTC](https://discuss.ray.io/t/parallel-workers-compute-action/2421/4 "2021-06-11T10:04:01Z")

</div>

Hey @vishalrangras , correct: `evaluation_num_workers` and `evaluation_num_episodes` is only relevant if you have `evaluation_interval` set to some int (not None!) and you would like evaluation to automatically happen every `evaluation_interval` train iterations.

However, you can still call `trainer._evaluate()` (`trainer.evaluate()`) manually if you’d like, even after restoring an agent from a checkpoint.

`evaluation_num_workers` goes into the main RLlib config, the one you use for everything else as well. See: `ray.rllib.agents.trainer.py` for all default config options.

---

<div class="post-metadata">

**Author:** ![vishalrangras](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/vishalrangras/32/976_2.png) [@vishalrangras](https://discuss.ray.io/u/vishalrangras)\
**Post date:** [June 12, 2021, 4:18pm UTC](https://discuss.ray.io/t/parallel-workers-compute-action/2421/5 "2021-06-12T16:18:24Z")

</div>

Thanks @sven1977 for your reply, this seems to be helpful for me. One more thing I need help with is, my environment class is designed in such a way that, for evaluation, I step into the environment one step at a time, then I call some helper methods from environment class to compute the evaluation metrics I need, add them into the data structure, and use them later to plot graphs. I am not using custom\_callback\_metrics feature of Ray for evaluation purposes, using it only for training related tensorboard logs.

I believe the trainer.\_evaluate() would work like trainer.\_train() and would rollout my policy for an entire episode or few episodes. I believe it won’t provide me a fine-grained control where I can call my custom methods at every step to compute my evaluation metrics.

Is there a way that I can step into the environment one step at a time using env.step() while still using multiple cores for parallel rollouts?