# Vector Environment With A Single Worker **URL:** \ **Category:** RLlib\ **Created:** [June 6, 2023, 10:11am UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930 "2023-06-06T10:11:09Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Adam\_Price](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/adam_price/32/4546_2.png) [@Adam\_Price](https://discuss.ray.io/u/Adam_Price)\ **Post date:** [June 6, 2023, 10:11am UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/1 "2023-06-06T10:11:09Z")
Hello all, I am working with an environment with a large DNN in its step function. To reduce wall-clock time, I want to run parallel environments and pass batches to the DNN, as I don’t have enough VRAM to create many sub-environments (like how the standard vectorised environment works). Instead, I’d like to pass action vectors into a single instance of the environment so the DNN does not need to be cloned. Does anyone know how I could do this? ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [June 6, 2023, 10:22am UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/2 "2023-06-06T10:22:26Z")
Hello @Adam_Price, Welcome to the forumns. The setting you are looking for is `num_envs_per_worker` [https://docs.ray.io/en/latest/rllib/rllib-env.html#performance](https://docs.ray.io/en/latest/rllib/rllib-env.html#performance) ---
**Author:** ![Adam\_Price](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/adam_price/32/4546_2.png) [@Adam\_Price](https://discuss.ray.io/u/Adam_Price)\ **Post date:** [June 6, 2023, 10:48am UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/4 "2023-06-06T10:48:51Z")
Hi, Thank you, but I don’t think that this will work. This approach uses VectorEnv, which creates clones of the environment. As I said in the first post, I don’t want to create clones of the environment. ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [June 6, 2023, 11:27am UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/5 "2023-06-06T11:27:23Z")
I misinterpreted this statement > [@Adam\_Price](#): > > I want to run parallel environments So if I understand better now, you want to run 1 environment and 1 policy. What is there to vectorize or batch in that case? ---
**Author:** ![Adam\_Price](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/adam_price/32/4546_2.png) [@Adam\_Price](https://discuss.ray.io/u/Adam_Price)\ **Post date:** [June 6, 2023, 11:45am UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/6 "2023-06-06T11:45:59Z")
The Env uses a DNN in the step function, so I want to pass multiple observations through it each timestep, taking multiple actions. So the env would be parallel, but I would only need one instance of the DNN. ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [June 6, 2023, 12:17pm UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/7 "2023-06-06T12:17:17Z")
OK, I think the most straightforward way to do this would be to implement your own custom VectorEnv. Thus env will be one instance and you would handle the parallelism how ever you intend to. Combine this with num\_envs\_per\_worker to control how many Parallel simulations you want that environment to manage. The other option that I think would work is to treat it as a multiagent environment. Make each agent\_id env\_{0,…,n} and map all the agents to the same policy. It should handle envs being done fine but the main thing you will have to work out is how the multiagent env finishes. Once a subenv finishes you won’t be able to reset it until they are all done. Personally I would avoid this approach and work out the custom VectorEnv. ---
**Author:** ![Adam\_Price](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/adam_price/32/4546_2.png) [@Adam\_Price](https://discuss.ray.io/u/Adam_Price)\ **Post date:** [June 7, 2023, 12:34pm UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/8 "2023-06-07T12:34:24Z")
Thank you for the suggestions! I agree that writing a custom version of VectorEnv is probably the best way to go. Looking at the code, it looks like a pretty big rewrite, though. I have ended up bodging it a bit by overwriting the `training_step()` function of the algorithm I’m using. I just call my custom data collector instead of collecting samples using the workers. ---
**Author:** ![TooSchoolForCool](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/tooschoolforcool/32/5903_2.png) [@TooSchoolForCool](https://discuss.ray.io/u/TooSchoolForCool)\ **Post date:** [March 27, 2024, 9:37am UTC](https://discuss.ray.io/t/vector-environment-with-a-single-worker/10930/9 "2024-03-27T09:37:58Z")
I have implemented a custom VectorEnv that takes a batch of actions and returns a batch of observations via one instance of DNN. Here’s a pseudo code of my env: ```python class MyVecEnv(VectorEnv): def vector_step(self, actions): # obs.shape == (num_envs, obs_space_dim) # actions.shape == (num_envs, act_space_dim) obs = self.dnn(actions) # ... return obs, ... ``` I train a PPO model for this `MyVecEnv` via ray Tuner with `num_envs = 16`. However, I found that performances and results differ greatly when `num_envs_per_worker` is varied. The best performance is reached when `num_envs_per_worker == num_envs`. You mentioned: “Combine this with num\_envs\_per\_worker to control how many Parallel simulations you want that environment to manage.” Does this indicate that we have to set `num_envs_per_worker` equal to `num_envs`?