I ran ARS on Cartpole-v0 with the recommended configurations (ray/cartpole-ars.yaml at master · ray-project/ray · GitHub) and noticed that when increasing the number of workers the performance degrades. I ran with 2 workers, and 30 workers.
Intuitively adding more parallel workers should not temper with performance. However, I might have misunderstood the usage of num_workers here. Is this expected ?
If logs are requested, I can attach them. I dont want to spam the text field for now 