I have an environment where it is important to eliminate any side effects between workers.
In particular, my environment runs benchmarks that are dependent on the current cache pressure, so I don’t want to mix different workers (each running a benchmark) up.
Is it possible to set CPU Affinities (assign worker processes to fixed HW threads) in the current version of RLlib?
If not, where would be the best place to start to implement this feature?
Many Thanks!
Hi @alexanderb14 ,
When running your experiments via Ray tune (tune.run()) in non-local mode, RLlib makes sure that all RolloutWorkers get their own CPU, so there shouldn’t be any interference into a Worker’s environment coming from another Worker (running in its own env).
Hi @sven1977
Thanks for your fast response! However I think this isn’t a solution for my usecase.
I need one worker to run on a specific HW thread that I set.
Background is that each worker shall run in a thread of an own QoS unit of the CPU (a Ryzen Threadripper), in order to not interfer on the L3 cache. Each such QoS unit has a own dedicated L3 cache.
Hmm, yeah, we do not support binding a worker or the driver to a particular HW thread.
However, could you also ask this question under the Ray Core category? You may get a more detailed answer there.
Hi Sven, thanks a lot for your response. I created a new discussion thread now, tagged with Ray Core: CPU Affinity for Ray Workers
Hi @sven1977,
When running your experiments via Ray tune (
tune.run()) in non-local mode, RLlib makes sure that all RolloutWorkers get their own CPU
Does that mean that if I’m using ‘trainer.train()’ RLlib will not make sure of that?