I’m using RLlib DQN to train a model on a custom simulator.
I’ve been playing around with it and can clearly see how using multiple num_workers and num_cpus_per_worker benefits since I can collect more data much faster. However, I cannot seem to find any benefit in increasing num_cpus_for_driver. There’s really no difference between setting it to 1 or 10 in my case. I’m using Tune.run to run the training, so I assume it should be in effect as suggested in the documentation: “Number of CPUs to allocate for the trainer. Note: this only takes effect when running in Tune. Otherwise, the trainer runs in the main program.”
What should I expect by increasing num_cpus_for_driver? Thanks in advance!
num_cpus_per_driver will be most important if your models are on CPU. If you are training models on CPUs, you will probably want to reserve some cores for the model-training part (the driver) to operate without thread thrashing.
1 Like