Does rllib support multi-gpu plus multi-cpu training?

If I have two gpu machines and one CPU machine, how can I use the gpu of two nodes and the cpu of three nodes?
If ppo is used, only one gpu can be used; If use ddppo, the CPU node is not used.