The tune.Tuner.fit is not using GPU with 'num_gpu=1' setting

RLlib Configure Algorithm, Training, Evaluation, Scaling
1

I am using the following PPOConfig:

config = ppo.PPOConfig()
	.environment(env_class, env_config=self.env_config)
	.framework("torch")
	.rollouts(
		create_env_on_local_worker=True,
		batch_mode="complete_episodes",
	)
	.training(
		lr=1e-4,
		entropy_coeff=0.01,
		train_batch_size=150,
		sgd_minibatch_size=10,
	)
	.resources(num_gpus=1)
	.multi_agent(
		policies=self.agent.get_policy_list(),
		policies_to_train=list(self.agent.get_policy_list().keys()),
		policy_mapping_fn=self.agent.get_policy_mapping_fn(),
	)

with the above config, I am trying to train the policy using the tune.Tuner in my local machine which has Nvidia GeForce RTX 3060 GPU with the following code:

tuner = tune.Tuner(
            "PPO",
            param_space=self.config.to_dict(),
            run_config=air.RunConfig(
                stop=stop_criteria,
                verbose=1,
                checkpoint_config=air.CheckpointConfig(
                    checkpoint_frequency=20, checkpoint_at_end=True
                ),
            ),
        )
tuner.fit()

From the logs, I can see that 3 workers were started in CPUs, but not GPUs as specified in the config using .resources(num_gpus=1).

I verified that the correct version of cuda and PyTorch are installed and RLLlib is detecting using the following code snippet:

assert torch.cuda.is_available()

Am I missing anything here? I cannot understand why the Tuner is not launching in GPU.

2

I resolved this by removing the ray.init(num_gpus=1) call at the start of the code. As suggested in [RLlib] Can not allocate GPU resource · Issue #38060 · ray-project/ray · GitHub.