How to improve the utilization of gpu by change the trainer config

RLlib
1

i know ppo cost most time in sample collected, so i need more cpu kernel.
so how to edit the config make my train more quick by the machine right now？
my config:

    config = {
        "batch_mode": "truncate_episodes",
        "num_envs_per_worker": 1,
        "num_gpus": 1,
        "num_sgd_iter": 10,
        "num_workers": 16,
        "observation_filter": "NoFilter",
        "sgd_minibatch_size": 500,
        "train_batch_size": 10000,
        # "lr": 0.0001,

        "lambda": 0.95,
        "clip_param": 0.1,
        "entropy_coeff": 0.01,
        "env": CalendarSpreadArbitrage,
        "env_config": {
            "symbol": symbol,
            "commission": commission,
            "period": period,
            "train": True
        },
    }

    trainer = ppo.PPOTrainer(config=config)

gpu info:

微信截图_20210805172013
微信截图_202108051720131877×442 197 KB

2

It is better to use gpustat to check on GPU utilization: GitHub - wookayin/gpustat: 📊 A simple command-line utility for querying and monitoring GPU status

If you keep CPU constant, you can utilize GPU more often if you increase num_sgd_iter (more gradient steps) and decrease sgd_minibatch_size. You can also increase data collection per worker by increasing num_envs_per_worker, which represents how many environments there are per worker (vectorized).

1 Like
3

Thank you

I’ll try it out.