I’m trying to measure model inference times. To do so, I have trained some Pong-v0 models with PPO and using a GPU for the training process. Later I have modified the rollout.py script to include timers in order to get information about the inference time. But when I try different configurations for the rollout (as for example increasing or reducing the number of workers or enabling/disabling the GPU I get that this measured time remains almost the same in all cases.
Even when I specify more resources than available for the workers, a message appears giving me information about the imposibility of assigning these resources but rollout continues without problem. So, my cuestion is: When you change resources for rollout (such as the number of GPUs for each worker), does it make sense that times reamins unchanged? Is the script running these rollouts using the resources specified?
When I measure time I refer to these change in rollout.py script:
###############
t0= time.time()
################
if p_use_lstm:
a_action, p_state, _ = agent.compute_action(
a_obs,
state=agent_states[agent_id],
prev_action=prev_actions[agent_id],
prev_reward=prev_rewards[agent_id],
policy_id=policy_id)
agent_states[agent_id] = p_state
else:
a_action = agent.compute_action(
a_obs,
prev_action=prev_actions[agent_id],
prev_reward=prev_rewards[agent_id],
policy_id=policy_id)
########################
t1 = time.time()
model_times_this_episode.append(t1-t0)
this_episode_time += (t1-t0)
########################
Thanks in advance foryour help!
P.D: I’m new to Ray and RLlib (I’m using irt for my bachelor’s final thesis) and maybe there are some concepts that I’m missing or not understanding accurately.