Best practice for using `get_action` and `log_action` together?

aviskarkc10 · August 19, 2021, 2:48am

I have been using get_action and log_action based on the comment here:

ray-project/ray/blob/master/rllib/examples/serving/cartpole_client.py#L34

    
      
          
          
In "remote" inference mode, the PolicyClient will send action requests to the
          server and not compute its own actions locally. The server then performs the
          inference forward pass and returns the action to the client.
          
          
In either case, the user of PolicyClient must:
          - Declare new episodes and finished episodes to the PolicyClient.
          - Log rewards to the PolicyClient.
          - Call `get_action` to receive an action from the PolicyClient (whether it'd be
            computed locally or remotely).
          - Besides `get_action`, the user may let the PolicyClient know about
            off-policy actions having been taken via `log_action`. This can be used in
            combination with `get_action`, but will only work, if the connected server
            runs an off-policy RL algorithm (such as DQN, SAC, or DDPG).
          """
          
          
import argparse
          import gym
          
          
from ray.rllib.env.policy_client import PolicyClient

When using these two functions together, should I be using two different episodes or is it all right if both are called within the same episode?

michaelzhiluo · August 19, 2021, 9:18am

They should be within the same episode and timestep. log_action takes action as input, which should be ideally retrieved via get_action.

Topic		Replies	Views
Best practice for training on policy and off policy action together? RLlib	4	359	September 27, 2021
What is the difference between `log_action` and `get_action` and when to use them? RLlib	13	746	August 5, 2021
Definition of "one timestep" when using ExternalEnv RLlib	4	366	June 23, 2022
External On-Policy Actions in PPO RLlib	3	659	June 23, 2021
Log multi agent rewards from policy_client RLlib	1	363	April 7, 2022

Best practice for using `get_action` and `log_action` together?

Related topics