Best practice for using `get_action` and `log_action` together?

aviskarkc10 · August 19, 2021, 2:48am

I have been using get_action and log_action based on the comment here:

ray-project/ray/blob/master/rllib/examples/serving/cartpole_client.py#L34

    
      
          
          
In "remote" inference mode, the PolicyClient will send action requests to the
          server and not compute its own actions locally. The server then performs the
          inference forward pass and returns the action to the client.
          
          
In either case, the user of PolicyClient must:
          - Declare new episodes and finished episodes to the PolicyClient.
          - Log rewards to the PolicyClient.
          - Call `get_action` to receive an action from the PolicyClient (whether it'd be
            computed locally or remotely).
          - Besides `get_action`, the user may let the PolicyClient know about
            off-policy actions having been taken via `log_action`. This can be used in
            combination with `get_action`, but will only work, if the connected server
            runs an off-policy RL algorithm (such as DQN, SAC, or DDPG).
          """
          
          
import argparse
          import gym
          
          
from ray.rllib.env.policy_client import PolicyClient

When using these two functions together, should I be using two different episodes or is it all right if both are called within the same episode?

michaelzhiluo · August 19, 2021, 9:18am

They should be within the same episode and timestep. log_action takes action as input, which should be ideally retrieved via get_action.

Topic		Replies	Views
Best practice for training on policy and off policy action together? RLlib	4	298	September 27, 2021
What is the difference between `log_action` and `get_action` and when to use them? RLlib	13	556	August 5, 2021
PolicyClient should be agent-done smart RLlib	1	189	January 12, 2022
Definition of "one timestep" when using ExternalEnv RLlib	4	258	June 23, 2022
Query policy from within environment, without logging action? RLlib	4	204	September 27, 2022

Best practice for using `get_action` and `log_action` together?

Related Topics