Coud we use continuous action space for parametric action spaces

Shanchao_Yang · May 21, 2021, 2:17am

Hi, though the example below shows the usage of parametric action space for cartpole, I would like to ask that if we can use continous action space to produce the parametric action.

Instead of letting the policy network produce a latent action vector pi_t, and we obtain the discrete action dist dist = Discrete ( dot(pi_t, e) ), where e is the available action embedding, can we consider the pi_t is sampled from a continuous action dist (like gaussian), and we then dot it with all action embeddings? But this would introduce two action distribution (gaussian + discrete). And I am not sure it is correct or not.

Thanks for any advice!

github.com

ray-project/ray/blob/master/rllib/examples/parametric_actions_cartpole.py

"""Example of handling variable length and/or parametric action spaces.

This is a toy example of the action-embedding based approach for handling large
discrete action spaces (potentially infinite in size), similar to this:

    https://neuro.cs.ut.ee/the-use-of-embeddings-in-openai-five/

This currently works with RLlib's policy gradient style algorithms
(e.g., PG, PPO, IMPALA, A2C) and also DQN.

Note that since the model outputs now include "-inf" tf.float32.min
values, not all algorithm options are supported at the moment. For example,
algorithms might crash if they don't properly ignore the -inf action scores.
Working configurations are given below.
"""

import argparse
import os

import ray

This file has been truncated. show original

Topic		Replies	Views
Continuous action space and custom model RLlib	4	1371	July 17, 2021
Variable-length / Parametric Action Spaces RLlib	1	516	August 31, 2021
Does RLlib algorithm support both discrete and continuous action spaces simultaneously? RLlib	7	1526	February 22, 2023
Mixed continuous and discrete actions algorithm using deterministic RLlib	1	310	July 1, 2021
Example for action masking (without action embeddings) for tuple action space RLlib	2	667	October 27, 2021

Coud we use continuous action space for parametric action spaces

Related topics