MARWIL with gymnasium Dict as action Space

RLlib
8

This issue—where MARWIL appears to run but the dataset progress stays at 0.00 row/s—often indicates a problem with how the Parquet file encodes nested structures, especially with Dict action spaces. If the action column contains dicts with numpy arrays or non-JSON-serializable types, Ray Data may silently fail to read or parse the rows, resulting in no data being processed. This is a common pitfall when saving nested dicts with numpy arrays directly to Parquet, as Parquet expects flat, serializable data types (Ray Data Parquet limitations).

To resolve this, ensure that before writing to Parquet, all nested values in your action dicts are converted to native Python types (e.g., lists for Box, ints for Discrete), and avoid numpy arrays or objects. You can preprocess your data like this before saving:

def to_serializable_action(action):
    return {
        "rotate": int(action["rotate"]),
        "thrust": [float(x) for x in action["thrust"]]
    }

for row in data:
    row["actions"] = to_serializable_action(row["actions"])

Then write the dataset as before. This ensures Ray Data can read and batch the rows, and MARWIL will process them. Would you like more detail or a code snippet for re-writing your dataset?

Sources:

Hint: Mention @RunLLM in the post for followups.