How to vectorize RLlib MultiAgentEnv similar to Gymnasium's VectorEnv?

Hello all! I am attempting to use an RLlib MultiAgentEnv-based environment outside of RLlib for prototyping before I work in RLlib, and I was wondering if it was possible to vectorize environments similar to Gymnasium’s VectorEnv? Although MultiAgentEnv is based on Gym Env, VectorEnv doesn’t support it due to the dictionary-based action space. Does RLlib something have something similar that does support the dictionary action spaces?

The closest thing I could find was this in the GitHub: However I’m not quite sure how to use it as there are no examples.