Example code failed---multi_agent_two_trainers.py

SS_D · March 20, 2024, 3:40pm

High: It blocks me to complete my task.

When attempting to run the example script multi_agent_two_trainers.py from ray/rllib/examples, I encountered a TypeError indicating an unexpected keyword argument _enable_new_api_stack in the AlgorithmConfig.experimental() method.
After commenting out the line involving _enable_new_api_stack=False, I encountered another error during the execution, related to a mismatch in the state dictionary keys when trying to load weights for the FullyConnectedNetwork_as_DQNTorchModel, suggesting missing keys and unexpected keys in the state dict.

part modified in original file

ppo_config = (
    PPOConfig()
    # .experimental(_enable_new_api_stack=False)  ### First try without change
    # .experimental(  )  ### Second try without param setting inside

source code

github.com

ray-project/ray/blob/master/rllib/examples/multi_agent_two_trainers.py

"""Example of using two different training methods at once in multi-agent.

Here we create a number of CartPole agents, some of which are trained with
DQN, and some of which are trained with PPO. We periodically sync weights
between the two algorithms (note that no such syncing is needed when using just
a single training method).

For a simpler example, see also: multiagent_cartpole.py
"""
# TODO (Kourosh): Migrate this example to the RLModule API.
import argparse

import gymnasium as gym
import os

import ray
from ray.rllib.algorithms.dqn import DQNConfig, DQNTFPolicy, DQNTorchPolicy
from ray.rllib.algorithms.ppo import (
    PPOConfig,
    PPOTF1Policy,

This file has been truncated. show original

Versions / Dependencies

Name: torch
Version: 2.1.0

Name: ray
Version: 2.7.1

Python 3.11.4

Topic		Replies	Views
Experimental() got an unexpected keyword argument '_enable_new_api_stack' RLlib	2	229	January 24, 2024
RLlib + PPO -> Value Error: Expected parameter loc Configure Algorithm, Training, Evaluation, Scaling	1	366	February 24, 2024
[RLlib] Error! TypeError: create_policy_mapping_fn.<locals>.mapping_fn() got an unexpected keyword argument 'worker' RLlib	0	363	June 21, 2023
NotImplementedError: Unsupported args Debugging and performance tuning	0	39	September 13, 2024
RayTaskError(AttributeError) : ray::RolloutWorker.par_iter_next() RLlib	12	1388	February 21, 2022

Example code failed---multi_agent_two_trainers.py

part modified in original file

source code

Versions / Dependencies

Related topics