Hi,
I am confused about the documentation and usage of num_aggregator_actors_per_learner
in the new API using PPO.
In the docstring it says “Must be at least 1” while the default is set to 0.
If I set it to 1, I get the following error:
ValueError: If episodes provided for training, Learner (<ray.train._internal.worker_group._WrappedExecutable object at 0x7fb75c49d1b0>) must have a LearnerConnector pipeline (but pipeline is None)!), taking actor 0 out of service.
Am I supposed to just keep the default?
If I would like to set a higher value am I supposed to provide a custom LearnerConnector
pipeline?
1. Severity of the issue: (select one)
Low: Annoying but doesn’t hinder my work.
2. Environment:
- Ray version: 2.44.1
- Python version: 3.10.15
- OS: Ubuntu 22.04
- Cloud/Infrastructure: AKS
- Other libs/tools (if relevant):
3. What happened vs. what you expected:
- Expected: PPO can make use of aggregator actors by default.
- Actual: Fails with an error telling me to provide a LearnerConnector.
Gihub Issue on this topic: [RLlib] Documentation of num_aggregator_actors_per_learner · Issue #51874 · ray-project/ray · GitHub