Can't pass over custom_model_config to custom_model

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

Was expecting custom_model_config to be passed over to custom_model but it keeps giving me an error saying duplicated input:

from ray.rllib.algorithms.ppo import PPOConfig
from ray.tune.logger import pretty_print

from ray.rllib.models.torch.fcnet import FullyConnectedNetwork
from ray.rllib.models import ModelCatalog
ModelCatalog.register_custom_model("ray_torch_fcnet", FullyConnectedNetwork)

algo = (
            "custom_model": "ray_torch_fcnet",
            "custom_model_config": {'num_outputs':None},
            "vf_share_layers": True,

for i in range(10):
    result = algo.train()
    if i % 5 == 0:
        checkpoint_dir =
        print(f"Checkpoint saved in directory {checkpoint_dir}")

The error:

(RolloutWorker pid=32453) 2023-03-09 03:37:33,762       ERROR -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=32453, ip=, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f0e30887100>)
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/evaluation/", line 625, in __init__
(RolloutWorker pid=32453)     self._build_policy_map(
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/evaluation/", line 1899, in _build_policy_map
(RolloutWorker pid=32453)     self.policy_map.create_policy(
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/policy/", line 134, in create_policy
(RolloutWorker pid=32453)     policy = create_policy_for_framework(
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/utils/", line 116, in create_policy_for_framework
(RolloutWorker pid=32453)     return policy_class(observation_space, action_space, merged_config)
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/algorithms/ppo/", line 51, in __init__
(RolloutWorker pid=32453)     TorchPolicyV2.__init__(
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/policy/", line 84, in __init__
(RolloutWorker pid=32453)     model, dist_class = self._init_model_and_dist_class()
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/policy/", line 452, in _init_model_and_dist_class
(RolloutWorker pid=32453)     model = ModelCatalog.get_model_v2(
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/models/", line 648, in get_model_v2
(RolloutWorker pid=32453)     raise e
(RolloutWorker pid=32453)   File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/site-packages/ray/rllib/models/", line 622, in get_model_v2
(RolloutWorker pid=32453)     instance = model_cls(
(RolloutWorker pid=32453) TypeError: __init__() got multiple values for argument 'num_outputs'

Hi @zhh210,

The custom model config dictionary is passed in as kwargs to the model class init dunder. But that method already has a positional argument named num_outputs. This results in an argument with the same name passed twice which causes the error you see. You will need to rename your custom argument to have a unique name.

thanks @mannyv, what if I want to override the num_outputs in the fcnet model? Do I have to redefine a customized model that copies the built-in fcnet model?

Usually that is based on your hidden sizes and your action space so you would adjust one. of those but yes in general you could create a custom model and ignore the parameter.