How severe does this issue affect your experience of using Ray?
- Medium: It contributes to significant difficulty to complete my task, but I can work around it.
It set up a client server training setup that follows the cartpole client/server example. The environment is a multi agent setup and therefore uses a policy mapping function. When i run the client and the server setup on the same machine everthing works perfectly for remote and local action inference.
Problem:
When i run the client on another machine it works in remote inference mode but not in local mode. I get:
KeyError: “policy_mapping_fn returned invalid policy id ‘None’!”
My policy function looks like this:
def policy_mapping_fn(agent_id, episode, worker, **kwargs):
if agent_id.startswith("p_d_"):
return "pot_decider_policy"
elif agent_id.startswith("p_m_"):
return "pot_move_policy"
Theoretically it is possible that it returns None (if the agent_id isn’t matching the patterns in my if statements) but when i add an additional else statement to my mapping function to catch the agent_id that caused the problem i get an agent_id that should actually match my if-statements…
Debugging attempt:
def policy_mapping_fn(agent_id, episode, worker, **kwargs):
if agent_id.startswith("p_d_"):
return "pot_decider_policy"
elif agent_id.startswith("p_m_"):
return "pot_move_policy"
else:
print(agent_id)
Result:
The else statement prints “p_m_0_0” before the error is thrown.
I have no idea what could cause the issue here so if anyone could provide help or a hint where to look it would help me out a lot! Thanks in advance!