Unity ml-agents integration

RLlib
1

Hello,

I am in the process of developing a project where I want to set Unity3D as the external simulation environment and use RLlib for training a RL algorithm, following a client-server approach.

I have been able to export the tf model and convert it to .onnx format, but still I cannot manage to use it as the brain inside Unity in order to run inference with ML-agents. I suspect that the problem is that ML-agents expects a specific naming convention for input and output tensor names. Is there a way to set the model layer names inside RLlib with the desired conventions? Any other possible suggestions?

Thank you in advance,
Matthew

1 Like
2

Hey @mattkmp , great question. Iā€™m actually not sure either what the exact requirements are for importing ONNX models as an ML-Agents brain. Could you find out from the Unity side what these conditions are? If the layers require certain tf-names, you could write your own custom model based on our ray.rllib.models.tf.fcnet.py::FullyConnectedNetwork and have RLlib use that instead via:

config:
    model:
        custom_model: [your custom model class]