Requesting Guidance on External Simulator

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I have a C++ based simulator running on my local machine. I am communicating with said C++ based simulator through a python client program. The python client program requests information from the simulator, once the client has information it uploads it into a neo4j graph database. I can then create embeddings from said database which I could use for observations.

Here is my question:

Based on the sound of this (C++ simulator which acts as a server that I communicate to via a python client, and I represent the environment through a neo4j graph), which path of environment choice would be best for me?

After browsing RLlib, it seems that I should use an external environment. However, as I understand it the server in the external environment relationship would essentially be what my client (interface to my simulator) is right now. Or maybe I am misunderstanding. Perhaps the relationship would be:

ExternalEnv Server <-> Python Client <-> C++ Simulator Server ?

I appreciate any guidance, assistance, etc.

(note, I can also change the c++ simulator to act as a client, though that would change the way I am currently representing the environment. So if you do suggest that please provide discussion on how to represent my environment to the agent, since it won’t be able to “see” inside of the simulator, only receive information as requested. Which is why I have constructed a graph.)

Thank you!

@Lars_Simon_Zehnder is also dealing with an external C++ based simulator and has already added a question to the [office hours docs] (RLlib Office Hours - Google Docs) for the next office hours. Please consider registering your question and joining the session so that we can pool topics a little bit!


Your understanding is accurate.
For your setup, you can imagine your client acting as a controller, coordinate data flowing between simulator server and RLlib server.
Essentially, your client would get obs from C++ simulator / graph_db, send it to RLlib server, get some actions back, feed the actions back to the C++ server, and do this repeatedly.
Would this work for you?