1. Severity of the issue: (select one)
None: I’m just curious or want clarification.
Low: Minor issue or workaround available.
Medium: Intermittent failures or limited functionality.
High: Blocking functionality or affecting production.
2. Environment:
- Ray version: 2.41.0
- Python version: 3.11
- OS: Linux
- Other relevant libraries/tools:
orbax(used for checkpointing)uvloop(used with Ray serve)nest_asyncio(used internally by Orbax)
3. Description of the issue:
When using Ray Serve in combination with Orbax checkpointers, there’s a conflict between nest_asyncio (used by Orbax) and uvloop (used by Ray Serve) when attempting to save a checkpoint.
This issue does not occur with default Ray execution (i.e., outside of Serve), where checkpointing works as expected. The incompatibility appears to be related to the event loop, with uvloop and nest_asyncio being mutually incompatible in some scenarios.
4. Question / Feature Request:
Is there a flag or configuration option to disable uvloop in Ray Serve or Ray in general, for only some parts of the code, to allow compatibility with libraries like Orbax that rely on nest_asyncio?
Alternatively, are there any recommended workarounds or configurations for using Orbax checkpointing with Ray Serve that avoid this conflict?
Thank you for your help!!