Hi, I would like to know ray offers a way to create an on-demand profiler to monitor function call durations in Ray. It can be toggled on and off during runtime.
Ray does not provide a built-in, public API for an on-demand profiler that can be toggled at runtime to monitor function call durations. However, you can use the Ray Dashboard to profile worker processes on demand (e.g., via “Stack Trace” or “CPU Flame Graph” actions), but this is done through the web UI, not programmatically within your code. For code-level profiling, you can use Python’s cProfile or Ray’s private profiling API, but toggling profiling on/off at runtime is not natively supported as a public feature. The private API (ray._private.profiling.profile) can be used as a context manager, but it is not guaranteed to be stable and may be deprecated without warning in future releases. See Ray Dashboard profiling guide and discussion on custom events.
Would you like more detail or code examples for using cProfile or the private profiling API?
Sources:
- Ray Dashboard profiling guide
- Ray Timeline custom events discussion
- ray._private.profiling.py (private API)
Hint: Mention @RunLLM in the post for followups.