Timeline does not match my script

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I run a script with ray.timeline() with ray 1.13.0. When I visualized the JSON file it doesn’t match my script.

import ray
import ray._private.profiling as profiling

import os

os.environ['RAY_PROFILING']='1'

ray.init(num_cpus=1)

@ray.remote
def f(x):
    with profiling.profile("compute", extra_data={'key': 'val'}):
        r=x*x
    return r
with profiling.profile("remote", extra_data={'key': 'val'}):
    future = f.remote(2.0)
with profiling.profile("get"):
    value = ray.get(future)
print(value)
y=ray.put('xxx')
print(ray.get(y))
import time
time.sleep(1)
ray.timeline(filename="./ray_timeline.json")

there is the timeline shown in chrome.

image
image1494×131 11.5 KB

I try to figure out where or who gives fetch_and_run_function. Such as submit_task and task::f() I can understand the meanings.

I profiled get and remote and I can’t find them anywhere. I reviewed the ray core’s code. there is also a profile about ray.get but it doesn’t show in the timeline.

2

fetch_and_run_function are internal events that refer to the Python working starting up and fetching any function definitions.

I’m not sure why get and remote don’t appear. ray.get may not be appearing just because it’s at the end of the application (profile collection is asynchronous). You could try running this program in a loop multiple times to see if that collects more of the profile. Otherwise, it’s most likely a bug: please file a GitHub issue so we can get it resolved!

3

There is another topic mentioned the profile collection should wait for a while. I follow the solution which adds wait at the end of the script.

I also use ray.put and there is no output in the timeline. It confused me. I use Ray 1.13.0.

I try to run this script in a loop 5 times and there is no output about remote or get or put. I have filed an issue on GitHub.