Issue when trying to measure CPU time with ray and psutil

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi,

I have a ray function in which I need to get the CPU time. I want to use psutil for that. When calling cpu_times I get the following error:

022-06-14 11:12:19,522 ERROR serialization.py:311 – Failed to unpickle serialized exception
Traceback (most recent call last):
File “/Users/Elias/Desktop/PhD Code/torero_selector/venv/lib/python3.8/site-packages/ray/exceptions.py”, line 38, in from_ray_exception
return pickle.loads(ray_exception.serialized_exception)
TypeError: init() missing 1 required positional argument: ‘pid’
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/Users/Elias/Desktop/PhD Code/torero_selector/venv/lib/python3.8/site-packages/ray/serialization.py”, line 309, in deserialize_objects
obj = self._deserialize_object(data, metadata, object_ref)
File “/Users/Elias/Desktop/PhD Code/torero_selector/venv/lib/python3.8/site-packages/ray/serialization.py”, line 238, in _deserialize_object
return RayError.from_bytes(obj)
File “/Users/Elias/Desktop/PhD Code/torero_selector/venv/lib/python3.8/site-packages/ray/exceptions.py”, line 32, in from_bytes
return RayError.from_ray_exception(ray_exception)
File “/Users/Elias/Desktop/PhD Code/torero_selector/venv/lib/python3.8/site-packages/ray/exceptions.py”, line 41, in from_ray_exception
raise RuntimeError(msg) from e
RuntimeError: Failed to unpickle serialized exception

Funny enough this does not occur when only calling .pid.

Any ideas on how to work around this?

Code to reproduce this:

import ray
import psutil


ray.init()

@ray.remote
def f(x):
    for proc in psutil.process_iter():
        processName = proc.name()
        if processName in ["Python"]:
            print(proc.pid)
            print(proc.cpu_times())
    return x



result_ids = []
for i in range(4):
    result_ids.append(f.remote(i))

results = ray.get(result_ids)

Thank you

2

TypeError: init () missing 1 required positional argument: ‘pid’

It seems like there is some error in your code. Do you mind try catch within the f and show what’s happening there:

@ray.remote
def f(x):
   try:
      for proc in psutil.process_iter():
          processName = proc.name()
          if processName in ["Python"]:
              print(proc.pid)
              print(proc.cpu_times())
      return x
   except Exception as e:
      print("ERROR", e)
3

That helped a lot. Thank you!