Errno 2 due to space in 'Program Files'

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi, I’ve been trying to use Ray Tune for the first time, mostly just following the example here, though applying it to my own model. When I try to run it, I get:

(pid=) C:\Program Files\Python39\python.exe: can't open file 'C:\Program': [Errno 2] No such file or directory
(pid=) 2022-07-01 13:50:15,380  INFO context.py:67 -- Exec'ing worker with command: "C:\Program Files\Python39\python.exe" C:\Program Files\Python39\lib\site-packages\ray\workers/default_worker.py --node-ip-address=127.0.0.1 --node-manager-port=1085 --object-store-name=tcp://127.0.0.1:22143 --raylet-name=tcp://127.0.0.1:25913 --redis-address=None --storage=None --temp-dir=C:\Users\jake_\AppData\Local\Temp\ray --metrics-agent-port=50667 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=127.0.0.1:49716 --redis-password=5241590000000000 --startup-token=24 --runtime-env-hash=1670958544
(pid=) [2022-07-01 13:51:14,763 E 33056 8528] (raylet.exe) worker_pool.cc:502: Some workers of the worker process(48816) have not registered within the timeout. The process is dead, probably it crashed during start.

I can’t figure out what I’ve done that could’ve caused this issue. I’ve also tried running the example exactly as shown and I get the same error. As most people have a space in ‘Program Files’ and I can’t seem to find anyone else with the same issue, I assume it’s some other issue with my setup, but I can’t figure out what it could be.

I’d really appreciate any help at all on this, so thanks in advance!

2

@Jake_Martin
It seems that Ray is not working properly in your environment.
Could you just open a python shell and do:
import ray; ray.init(); ray.cluster_resource()?

Just to make sure the basics are right. Thanks!

3

Thanks! Here’s the output:

>>> import ray
>>> ray.init()
RayContext(dashboard_url='', python_version='3.9.5', ray_version='1.13.0', ray_commit='e4ce38d001dbbe09cd21c497fedd03d692b2be3e', address_info={'node_ip_address': '127.0.0.1', 'raylet_ip_address': '127.0.0.1', 'redis_address': None, 'object_store_address': 'tcp://127.0.0.1:50980', 'raylet_socket_name': 'tcp://127.0.0.1:38929', 'webui_url': '', 'session_dir': 'C:\\Users\\jake_\\AppData\\Local\\Temp\\ray\\session_2022-07-02_00-05-27_234872_17396', 'metrics_export_port': 57018, 'gcs_address': '127.0.0.1:38457', 'address': '127.0.0.1:38457', 'node_id': '7ba721bfc63368cdb6763264ba8a9b9c7b2675ebf8488410323b46eb'})
>>> ray.cluster_resources()
{'node:127.0.0.1': 1.0, 'object_store_memory': 5504618496.0, 'GPU': 1.0, 'CPU': 24.0, 'memory': 11009236992.0}

I don’t think there’s an issue there? I tried uninstalling and reinstalling Ray too, but that didn’t fix the issue.

4

Thanks for trying it out. So you are right, local Ray Cluster seems fine.

Could you help me try one more thing? Can you try the following script?

import ray

@ray.remote
def f():
	print("Hello World!")

ray.get(f.remote())

Can you get similar result like this?

(ray2)  ✘ xwjiang@xw  ~/ray   session3  python tryout_remote_func.py
2022-07-02 08:10:06,093	INFO services.py:1483 -- View the Ray dashboard at http://127.0.0.1:8265
(f pid=29043) Hello World!

Just want to make sure that basic executing of remote function is working fine. Thanks!

5

It seems to work, though I don’t get quite the same output? I get:

> python tryout_remote_func.py
(f pid=47320) Hello World!
6

If anyone has any suggestion on how I could fix this, I’d really appreciate it. The issue is blocking me from completing my dissertation.

7

Update: A completely fresh install of Python (and all packages) solved the issue.

1 Like
8

@Jake_Martin
In your case, your python path has a space. Right now, my python path also has a space but raylet detects it correctly. The problem is with my virtual environment path which also has a space. raylet isn’t able to handle this correctly.

(raylet) [C:\Users\Chukwudi](file:///C:/Users/Chukwudi) Ajoku\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\python.exe: can't open file 'c:\\Users\\Chukwudi\\ Ajoku\\Projects\\llm-applications\\.venv\\Lib\\site-packages\\ray\\_private\\workers\\default_worker.py': [Errno 2] No such file or directory

As shown above, my username has a space and raylet is misinterpreting it.

Can someone help me? Even a fresh install of python works not.