Failed to register worker to raylet (2)

RLlib
1

1. Severity of the issue: (select one)
None: I’m just curious or want clarification.
Low: Annoying but doesn’t hinder my work.
Medium: Significantly affects my productivity but can find a workaround.
[ x] High: Completely blocks me.

2. Environment:

  • Ray version: 2.46
  • Python version: 3.12
  • OS: windows
  • Cloud/Infrastructure: local
  • Other libs/tools (if relevant):

3. What happened vs. what you expected:

  • Expected:
  • Actual:

Running GitHub PopGym which uses ray. when I run it local after downloading the repo, I get this issue.

2025-06-18 00:27:37,494 ERROR services.py:1362 – Failed to start the dashboard , return code 3221226505
2025-06-18 00:27:37,494 ERROR services.py:1387 – Error should be written to ‘dashboard.log’ or ‘dashboard.err’. We are printing the last 20 lines for you.
2025-06-18 00:27:37,511 ERROR services.py:1431 –
The last 20 lines of C:\Users\billy\AppData\Local\Temp\ray\session_2025-06-18_00-27-30_852800_61304\logs\dashboard.log (it contains the error message from the dashboard):
Traceback (most recent call last):
File “C:\Users\billy\Desktop\PopGym2\popgym\env\Lib\site-packages\ray\dashboard\dashboard.py”, line 247, in
logging_utils.redirect_stdout_stderr_if_needed(
File “C:\Users\billy\Desktop\PopGym2\popgym\env\Lib\site-packages\ray_private\logging_utils.py”, line 47, in redirect_stdout_stderr_if_needed
sys.stdout = open_log(stdout_fileno, unbuffered=True, closefd=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\billy\Desktop\PopGym2\popgym\env\Lib\site-packages\ray_private\utils.py”, line 446, in open_log
stream = open(path, **kwargs)
^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 6] The handle is invalid

2025-06-18 00:27:38,164 INFO worker.py:1888 – Started a local Ray instance.
[2025-06-18 00:27:41,491 E 61304 57588] core_worker.cc:513: Failed to register worker to Raylet: IOError: [RayletClient] Unable to register worker with raylet. Unknown error worker_id=01000000ffffffffffffffffffffffffffffffffffffffffffffffff

Here is the raylet.err file:
[2025-06-18 00:17:21,258 I 66712 58584] (raylet.exe) main.cc:226: Setting cluster ID to: 9557188f2111b85c3447ef939f16a99cb1c1d4c74fda6cec28cc8100
[2025-06-18 00:17:21,262 I 66712 58584] (raylet.exe) main.cc:497: Raylet is not set to kill unknown children.
[2025-06-18 00:17:21,262 I 66712 58584] (raylet.exe) io_service_pool.cc:36: IOServicePool is running with 1 io_service.
[2025-06-18 00:17:21,263 I 66712 58584] (raylet.exe) main.cc:497: Setting node ID node_id=a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca
[2025-06-18 00:17:21,263 I 66712 58584] (raylet.exe) store_runner.cc:50: Allowing the Plasma store to use up to 10.6162GB of memory.
[2025-06-18 00:17:21,263 I 66712 58584] (raylet.exe) store_runner.cc:66: Starting object store with directory C:\Users\billy\AppData\Local\Temp, fallback C:\Users\billy\AppData\Local\Temp\ray\session_2025-06-18_00-17-13_539216_18736, and huge page support disabled
[2025-06-18 00:17:21,264 I 66712 68500] (raylet.exe) dlmalloc.cc:324: Setting dlmalloc config: plasma_directory=C:\Users\billy\AppData\Local\Temp, fallback_directory=C:\Users\billy\AppData\Local\Temp\ray\session_2025-06-18_00-17-13_539216_18736, hugepage_enabled=0, fallback_enabled=1
[2025-06-18 00:17:21,264 I 66712 68500] (raylet.exe) store.cc:583: Plasma store debug dump:
Current usage: 0 / 10.6162 GB

  • num bytes created total: 0
    0 pending objects of total size 0MB

  • objects spillable: 0

  • bytes spillable: 0

  • objects unsealed: 0

  • bytes unsealed: 0

  • objects in use: 0

  • bytes in use: 0

  • objects evictable: 0

  • bytes evictable: 0

  • objects created by worker: 0

  • bytes created by worker: 0

  • objects restored: 0

  • bytes restored: 0

  • objects received: 0

  • bytes received: 0

  • objects errored: 0

  • bytes errored: 0

[2025-06-18 00:17:21,269 I 66712 58584] (raylet.exe) grpc_server.cc:138: ObjectManager server started, listening on port 52163.
[2025-06-18 00:17:21,271 I 66712 58584] (raylet.exe) worker_killing_policy.cc:107: Running GroupByOwner policy.
[2025-06-18 00:17:21,271 W 66712 58584] (raylet.exe) memory_monitor.cc:68: Not running MemoryMonitor. It is currently supported only on Linux.
[2025-06-18 00:17:21,271 I 66712 58584] (raylet.exe) node_manager.cc:300: Initializing NodeManager node_id=a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca
[2025-06-18 00:17:21,271 I 66712 58584] (raylet.exe) grpc_server.cc:138: NodeManager server started, listening on port 52165.
[2025-06-18 00:17:21,277 I 66712 62472] (raylet.exe) agent_manager.cc:81: Monitor agent process with name dashboard_agent
[2025-06-18 00:17:21,280 I 66712 54696] (raylet.exe) agent_manager.cc:81: Monitor agent process with name runtime_env_agent
[2025-06-18 00:17:21,283 I 66712 58584] (raylet.exe) event.cc:500: Ray Event initialized for RAYLET
[2025-06-18 00:17:21,283 I 66712 58584] (raylet.exe) event.cc:331: Set ray event level to warning
[2025-06-18 00:17:21,284 I 66712 58584] (raylet.exe) raylet.cc:138: Raylet of id, a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca started. Raylet consists of node_manager and object_manager. node_manager address: 127.0.0.1:52165 object_manager address: 127.0.0.1:52163 hostname: William_Hoy
[2025-06-18 00:17:21,286 I 66712 58584] (raylet.exe) node_manager.cc:535: [state-dump] NodeManager:
[state-dump] Node ID: a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca
[state-dump] Node name: 127.0.0.1
[state-dump] InitialConfigResources: {GPU: 1, object_store_memory: 1.06162e+10, accelerator_type:G: 1, memory: 2.47711e+10, CPU: 32, node:internal_head: 1, node:127.0.0.1: 1}
[state-dump] ClusterTaskManager:
[state-dump] ========== Node: a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca =================
[state-dump] Infeasible queue length: 0
[state-dump] Schedule queue length: 0
[state-dump] Dispatch queue length: 0
[state-dump] num_waiting_for_resource: 0
[state-dump] num_waiting_for_plasma_memory: 0
[state-dump] num_waiting_for_remote_node_resources: 0
[state-dump] num_worker_not_started_by_job_config_not_exist: 0
[state-dump] num_worker_not_started_by_registration_timeout: 0
[state-dump] num_tasks_waiting_for_workers: 0
[state-dump] num_cancelled_tasks: 0
[state-dump] cluster_resource_scheduler state:
[state-dump] Local id: 3561657192649176863 Local resources: {“total”:{object_store_memory: [106161819640000], GPU: [10000], memory: [247710912520000], accelerator_type:G: [10000], node:internal_head: [10000], CPU: [320000], node:127.0.0.1: [10000]}}, “available”: {GPU: [10000], object_store_memory: [106161819640000], accelerator_type:G: [10000], memory: [247710912520000], CPU: [320000], node:internal_head: [10000], node:127.0.0.1: [10000]}}, “labels”: is_draining: 0 is_idle: 1 Cluster resources (at most 20 nodes are shown): node id: 3561657192649176863{“total”:{object_store_memory: 106161819640000, accelerator_type:G: 10000, GPU: 10000, memory: 247710912520000, node:internal_head: 10000, node:127.0.0.1: 10000, CPU: 320000}}, “available”: {object_store_memory: 106161819640000, accelerator_type:G: 10000, memory: 247710912520000, GPU: 10000, node:internal_head: 10000, node:127.0.0.1: 10000, CPU: 320000}}, “labels”:{“ray.io/node_id":"a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca”,}, “is_draining”: 0, “draining_deadline_timestamp_ms”: -1} { “placement group locations”: , “node to bundles”: }
[state-dump] Waiting tasks size: 0
[state-dump] Number of executing tasks: 0
[state-dump] Number of pinned task arguments: 0
[state-dump] Number of total spilled tasks: 0
[state-dump] Number of spilled waiting tasks: 0
[state-dump] Number of spilled unschedulable tasks: 0
[state-dump] Resource usage {
[state-dump] }
[state-dump] Backlog Size per scheduling descriptor :{workerId: num backlogs}:
[state-dump]
[state-dump] Running tasks by scheduling class:
[state-dump] ==================================================
[state-dump]
[state-dump] ClusterResources:
[state-dump] LocalObjectManager:
[state-dump] - num pinned objects: 0
[state-dump] - pinned objects size: 0
[state-dump] - num objects pending restore: 0
[state-dump] - num objects pending spill: 0
[state-dump] - num bytes pending spill: 0
[state-dump] - num bytes currently spilled: 0
[state-dump] - cumulative spill requests: 0
[state-dump] - cumulative restore requests: 0
[state-dump] - spilled objects pending delete: 0
[state-dump]
[state-dump] ObjectManager:
[state-dump] - num local objects: 0
[state-dump] - num unfulfilled push requests: 0
[state-dump] - num object pull requests: 0
[state-dump] - num chunks received total: 0
[state-dump] - num chunks received failed (all): 0
[state-dump] - num chunks received failed / cancelled: 0
[state-dump] - num chunks received failed / plasma error: 0
[state-dump] Event stats:
[state-dump] Global stats: 0 total (0 active)
[state-dump] Queueing time: mean = -nan(ind) s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] Execution time: mean = -nan(ind) s, total = 0.000 s
[state-dump] Event stats:
[state-dump] PushManager:
[state-dump] - num pushes in flight: 0
[state-dump] - num chunks in flight: 0
[state-dump] - num chunks remaining: 0
[state-dump] - max chunks allowed: 409
[state-dump] OwnershipBasedObjectDirectory:
[state-dump] - num listeners: 0
[state-dump] - cumulative location updates: 0
[state-dump] - num location updates per second: 1173811313216000.000
[state-dump] - num location lookups per second: 0.000
[state-dump] - num locations added per second: 0.000
[state-dump] - num locations removed per second: 1173811313216000.000
[state-dump] BufferPool:
[state-dump] - create buffer state map size: 0
[state-dump] PullManager:
[state-dump] - num bytes available for pulled objects: 10616181964
[state-dump] - num bytes being pulled (all): 0
[state-dump] - num bytes being pulled / pinned: 0
[state-dump] - get request bundles: BundlePullRequestQueue{0 total, 0 active, 0 inactive, 0 unpullable}
[state-dump] - wait request bundles: BundlePullRequestQueue{0 total, 0 active, 0 inactive, 0 unpullable}
[state-dump] - task request bundles: BundlePullRequestQueue{0 total, 0 active, 0 inactive, 0 unpullable}
[state-dump] - first get request bundle: N/A
[state-dump] - first wait request bundle: N/A
[state-dump] - first task request bundle: N/A
[state-dump] - num objects queued: 0
[state-dump] - num objects actively pulled (all): 0
[state-dump] - num objects actively pulled / pinned: 0
[state-dump] - num bundles being pulled: 0
[state-dump] - num pull retries: 0
[state-dump] - max timeout seconds: 0
[state-dump] - max timeout request is already processed. No entry.
[state-dump]
[state-dump] WorkerPool:
[state-dump] - registered jobs: 0
[state-dump] - process_failed_job_config_missing: 0
[state-dump] - process_failed_rate_limited: 0
[state-dump] - process_failed_pending_registration: 0
[state-dump] - process_failed_runtime_env_setup_failed: 0
[state-dump] - num PYTHON workers: 0
[state-dump] - num PYTHON drivers: 0
[state-dump] - num PYTHON pending start requests: 0
[state-dump] - num PYTHON pending registration requests: 0
[state-dump] - num object spill callbacks queued: 0
[state-dump] - num object restore queued: 0
[state-dump] - num util functions queued: 0
[state-dump] - num idle workers: 0
[state-dump] TaskDependencyManager:
[state-dump] - task deps map size: 0
[state-dump] - get req map size: 0
[state-dump] - wait req map size: 0
[state-dump] - local objects map size: 0
[state-dump] WaitManager:
[state-dump] - num active wait requests: 0
[state-dump] Subscriber:
[state-dump] Channel WORKER_REF_REMOVED_CHANNEL
[state-dump] - cumulative subscribe requests: 0
[state-dump] - cumulative unsubscribe requests: 0
[state-dump] - active subscribed publishers: 0
[state-dump] - cumulative published messages: 0
[state-dump] - cumulative processed messages: 0
[state-dump] Channel WORKER_OBJECT_EVICTION
[state-dump] - cumulative subscribe requests: 0
[state-dump] - cumulative unsubscribe requests: 0
[state-dump] - active subscribed publishers: 0
[state-dump] - cumulative published messages: 0
[state-dump] - cumulative processed messages: 0
[state-dump] Channel WORKER_OBJECT_LOCATIONS_CHANNEL
[state-dump] - cumulative subscribe requests: 0
[state-dump] - cumulative unsubscribe requests: 0
[state-dump] - active subscribed publishers: 0
[state-dump] - cumulative published messages: 0
[state-dump] - cumulative processed messages: 0
[state-dump] num async plasma notifications: 0
[state-dump] Event stats:
[state-dump] Global stats: 27 total (13 active)
[state-dump] Queueing time: mean = 1.764 ms, max = 12.420 ms, min = 7.300 us, total = 47.637 ms
[state-dump] Execution time: mean = 926.489 us, total = 25.015 ms
[state-dump] Event stats:
[state-dump] PeriodicalRunner.RunFnPeriodically - 11 total (2 active, 1 running), Execution time: mean = 148.627 us, total = 1.635 ms, Queueing time: mean = 4.329 ms, max = 12.420 ms, min = 32.800 us, total = 47.620 ms
[state-dump] ClusterResourceManager.ResetRemoteNodeView - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] NodeManager.deadline_timer.spill_objects_when_over_threshold - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] NodeManager.GCTaskFailureReason - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] NodeManager.CheckForUnexpectedWorkerDisconnects - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] NodeManager.deadline_timer.debug_state_dump - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] ray::rpc::InternalKVGcsService.grpc_client.GetInternalConfig - 1 total (0 active), Execution time: mean = 687.800 us, total = 687.800 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] ray::rpc::InternalPubSubGcsService.grpc_client.GcsSubscriberCommandBatch - 1 total (0 active), Execution time: mean = 356.700 us, total = 356.700 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] NodeManager.deadline_timer.record_metrics - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] NodeManager.deadline_timer.flush_free_objects - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] RayletWorkerPool.deadline_timer.kill_idle_workers - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] ray::rpc::InternalKVGcsService.grpc_client.GetInternalConfig.OnReplyReceived - 1 total (0 active), Execution time: mean = 21.299 ms, total = 21.299 ms, Queueing time: mean = 10.500 us, max = 10.500 us, min = 10.500 us, total = 10.500 us
[state-dump] ray::rpc::NodeInfoGcsService.grpc_client.RegisterNode - 1 total (0 active), Execution time: mean = 756.300 us, total = 756.300 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] NodeManager.ScheduleAndDispatchTasks - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] ray::rpc::NodeInfoGcsService.grpc_client.RegisterNode.OnReplyReceived - 1 total (0 active), Execution time: mean = 280.200 us, total = 280.200 us, Queueing time: mean = 7.300 us, max = 7.300 us, min = 7.300 us, total = 7.300 us
[state-dump] ray::rpc::InternalPubSubGcsService.grpc_client.GcsSubscriberPoll - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] ray::rpc::InternalPubSubGcsService.grpc_client.GcsSubscriberCommandBatch.OnReplyReceived - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
[state-dump] DebugString() time ms: 0
[state-dump]
[state-dump]
[2025-06-18 00:17:21,286 I 66712 58584] (raylet.exe) accessor.cc:765: Received notification for node, IsAlive = 1 node_id=a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca
[2025-06-18 00:17:21,370 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 50020, the token is 0
[2025-06-18 00:17:21,372 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 64680, the token is 1
[2025-06-18 00:17:21,376 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 69896, the token is 2
[2025-06-18 00:17:21,379 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 34360, the token is 3
[2025-06-18 00:17:21,382 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 55808, the token is 4
[2025-06-18 00:17:21,386 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 66276, the token is 5
[2025-06-18 00:17:21,389 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 70812, the token is 6
[2025-06-18 00:17:21,394 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 68424, the token is 7
[2025-06-18 00:17:21,398 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 71616, the token is 8
[2025-06-18 00:17:21,402 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 68012, the token is 9
[2025-06-18 00:17:21,407 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 67412, the token is 10
[2025-06-18 00:17:21,411 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 67208, the token is 11
[2025-06-18 00:17:21,416 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 30120, the token is 12
[2025-06-18 00:17:21,421 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 70552, the token is 13
[2025-06-18 00:17:21,426 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 34612, the token is 14
[2025-06-18 00:17:21,431 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 52564, the token is 15
[2025-06-18 00:17:21,436 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 58556, the token is 16
[2025-06-18 00:17:21,442 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 54652, the token is 17
[2025-06-18 00:17:21,447 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 12268, the token is 18
[2025-06-18 00:17:21,453 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 54972, the token is 19
[2025-06-18 00:17:21,458 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 1940, the token is 20
[2025-06-18 00:17:21,464 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 27180, the token is 21
[2025-06-18 00:17:21,470 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 69636, the token is 22
[2025-06-18 00:17:21,476 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 65252, the token is 23
[2025-06-18 00:17:21,483 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 51844, the token is 24
[2025-06-18 00:17:21,488 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 67428, the token is 25
[2025-06-18 00:17:21,494 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 10164, the token is 26
[2025-06-18 00:17:21,500 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 54344, the token is 27
[2025-06-18 00:17:21,506 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 45624, the token is 28
[2025-06-18 00:17:21,512 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 63356, the token is 29
[2025-06-18 00:17:21,518 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 67768, the token is 30
[2025-06-18 00:17:21,524 I 66712 58584] (raylet.exe) worker_pool.cc:527: Started worker process with pid 60128, the token is 31
[2025-06-18 00:17:24,919 I 66712 54696] (raylet.exe) agent_manager.cc:83: Agent process with name runtime_env_agent exited, exit code -1073740791.
[2025-06-18 00:17:24,919 E 66712 54696] (raylet.exe) agent_manager.cc:87: The raylet exited immediately because one Ray agent failed, agent_name = runtime_env_agent.
The raylet fate shares with the agent. This can happen because

  • The version of grpcio doesn’t follow Ray’s requirement. Agent can segfault with the incorrect grpcio version. Check the grpcio version pip freeze | grep grpcio.
  • The agent failed to start because of unexpected error or port conflict. Read the log cat /tmp/ray/session_latest/logs/{dashboard_agent|runtime_env_agent}.log. You can find the log file structure here Configuring Logging — Ray 3.0.0.dev0.
  • The agent is killed by the OS (e.g., out of memory).
    [2025-06-18 00:17:24,920 I 66712 58584] (raylet.exe) main.cc:307: Raylet graceful shutdown triggered, reason = UNEXPECTED_TERMINATION, reason message = runtime_env_agent failed and raylet fate-shares with it.
    [2025-06-18 00:17:24,920 I 66712 58584] (raylet.exe) main.cc:310: Shutting down…
    [2025-06-18 00:17:24,920 I 66712 58584] (raylet.exe) accessor.cc:515: Unregistering node node_id=a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca
    [2025-06-18 00:17:24,931 I 66712 58584] (raylet.exe) accessor.cc:528: Finished unregistering node info, status = OK node_id=a9e15ac0ed6b999f1db9e78b067a73cc05cba789b8c258404942c6ca
    [2025-06-18 00:17:24,931 W 66712 68500] (raylet.exe) store.cc:368: Disconnecting client due to connection error with code 2: End of file
    [2025-06-18 00:17:24,942 I 66712 58584] (raylet.exe) agent_manager.cc:116: Killing agent dashboard_agent, pid 69920.
    [2025-06-18 00:17:24,947 I 66712 62472] (raylet.exe) agent_manager.cc:83: Agent process with name dashboard_agent exited, exit code 1067.
    [2025-06-18 00:17:24,947 I 66712 58584] (raylet.exe) agent_manager.cc:116: Killing agent runtime_env_agent, pid 63760.
    [2025-06-18 00:17:24,949 I 66712 58584] (raylet.exe) io_service_pool.cc:48: IOServicePool is stopped.
    [2025-06-18 00:17:25,080 I 66712 58584] (raylet.exe) stats.h:120: Stats module has shutdown.
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: *** SIGSEGV received at time=1750220245 ***
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65E4DD739 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF8DD36E7BF (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF8FC76623F (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF8FC6745D7 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF8FC765B7E (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DDDDEFA (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DEA4234 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DEA2C9F (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DEA61BD (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DDDFDF2 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DDDA9F0 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DD4DC14 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DBED788 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DC06949 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DF410A9 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DF39E3A (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DF3A384 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65E44F80C (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65E454AA4 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65E453E03 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65DB7FDB9 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF65E498598 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF8FBBFE8D7 (unknown) (unknown)
    [2025-06-18 00:17:25,081 E 66712 58584] (raylet.exe) logging.cc:496: @ 00007FF8FC63C34C (unknown) (unknown)

and here is dashboard.log
2025-06-18 00:17:14,945 ERROR dashboard.py:305 – The dashboard on node William_Hoy failed with the following error:
Traceback (most recent call last):
File “C:\Users\billy\Desktop\PopGym2\popgym\env\Lib\site-packages\ray\dashboard\dashboard.py”, line 247, in
logging_utils.redirect_stdout_stderr_if_needed(
File “C:\Users\billy\Desktop\PopGym2\popgym\env\Lib\site-packages\ray_private\logging_utils.py”, line 48, in redirect_stdout_stderr_if_needed
sys.stderr = open_log(stderr_fileno, unbuffered=True, closefd=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\billy\Desktop\PopGym2\popgym\env\Lib\site-packages\ray_private\utils.py”, line 446, in open_log
stream = open(path, **kwargs)
^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 6] The handle is invalid

Any help would be great!

2
3

I forked a public repo and was just following their setup to run it and with that I need rlib. It is a reinforcement learning problem so I think there was suppose to dashboards/prints/ etc.