I’m using ray debugger on vscode, I followed the official instructions to set breakpoint and run my code, the code stops at the breakpoint, then I use the ray distributed debugger to connect to that port by adding one cluster 127.0.0.1:46898, the following error happened:
(main_task pid=77949) RemotePdb session open at localhost:34269, use 'ray debug' to connect...
(main_task pid=77949) RemotePdb accepted connection from ('127.0.0.1', 46898).
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/root/autodl-tmp/segzero/verl/trainer/main.py", line 95, in <module>
main()
File "/root/autodl-tmp/segzero/verl/trainer/main.py", line 44, in main
ray.get(main_task.remote(ppo_config))
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/site-packages/ray/_private/worker.py", line 2745, in get
values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/site-packages/ray/_private/worker.py", line 901, in get_objects
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ConnectionResetError): ray::main_task() (pid=77949, ip=172.17.0.11)
File "/root/autodl-tmp/segzero/verl/trainer/main.py", line 90, in main_task
trainer.init_workers()
^^^^^^^
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/bdb.py", line 100, in trace_dispatch
return self.dispatch_line(frame)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/bdb.py", line 124, in dispatch_line
self.user_line(frame)
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/pdb.py", line 329, in user_line
self.interaction(frame, None)
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/pdb.py", line 428, in interaction
self._cmdloop()
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/pdb.py", line 390, in _cmdloop
self.cmdloop()
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/cmd.py", line 132, in cmdloop
line = self.stdin.readline()
^^^^^^^^^^^^^^^^^^^^^
File "/root/miniconda3/envs/visionreasoner/lib/python3.12/socket.py", line 720, in readinto
return self._sock.recv_into(b)
^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer