VScode breakpoint will be bypassed even with local_mode=True

Ray Train
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I am writing a DL training framework that leverages Ray Train and Ray Tune. Unlike my previous experience with RLLib, VScode breakpoints that I set in Ray Train processes would be passed despite I set local_mode=True in Ray 2.1.0. Which forced me to use Ray Debug, which is not convenient for Test Driven Development.

Is this an expected behavior for Ray Train, or is it a bug? Or is there anything I missed?

2

Hi Team, is this a known issue? Or we have plan to allow synchronized debugging in Ray Train?

3

Hey @heng2j, sorry for the late response. Local_mode has been hard to maintain and I suspect it won’t work quite well with Ray Train.

A couple options for you include:

  1. Setting num_workers=0 or num_workers=1.
  2. Using Ray Debug.

I’d be curious about your experience with Ray Debug. What makes it inconvenient?

1 Like
4

I am searching for a way to use the vscode debugger with Ray. I have used the ray debug mode, and while it works and is effective for quick checks, it lacks all modern QOL which reduces development speed by orders of magnitude:

  • Fast breakpoint deactivation/activation
  • Adding breakpoints during a live run
  • Interactive gui scope for quick variable interrogation
  • ability to copy paste multi line functions or loops without indentation considerations
  • simple things like being able to use the up arrow to re-execute and modify previous commands

Any chance vscode debugging will eventually be supported in Ray?

Cheers,

Robert

5

Hi @rliaw, thank you for getting back to me.

For my recent projects, I only use Ray Train when needed due to 1) personal preference of the VSCode debugger workflow. 2) The scope of the RLLib project allows me to debug with your suggested num_workers=1 and using local_mode. Especially when I can set "justMyCode": false in my VScode launch script

6

@rliaw is it possible to attach Ray Debugger to vscode’s remote debugging? thus inspecting the variables and stepping through the code visually in vs code not the terminal.

7

We would also greatly appreciate this feature.
Even just being able to use the up-arrow to retrieve the last run command would be a big improvement (this is a standard pdb/terminal feature). Currently, pressing the up-arrow in a ray debug session does not retrieve the previously run command.