[Core] Capture total logs for an application

Hi,
I’d need to capture the logs for teh application, not the per worker logs, and we grab the result from stout, probelm is we get a lag and not all the logs get printed out. Is there a way to get all the logs?

Hi @delioda79, by default all of the logs should be streamed to the stdout of the driver (the main python script you run). There may be a slight delay before these logs show up because they are streamed from all of the nodes in the cluster. Could you elaborate on what you mean by there is a lag? Is this because the program is hanging?

1 Like

Thanks, it was just something totally stupid. We had forgotten to set PYTHONUNBUFFERED=1 in our environment

1 Like