Use tqdm_ray in remote tasks

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

I want to use tqdm_ray to monitor progress on worker tasks I have submitted to the ray cluster.

import random
from time import sleep

import ray
from ray.experimental import tqdm_ray


@ray.remote
def worker(bar: tqdm_ray.tqdm):
    dt = random.random()
    sleep(dt)
    bar.update(1)


if __name__ == '__main__':
    n = 10
    bar = tqdm_ray.tqdm(total=n)
    ray.get([worker.remote(bar) for _ in range(n)])
    bar.close()

I would expect to see a single tqdm progress bar, that increments every time one of the worker tasks finishes. Instead I see the empty progress bar remaining at 0 and vanishing after all workers have completed.

What am I doing wrong?

2

In case anyone else is looking for this, the solution is actually quite simple: derive an actor from tqdm_ray and use that:

import random
from time import sleep

import ray
from ray.experimental import tqdm_ray

remote_tqdm = ray.remote(tqdm_ray.tqdm)

@ray.remote
def worker(bar: tqdm_ray.tqdm):
    dt = random.random()
    sleep(dt)
    bar.update.remote(1)


if __name__ == '__main__':
    n = 10
    bar = remote_tqdm.remote(total=n)
    ray.get([worker.remote(bar) for _ in range(n)])
    bar.close()
3

@Valentin_Stauber Can I close this since you found the solution?

1 Like
4

Hi there,

this thread is quite old but its basically the only example on how to use tqdm_ray for this usecase, so monitoring the progress of how many remota tasks have been accomplished.

sadly this does not work for me at all, the progressbar just bugs out, sometimes i have none, sometimes multiple, and they also dont go away anymore…

Is there a simple way to show how many tasks have been concluded?

so basically

pbar = ...
ray.get([some_remote_fc.remote(data, pbar) for data in datas])

I just need a way to see if progress is being made or something is stuck…