Use tqdm_ray in remote tasks

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

I want to use tqdm_ray to monitor progress on worker tasks I have submitted to the ray cluster.

import random
from time import sleep

import ray
from ray.experimental import tqdm_ray


@ray.remote
def worker(bar: tqdm_ray.tqdm):
    dt = random.random()
    sleep(dt)
    bar.update(1)


if __name__ == '__main__':
    n = 10
    bar = tqdm_ray.tqdm(total=n)
    ray.get([worker.remote(bar) for _ in range(n)])
    bar.close()

I would expect to see a single tqdm progress bar, that increments every time one of the worker tasks finishes. Instead I see the empty progress bar remaining at 0 and vanishing after all workers have completed.

What am I doing wrong?

2

In case anyone else is looking for this, the solution is actually quite simple: derive an actor from tqdm_ray and use that:

import random
from time import sleep

import ray
from ray.experimental import tqdm_ray

remote_tqdm = ray.remote(tqdm_ray.tqdm)

@ray.remote
def worker(bar: tqdm_ray.tqdm):
    dt = random.random()
    sleep(dt)
    bar.update.remote(1)


if __name__ == '__main__':
    n = 10
    bar = remote_tqdm.remote(total=n)
    ray.get([worker.remote(bar) for _ in range(n)])
    bar.close()
3

@Valentin_Stauber Can I close this since you found the solution?

1 Like