Is there integration for wandb.log() from worker nodes?

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

I’m logging my experiments with W&B (wandb.ai). When I call wandb.log() from a remote worker, it throws:

ray.exceptions.RayTaskError: ray::mapper() (pid=9299, ip=172.31.2.150)
  File "python/ray/_raylet.pyx", line 480, in ray._raylet.execute_task
  File "/home/ubuntu/main.py", line 90, in mapper
    wandb.log({"mapper_finished": f"M-{mapper_id}"})
  File "/home/ubuntu/miniconda3/lib/python3.8/site-packages/wandb/sdk/lib/preinit.py", line 38, in preinit_wrapper
    raise wandb.Error("You must call wandb.init() before {}()".format(name))
wandb.errors.error.Error: You must call wandb.init() before wandb.log()

I’m not using Tune at the moment, but I see there is integration available (ray.tune.integration.wandb — Ray v2.0.0.dev0). I wonder if it would allow me to log from remote workers, then?

2

I would look through the code used for the tune integration. They have two solutions. In their Callback solution all the worker logs are passed back to the main thread to be logged as a dictionary which I think is what you’re looking for.