How to properly integrate Wandb and RayTune?

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

I have followed the docs to integrate wandb with tune yet the wandb side of things don’t even seem to initialize. On my wandb page, there are no reported runs either.

I am using the WandbLoggerCallback which says that metrics reported to Tune are automatically reported to the wandb API.

I configure my tune run with the wandb callback like so:

result = tune.run(
    partial(mm_train_fun1, checkpoint_dir='/tmp/MMModels'),
    config = tune_config,
    resources_per_trial={'gpu': 1},
    metric = 'eval_loss',
    mode = 'min',
    num_samples = num_samples,
    scheduler = scheduler,

    callbacks=[WandbLoggerCallback(
        project="MultiModalClassifier",
        group='raytune_hpsearch',
        api_key=config['WandB']['api_key'],
        log_config=True
    )])

In my training, I then use tune.report to report my results:

tune.report(loss = loss.item(), epoch = epoch + batch_idx / len(train_dataloader), eval_loss=(val_loss / val_steps), eval_f1=s_f1, eval_precision=s_precision, eval_recall=s_recall)

Why isn’t any information being logged to my wandb profile?

2

Hey @Luca_Guarro can you share what your stdout looks like? Thanks!

3

Hey @amogkam, here is a screenshot:

image
image1445×792 43 KB

The hyperparameter search seems to run fine but there is no information about wandb and the results reported to stdout do not get reported to wandb either. So it is as if, it glosses over the wandbcallback altogether.

4

I am familiar with using wandb via the huggingface api, and have gotten stdout messages about wandb in those cases like so:
image

So I would expect something similar for when I get it working properly.