Multiple callbacks in Tune

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Dear community,

would anyone have an example of using multiple Callbacks in a tuner? Next to the well-working WandbLoggerCallback, I would like to deploy a callback who checks for some values available at the end of each episode.

First idea is something like that deploy, but this runs into multiple issues rooted in ray/tune/callback.

 run_config=air.RunConfig(
        name="my_run", 
        stop=stopping_criteria,
        checkpoint_config=air.CheckpointConfig(
            checkpoint_score_attribute="episode_reward_mean",
            checkpoint_score_order="max",
            checkpoint_frequency=10,
            checkpoint_at_end=True,
        ),
        log_to_file=True,
        callbacks=[WandbLoggerCallback(project="wandb-project", log_config=True), MyTuneCallback]      
    ),
2

Tune callbacks require you to instantiate them before passing them into RunConfig(callbacks):

-MyTuneCallback
+MyTuneCallback()