I’m trying to do my own custom logging callback setup using Weights and Biases and basing it on the code integration provided for the WandbCallback. I use the same MultiProcessing setup and it works for me with some schedulers (Hyperband) but it fails on a simple one like FIFO. I can confirm that all parallel trials are logging and have their own subprocesses getting logged. WandB shows all the experiments, but only one of them is receiving active logs after setup.
I essentially have my own custom version of the WandBCallback provided, reusing a lot of the code, and it didn’t work with FIFO when running more than one experiment at once. I have a suspicion that WandbCallback suffers from the same issues but I haven’t had a chance to test it yet as I can’t directly plug that into my code. If anyone has this exact setup working please let me know.