Is there any way to export all of the training metrics shown in tensorboard to other services like wandby/comet/ML flow? From the documentation it seems like they all require you to manually call wandb.log() or session.report(), which makes it much more difficult for debugging. I guess the typical use case is you debug and get things working with tensorboard and then once your model is training decently you put the loss or average I see in the documentation that there is only lightweight integration with wandb, I guess that means it’s best to just use tensorboard? Using Weights & Biases with Tune — Ray 2.3.0
I don’t see anything about neptune officially but on neptune’s site they have a post showing a logger, which I’m going to try out but I wanted to ask here to see if anyone has advice for which tools they’ve found most helpful/best integrated.
Logging in Reinforcement Learning Frameworks - What You Need to Know - neptune.ai (edit: it lies and says 2023 in the blog post but the github is from 2018 and last updated 3 years ago, so definitely not up to date with current RLlib)
I also was wondering about hyperparameter sweeps with ray.tune compared with those from wandb or other frameworks, would think doing sweeps and tuning within ray would be the best way to go, so maybe for that reason a simple tool like tensorboard is perfectly fine for most people?