Hi! I am using Ray Tune with AxSearch to perform hyperopt on my model training, and I was wondering whether Tune should internally report to Ax the best or the last trial result.
I tried to dive deep into Ray Tune code, and what I found is that the TrialRunner allows every scheduler and search algorithm to make use of the result of every training step, however, when the on_trial_complete callback is called, the last result is returned as the result parameter. Since AxSearch only implements the on_trial_complete callback, I think it gets the last result, instead of the best one, which - I suppose - misleads the BO process in the bakcground.
I have also found this topic which seems similar to my question: Best model based on Checkpoint not Last epoch
Can someone tell me, whether my logic and findings are right? Or have I missed something maybe?