Report the current best accuracy

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

So I was running hyperparam exploration experiments on ray tune, and I want to keep track of the current best accuracy for each report, how could I store/access this list of values?
I.e. current best trial: ... with test_accuracy ==

image
image1571×211 103 KB

2

You could use a Tune Callback, which gives you a list of trials and their current scores:

https://docs.ray.io/en/master/tune/user-guide.html#callbacks

3

thanks! so in this case, does, for example, result[‘test_accuracy’] referring to the best config’s test accuracy of this report? If not do I need to manually search through the configs’ test accuracy and find the best accuracy to keep track of?

class MyCallback(Callback):
    def on_trial_result(self, iteration, trials, trial, result, **info):
        print(f"Got result: {result['test_accuracy']}")
4

Ah, it doesn’t refer to the best config’s accuracy.

I think you’ll want to just use the current trial’s accuracy (on_trial_result(..., trial, ...)) and manually keep track of the best accuracy.

1 Like