Additional metrics

Ray Tune
1

I have a few questions on usage of Ray Tune metrics.

Is there a documented authoritative list of possible metrics?

Is it possible for termination or stopping of trials based on validation set evaluation while training is proceeding?

Is it possible to do this based on F1 score, rather than accuracy or loss?

Thanks

2

Hey @new_to_ray, here is a list of metrics that Tune automatically reports. https://docs.ray.io/en/latest/tune/user-guide.html#auto-filled-metrics. You can also report custom metrics like in the example shown in the link. Any of these metrics that are either automatically reported by Tune or reported by you can be used as stopping conditions or passed into trial schedulers or search algorithms.

In you case you would probably want to report the “F1” metric from your training function, and then specify this metric in your stopping condition.