So in my config object, I have a customized evaluation function
{
"evaluation_num_workers": 1,
"evaluation_interval":1,
"custom_eval_function": custom_eval_function,
}
where custom_eval_function
is defined simply as
def custom_eval_function(algorithm, eval_workers):
import mlflow
mlflow.log_metric('test_metric0', 0)
return {'test_metric1': 1, 'test_metric2': 2, 'custom_metrics': {'test_metric3': 3}}
However, none of the metrics of test_metric*
were logged by mlflow. Is there a way to log the result from evaluation with mlflow? I also added customized metrics in training to 'custom_metrics'
in MLflowLoggerCallback
and mlflow is able to pick up the customized values.