I am getting
ValueError: Trial returned a result which did not include the specified metric(s) [‘val_ks’, ‘ks_diff’]
expects. Make sure your calls totune.report()
include the metric, or set the TUNE_DISABLE_STRICT_METRIC_CHECKING environment variable to 1.
how can I report custom metrics from train_breast_cancer function in
xgboost_ray/simple_tune.py at master · ray-project/xgboost_ray · GitHub ? I have tried train.report, session.report and tune.report nothing seems to work.
February 3, 2023, 7:48pm
the metrics should be generated by XGBoost which will then automatically report them to Ray Tune.
You can do this using a customized metric function that you pass to
Here are the relevant docs for XGBoost:
Custom Objective and Evaluation Metric — xgboost 1.7.3 documentation
And it could look like this
def ks(predt: np.ndarray, dtrain: xgb.DMatrix) -> Tuple[str, float]:
val_ks = ...
return 'val_ks', val_ks
def train_breast_cancer(config, ray_params):
bst = train(