XGBoostTrainer Warning: Saving into deprecated binary model format

Ray Libraries (Data, Train, Tune, Serve) Ray Train
1

There are warning messages that keep spamming during the training progress when using XGBoostTrainer + RunConfig & CheckpointConfig.

WARNING: /workspace/src/c_api/c_api.cc:1240: Saving into deprecated binary model format, please consider using json or ubj. Model format will default to JSON in XGBoost 2.2 if not specified.

Is there any argument to pass to specify the model format to suppress this warning?

2
3

Hey @daviddwlee84 , can you share which version of Ray you are using? I believe that XGBoostTrainer should be saving to json format already.

Related: Saving XGBoost model with json extension · Issue #41374 · ray-project/ray · GitHub

4

Sorry for the late reply, might be missing the notification.
I am using Ray 2.8.1, which should be using the json format.

I have raised the issue in GitHub.
It is kind of weird, I haven’t found the root cause. Seems only I can reproduce the issue.

=> No matter what I specify the XGBoostTrainer._save_model(), somehow it will call the legacy version of booster.save_model() again somewhere, and the checkpoint contents are not successfully copied from the temp directory to where it should be persisted on NAS. (In Ray 2.8+ will use ray.train.report to copy to persistent storage.

[Train] XGBoost continue train (resume_from_checkpoint) and get_model failed · Issue #41608 · ray-project/ray

5

The root cause is the XGBoostTrainer’s default _tune_callback_checkpoint_cls is using MODEL_KEY which is model and not aligned with XGBoostCheckpoint’s model.json.

Here is the workaround I found.

For the detail investigation check the issue itself.

1 Like