Failed to initialize Rabit when running XGBoost on Ray

Ray Libraries (Data, Train, Tune, Serve) Ray Train
1

I kept getting the following err when running XGBoost on Ray. There is no additional err msg. What might be the cause?

/build/build/ext/public/python/xgboost/1/7/3/build/python3.9/src/collective/rabit_communicator.h:47: Failed to initialize Rabit

2

Rabit seems to be a collective library that XGBoost uses:

You probably need to install it as a dependency.

3

Rabit will be installed as part of xgboost. The inability to initialize it usually means a networking issue.

4

Thanks. After this err the whole training failed. Is there a way to increase number of retries to tolerate this err?

5

I am not sure, but you should be able to use the built-in xgboost-ray fault tolerance to automaticaly restart the training if a failure is encountered