I have a large dataset. I’m using tune.with_parameters to pass the dataset to trainable function. Here is the code for tuning trainable function def xgboost_hyper_param(config, data = None): #max_depth = int(max_depth) #params = {'max_depth' : max_depth, 'learning_rate': learning_rate, 'gamma': …

Handling large datasets results in error

ashwini October 5, 2021, 11:12am 3

It is 405 mb. The custom metric function in xgboost had a global variable (pd.DataFrame) leading to big size. Thanks for the help.

Topic		Replies	Views
StatusCode.RESOURCE_EXHAUSTED Ray Tune	21	5168	April 11, 2023
XGBoost + large data with Ray Tune: How? Ray Data	0	25	July 29, 2025
Error in Colab: ImplicitFunc is very large and grpc_status”:8 Ray Tune	1	873	February 8, 2022
Again a "The actor ImplicitFunc is too large error" Ray Tune	2	2513	June 22, 2023
Tuning fails with "The actor ImplicitFunc is too large" Ray Tune	2	1237	September 1, 2021