I want to tune an sklearn ElasticNet model which is nested within a nested sklearn pipeline. More specifically, my trainable looks as follows:
from sklearn.linear_model import ElasticNet
from sklearn.preprocessing import RobustScaler
from sklearn.pipeline import Pipeline
from sklearn.compose import TransformedTargetRegressor
from ray.tune.sklearn import TuneSearchCV
pipeline = Pipeline([
('scaler', RobustScaler()),
('estimator', ElasticNet(, max_iter=100))
])
ELN = Pipeline([
('model', TransformedTargetRegressor(regressor=pipeline, transformer=RobustScaler()))
])
I want to tune ELN
with ray.tune
, i.e.:
ELN_tune = TuneSearchCV(
estimator=ELN,
scoring='neg_mean_squared_error',
param_distributions=pgrid,
cv=pds,
search_optimization='random',
early_stopping=True,
n_trials=4,
max_iters = 10,
random_state=123,
verbose=2,
)
However, trying to tune yields the following error:
ValueError: Early stopping is not supported because the estimator does not have
partial_fit, does not support warm_start, or is a tree classifier. Set
early_stopping=False.
Yet, ElasticNet()
does in fact support warm_start
. Also, when I use the sklearn tuning equivalent (e.g. sklearn.model_selection.GridSearchCV
), I get no error.
How do I solve this issue? It seems like Ray checks wether the estimator, in this case the TransformedTargetRegressor object, has the possibility of employing early stopping/partial fitting. However, the check wrongly fails in this case, as it should not check the TransformedTargetRegressor object, but the pipeline object within it. Thanks in advance.