MLflow with Ray in Databrick is throwing error?

1

I am using mlflow to track training experiments done using Ray in databricks in multi-worker distributed setting. When I am running training.fit(), the following error is thrown

mlflow.legacy_databricks_cli.configure.provider.InvalidConfigurationError: You haven't configured the CLI yet! Please configure by entering `/databricks/python/lib/python3.11/site-packages/ray/_private/workers/default_worker.py configure`

my training function has following form

def training_function(config):
       mlflow = setup_mlflow(config, tracking_uri="databricks", registry_uri="databricks-uc" ,experiment_name='raywiithmlflow', create_experiment_if_not_exists=True)
      
       for epoch in range(n_epochs_=):
             .....
             .....
             mlflow.log_metric(metrics, step=epoch)

How can I debug this issue?

2

This feels like more of an mlflow and databricks issue than something to do with Ray; can you try asking on those relevant discourses?

3

thanks, this was a databricks issue.