# Best trail printing None when input a nested config

**URL:** <https://discuss.ray.io/t/best-trail-printing-none-when-input-a-nested-config/4427>\
**Category:** Ray Tune\
**Created:** [December 13, 2021, 10:18am UTC](https://discuss.ray.io/t/best-trail-printing-none-when-input-a-nested-config/4427 "2021-12-13T10:18:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eleven1Liu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/eleven1liu/32/1093_2.png) [@Eleven1Liu](https://discuss.ray.io/u/Eleven1Liu)\
**Post date:** [December 13, 2021, 10:18am UTC](https://discuss.ray.io/t/best-trail-printing-none-when-input-a-nested-config/4427/1 "2021-12-13T10:18:10Z")

</div>

Hi,

I was trying to use `tune.run()` with a nested config like this:

```auto
config = {
    "seed": 1337
    "epochs": 200
    "batch_size": 16
    "learning_rate": tune.grid_search([0.001, 0.003, 0.0001, 0.0003]),
    "network_config": {
         "num_filter_per_size": tune.grid_search([50, 150, 250, 350, 450, 550]),
         "filter_sizes": tune.grid_search([[2], [4], [6], [8], [10]]),
         "dropout": tune.grid_search([0.2, 0.4, 0.6, 0.8])
    }
}

```

When passing in the nested config, parameters specified in the nested config are `None` in “Current best trial”. Please see the screenshot below.

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/3/361237decb0bf1454f18684461dcf87290ade66c.png)

The reason is that function `best_trail_str` is looking for `config['network_config/dropout']` (i.e., `p`) that doesn’t exist. It should be unflattened to `config['network_config']['dropout']`.

> <https://github.com/ray-project/ray/blob/b3a9d4d87d31255e0871852efaf1ddff7acdc390/python/ray/tune/progress_reporter.py#L790>

Is passing in nested config an expected behavior for running `tune.run()`?  
Is there any suggestion for using the nested config?

Thanks!

---

<div class="post-metadata">

**Author:** ![kai](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kai/32/3380_2.png) [@kai](https://discuss.ray.io/u/kai)\
**Post date:** [December 13, 2021, 11:54am UTC](https://discuss.ray.io/t/best-trail-printing-none-when-input-a-nested-config/4427/2 "2021-12-13T11:54:48Z")

</div>

For me it works as expected:

```auto
Current best trial: ffab5_00000 with _metric=5 and parameters={'seed': 1337, 'epochs': 200, 'batch_size': 16, 'learning_rate': 0.001, 'network_config': {'num_filter_per_size': 50}}

```

Which version of Ray are you using? Can you share more of your code with us or provide a reproducible example (running on the latest master or at least latest release)?

The fix should be easy (we’ll just use `unflattened_lookup` to get the right result), but to test this we need to be able to reproduce it.

---

<div class="post-metadata">

**Author:** ![Eleven1Liu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/eleven1liu/32/1093_2.png) [@Eleven1Liu](https://discuss.ray.io/u/Eleven1Liu)\
**Post date:** [December 14, 2021, 10:17am UTC](https://discuss.ray.io/t/best-trail-printing-none-when-input-a-nested-config/4427/3 "2021-12-14T10:17:55Z")

</div>

Hi @kai,

Sure. The ray version is 1.9.0.  
Please see the example below:

### Code

```python
from ray import tune

def objective(step, alpha, beta):
    return (0.1 + alpha * step / 100)**(-1) + beta * 0.1

def training_function(config):
    alpha, beta = config["learning_rate"], config["network_config"]["num_filter_per_size"]
    for step in range(10):
        intermediate_score = objective(step, alpha, beta)
        tune.report(mean_loss=intermediate_score)

metric, mode = 'mean_loss', 'min'
config = {
    "seed": 1337,
    "epochs": 200,
    "batch_size": 16,
    "learning_rate": tune.grid_search([0.001, 0.003, 0.0001, 0.0003]),
    "network_config": {
        "num_filter_per_size": tune.grid_search([50, 150, 250, 350, 450, 550])
    }
}
# the `parameter_columns` is for simplifying the column names in the CLIReporter
# for example, `network_config/num_filter_per_size` -> `num_filter_per_size` 
parameter_columns = {
      'learning_rate': 'learning_rate', 
      'network_config/num_filter_per_size': 'num_filter_per_size'
}
reporter = tune.CLIReporter(metric_columns=['mean_loss'],
                            parameter_columns=parameter_columns,
                            metric=metric, mode=mode,
                            sort_by_metric=True)
analysis = tune.run(
        training_function,
        num_samples=1,
        resources_per_trial={'cpu': 4, 'gpu': 1},
        progress_reporter=reporter,
        config=config)
print("Best config: ", analysis.get_best_config(metric=metric, mode=mode))

```

### Screenshot

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/6/6bdc4b1429748021ac5877c684f438048f810425.png)

Thanks!!

---

<div class="post-metadata">

**Author:** ![kai](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kai/32/3380_2.png) [@kai](https://discuss.ray.io/u/kai)\
**Post date:** [December 14, 2021, 4:57pm UTC](https://discuss.ray.io/t/best-trail-printing-none-when-input-a-nested-config/4427/4 "2021-12-14T16:57:34Z")

</div>

Thanks for the repro script - this issue only comes up when using custom `parameter_columns`, hence why I didn’t see it first.

With the reproduction script I was able to fix this here: [[tune] Fix best\_trial\_str for nested custom parameter columns by krfricke · Pull Request #21078 · ray-project/ray · GitHub](https://github.com/ray-project/ray/pull/21078)

This fix should be included in the next Ray release (1.10.0). Thanks again for raising this!

---

<div class="post-metadata">

**Author:** ![Eleven1Liu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/eleven1liu/32/1093_2.png) [@Eleven1Liu](https://discuss.ray.io/u/Eleven1Liu)\
**Post date:** [December 14, 2021, 11:07pm UTC](https://discuss.ray.io/t/best-trail-printing-none-when-input-a-nested-config/4427/5 "2021-12-14T23:07:28Z")

</div>

> [@kai](#):
>
> [tune] Fix best\_trial\_str for nested custom parameter columns by krfricke · Pull Request #21078 · ray-project/ray · GitHub

Great! Thanks again for the help! 😁