KeyError: "['experiment_tag'] not in index" - PopulationBasedTraining

[vishalrangras]

I am following Anyscale Academy YouTube Tutorial along with the Jupyter Notebook “03-Search-Algos-and-Schedulers.ipynb” to understand how to use PopulationBasedTraining.

I tried to display “experiment_tag” values from analysis dataframe as illustrated in the YouTube video, but I am getting: KeyError: “[‘experiment_tag’] not in index”.

Can you please help me with how to access such experimental hyperparam values?

I intend to design an experimental setup for my project based on available search and scheduling algorithms of Ray Tune. If I cannot access the experimental values (hyperparams) which provide optimum results, the experimental setup will not be of much use for me in that case.

[rliaw]

Can you post the code for what you were trying to do?

You can also use Guide to Population Based Training (PBT) — Ray v2.0.0.dev0 to help yourself out.

[vishalrangras]

Thank you for this tutorial, its helpful. As I mentioned, I am following the official tutorial from anyscale-academy which is available here: academy/03-Search-Algos-and-Schedulers.ipynb at main · anyscale/academy · GitHub

In this tutorial, it is mentioned that in order to see the actual values of hyperparameters tried by PBT search algorithm from analysis dataframe, we need to look at experimental_tag column of dataframe instead of config/lr and config/momentum columns. But I am getting KeyError when I try to read experimental_tag column. Can you recommend how I can access the actual hyperparameter values tried during PBT based Hyperparam search? Here is the screenshot for your reference:

[vishalrangras]

Also, I do not see Hyperparam tab in tensorboard when I try PBT, but such tab is available when I do Grid Search. Is it possible to see similar tab of Hyperparameters in tensorboard when doing PBT based search? I am talking about following tab of HPARAMS: