How does Ray Bayesian Optimization HyperBand (BOHB) work?

1

Hi, I need help with BOHB (Bayesian Optimization HyperBand). I am running a param_space with TuneBOHB search and HyperBandForBOHB scheduler with the following settings:
num_samples = 25, max_T = 2000, reduction_factor = 3, training_iteration = 400
How exactly is the search and scheduler supposed to work:

  • How many initial trials are run before the Bayesian Optimization starts constructing the surrogate function?
  • How does max_T interact with num_samples to decide on when BO proposes new trial parameters?
  • When I look at the output of my run I see 19 of the 25 trials running after 2 iterations from start (and all 25 trials are running within 6 iterations from the start). This seems too early for the BO to come up with all the trial parameter recommendations!

Thanks.