Config for layers and neurons for MLP classifier

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

Can someone help me with a config snippet on how I can custom create a search space for layers and neurons in Neural network. So far I have this,

"hidden_layer_sizes": tune.sample_from(lambda list: [list.append(tune.randint(1, 100)) for i in range(tune.randint(1, 50))]),

Thanks in advance!

2

I am running into the same problem…

I tried to define the total number of Neuron to use and then split them using a custom integer partitioning function for layer definition, and then pass it into the network using a list of numbers:

config = {"lr": tune.loguniform(1e-4, 1e-1),
          "batch_size": tune.choice([2, 4, 8, 16]),
          "h_total": tune.choice([10, 20, 30]),
          "h_branch": tune.sample_from(lambda spec: [ [N_FEATURES] +   # Append Input Layer as Number of Features
                                                      split_sampling(num_ele = h_seg_ele, 
                                                                     n_min = min_neuron_per_layer,
                                                                     out_dim = None) for h_seg_ele in spec.config.h_total ]),
         }

But this doesn’t seem to work, it gives me error when I try to refer to it in the model:

model = Net(N_SEGMENT, config["h_branch"]).to(device)

it returns error:

----> 9             self.hidden.append(NN_Branch(Layers[s_id]))
     10 

TypeError: 'Function' object is not subscriptable

It seems like Ray Tune does not allow the parameters to be multidimensional… but that’s the only way to do it when you have a flexible combination of layer and neurons. Could someone shed some lights onto this?

3

(Update)
I tried to get around the problem by creating a list of the NN architecture, and use a list of indices as the config parameters for Ray Tune. But it also failed…

config = {"lr": tune.loguniform(1e-4, 1e-1),
          "batch_size": tune.choice([2, 4, 8, 16]),
          "h_branch": tune.grid_search([1, 2, 3]),
         }
model = Net(N_SEGMENT, architecture_list, config["h_branch"])

Then it shows me KeyError…

> KeyError: "None of [Index(['grid_search'], dtype='object')] are in the [index]"

Anybody can give me some help on this…?