# How to define fcnet\_hiddens size and number of layers in rllib tune?
**URL:** https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504
**Category:** RLlib
**Created:** [June 14, 2022, 4:58pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504 "2022-06-14T16:58:38Z")
**Posts on this page:** 19
**Page:** 1
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 14, 2022, 4:58pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/1 "2022-06-14T16:58:38Z")
I wonder how to define in rllib tune layers and neurons in the layers.
I would like to do it with two parameters:
- number of layers 1,2 or 3
- neurons in layer 8 to 256
Each layer has the same number of neurons for simplicity.
```auto
"model": {
"fcnet_hiddens": tune.choice([[32, 32,32], [64, 64], [128, 128]]),
},
```
---
### Author:  [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [June 14, 2022, 5:31pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/2 "2022-06-14T17:31:08Z")
Hi @Peter_Pirog,
Here is one way to do it. You could also just use np.rand… in that lambda function instead if you wanted.
```auto
"fcnet_hiddens": tune.sample_from(lambda: [tune.choice(range(8,257)).sample()]*tune.choice(range(1,4)).sample())
```
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 14, 2022, 5:33pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/3 "2022-06-14T17:33:08Z")
Thank You very much for the answer.
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 14, 2022, 5:55pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/4 "2022-06-14T17:55:22Z")
@mannyv when i try to use optuna it gives the error:
> ValueError: Optuna search does not support parameters of type `Function` with samplers of type `NoneType`
---
### Author:  [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [June 14, 2022, 6:14pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/5 "2022-06-14T18:14:30Z")
Hi @Peter_Pirog,
I thought you were running with ray.tune in a grid search type setup. I would think with optuna you would specify a search config sorta like this
```auto
search_space = {
"num_layers": tune.ray.tune.randint(1,4),
"num_units": ray.tune.randint(8,257),
#...
}
```
And then in your “objective” function you would have something like:
```auto
def objective(config):
#...
rllib_config["model"]["fcnet_hiddens"] = [config["num_units"]]*config["num_layers"]
#...
```
---
### Author:  [@sirjay](https://discuss.ray.io/u/sirjay)
#### Post date: [June 15, 2022, 4:12am UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/6 "2022-06-15T04:12:19Z")
@mannyv @Peter_Pirog I have silly question, is it below correct? What’s `objective` param name from `tune.run()`? I don’t understand by [Execution (tune.run, tune.Experiment) — Ray 1.13.0](https://docs.ray.io/en/latest/tune/api_docs/execution.html)
```auto
def objective(config):
config["model"]["fcnet_hiddens"] = [config["num_units"]]*config["num_layers"]
return config
search_space = {
"env": MyEnv,
"num_layers": tune.ray.tune.randint(1,4),
"num_units": ray.tune.randint(8,257),
"lr": tune.uniform(5e-6, 3e-3),
}
analysis = tune.run(
objective, # what's param name?
config=search_space,
run_or_experiment="PPO",
metric='episode_reward_mean',
mode='max',
stop={"episodes_total": 100},
)
```
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 15, 2022, 3:33pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/7 "2022-06-15T15:33:22Z")
@mannyv @sirjay , sirjay, similar like You I don’t understand how the ‘objective’ is connected with input arguments of `tune.run`.
The first argument is `run_or_experiment` but this value is defined here:
```auto
run_or_experiment="PPO"
```
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 19, 2022, 12:22pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/8 "2022-06-19T12:22:06Z")
@mannyv @sirjay
I try to run code below:
```auto
from ray import tune
from ray.tune.suggest.optuna import OptunaSearch
def objective(config):
config["model"]["fcnet_hiddens"] = [config["num_units"]]*config["num_layers"]
return config
search_space = {
#"env": MyEnv,
"env": "LunarLander-v2",
"num_layers": tune.randint(1,4), #tune.ray.tune.randint(1,4),
"num_units": tune.randint(8,257), #ray.tune.randint(8,257),
"lr": tune.uniform(5e-6, 3e-3),
}
analysis = tune.run(
objective, # what's param name?
config=search_space,
search_alg=OptunaSearch(),
run_or_experiment="PPO",
metric='episode_reward_mean',
mode='max',
stop={"episodes_total": 100},
)
```
but the error is:
```auto
Traceback (most recent call last):
File "/home/ppirog/projects/Mastering-Reinforcement-Learning-with-Python/custom_scripts/optuna_tune.py", line 15, in
analysis = tune.run(
TypeError: run() got multiple values for argument 'run_or_experiment'
Process finished with exit code 1
```
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 19, 2022, 8:18pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/9 "2022-06-19T20:18:07Z")
@Yard1 , Yard I found your post here:
[https://github.com/ray-project/ray/issues/19698](https://github.com/ray-project/ray/issues/19698)
Could You explain how to use the define-by-run interface to define conditional hyperparameters? Link to the example doesn’t work.
---
### Author:  [@Yard1](https://discuss.ray.io/u/Yard1)
#### Post date: [June 19, 2022, 8:32pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/10 "2022-06-19T20:32:50Z")
Hey, you can find the example here - [ray/optuna\_define\_by\_run\_example.py at master · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/optuna_define_by_run_example.py)
Let me know if you need any help with it. It should be straightforward!
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 19, 2022, 8:50pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/11 "2022-06-19T20:50:19Z")
@Yard1 , Thank You very much for the quick answer 😀. Now, I will try to analyze it, and try to connect this example with rllib tuning.
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 20, 2022, 5:24pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/12 "2022-06-20T17:24:35Z")
@Yard1 Is there any example with rllib? i’m not sure how to put algorithm insted of `easy_objective`. In typical case I use 'PPO" as algorithm name but here I have some problem with objective function.
---
### Author:  [@sirjay](https://discuss.ray.io/u/sirjay)
#### Post date: [June 21, 2022, 7:14am UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/13 "2022-06-21T07:14:10Z")
@Yard1 I am interested too!
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 23, 2022, 10:47pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/14 "2022-06-23T22:47:22Z")
@sirjay I still try to combine rllib algorithm with objective function. I hope the examples below can help me somehow:
[https://github.com/ray-project/ray/blob/master/rllib/examples/custom\_experiment.py](https://github.com/ray-project/ray/blob/master/rllib/examples/custom_experiment.py)
[https://docs.ray.io/en/latest/tune/api\_docs/trainable.html#training-tune-trainable-tune-report](https://docs.ray.io/en/latest/tune/api_docs/trainable.html#training-tune-trainable-tune-report)
---
### Author:  [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [June 24, 2022, 1:27pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/15 "2022-06-24T13:27:42Z")
Hi @Peter_Pirog,
I had a few free minutes this morning. Here is an example script for how you could get this running.
There is a collab you can experiment with here: [Google Colab](https://colab.research.google.com/drive/1NV7_neINRoLLXGE1MQ-z7Qj2iWBMxpFG?usp=sharing)
If you found this useful check out this link: [Manny is helping people do cool things with RL](https://www.buymeacoffee.com/mannyv)
```python
import ray
from ray import tune
from ray.rllib.agents.ppo import PPOTrainer, PPOConfig
from ray.rllib.utils.typing import TrainerConfigDict
from ray.tune.suggest.optuna import OptunaSearch
# You can ignore this. Collab does not have enough CPUs to run with the default settings. This is a workaround.
ray.shutdown()
ray.init(num_cpus=10)
class MyPPO(PPOTrainer):
@classmethod
def get_default_config(cls) -> TrainerConfigDict:
ppo_config = PPOConfig().to_dict()
ppo_config["num_units"] = None
ppo_config["num_layers"] = None
return ppo_config
def validate_config(self, config: TrainerConfigDict) -> None:
if config["num_layers"] is not None and config["num_units"] is not None:
self.num_layers = config["num_layers"]
self.num_units = config["num_units"]
config["model"]["fcnet_hiddens"] = [self.num_units] * self.num_layers
super().validate_config(config)
tune.register_trainable("MyPPO", MyPPO)
search_space = {
"env": "LunarLander-v2",
"num_layers": tune.randint(1,4),
"num_units": tune.randint(8,257),
"lr": tune.uniform(5e-6, 3e-3),
}
analysis = tune.run(
config=search_space,
search_alg=OptunaSearch(),
run_or_experiment="MyPPO",
metric='episode_reward_mean',
mode='max',
stop={"episodes_total": 100},
num_samples=5,
)
print("Best hyperparameters found were: ", analysis.best_config)
ray.shutdown()
```
---
### Author:  [@Peter\_Pirog](https://discuss.ray.io/u/Peter_Pirog)
#### Post date: [June 24, 2022, 5:26pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/16 "2022-06-24T17:26:19Z")
Nice work, coffee for you ☕
---
### Author:  [@Dejan\_Grubisic](https://discuss.ray.io/u/Dejan_Grubisic)
#### Post date: [August 9, 2022, 4:19pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/17 "2022-08-09T16:19:54Z")
How about
```auto
num_layers = [3, 10, 20]
num_nodes = [50, 100, 500]
"model": {"fcnet_hiddens": tune.choice([[N] * L for N in num_nodes for L in num_layers ])}
```
---
### Author:  [@AI360](https://discuss.ray.io/u/AI360)
#### Post date: [January 19, 2023, 2:09pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/18 "2023-01-19T14:09:38Z")
Excuse me, when I run this code, I get the below error. could you help me how I could solve it?
/usr/local/lib/python3.8/dist-packages/ray/rllib/algorithms/algorithm.py in default\_resource\_request(cls, config)
2041
2042 # Convenience config handles.
→ 2043 cf = cls.get\_default\_config().update\_from\_dict(config)
2044 cf.validate()
2045 cf.freeze()
AttributeError: ‘dict’ object has no attribute ‘update\_from\_dict’
---
### Author:  [@AI360](https://discuss.ray.io/u/AI360)
#### Post date: [January 19, 2023, 2:39pm UTC](https://discuss.ray.io/t/how-to-define-fcnet-hiddens-size-and-number-of-layers-in-rllib-tune/6504/19 "2023-01-19T14:39:20Z")
/usr/local/lib/python3.8/dist-packages/ray/rllib/algorithms/algorithm.py in default\_resource\_request(cls, config)
2041
2042 # Convenience config handles.
→ 2043 cf = cls.get\_default\_config().update\_from\_dict(config)
2044 cf.validate()
2045 cf.freeze()
AttributeError: ‘dict’ object has no attribute ‘update\_from\_dict’