# Correct usage of tune sampling in AlgorithmConfig dicts **URL:** \ **Category:** Configure Algorithm, Training, Evaluation, Scaling\ **Created:** [April 16, 2023, 8:59pm UTC](https://discuss.ray.io/t/correct-usage-of-tune-sampling-in-algorithmconfig-dicts/10254 "2023-04-16T20:59:01Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![PhilippWillms](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/philippwillms/32/3885_2.png) [@PhilippWillms](https://discuss.ray.io/u/PhilippWillms)\ **Post date:** [April 16, 2023, 8:59pm UTC](https://discuss.ray.io/t/correct-usage-of-tune-sampling-in-algorithmconfig-dicts/10254/1 "2023-04-16T20:59:02Z")
I know that a similar topic came up in this [old thread](https://discuss.ray.io/t/error-when-setting-up-bayesian-optimizer-with-asha-scheduler/3807), but there was no answer communicated. Basically, the same thing is happening to me now. I would like to use the new `AlgorithmConfig` API and define 2 parameters in the `.training()` section as a result of tune sampling. Note that I am using framework “tf” on purpose, as “tf2” always leads to memory crashes on my workstation (even when setting eager\_tracing = False). ```auto trainer_config = ( PPOConfig() .environment(MyCustomEnv) .checkpointing( export_native_model_files=True, checkpoint_trainable_policies_only=True ) .framework("tf") .callbacks(MyCallback) .rollouts( num_rollout_workers=1 ) .resources( num_cpus_per_worker=1 ).debugging( log_level="INFO" ) .training( model={"custom_model": MyTFModelV2}, train_batch_size=10, sgd_minibatch_size=5, num_sgd_iter=2, gamma=tune.grid_search([0.9, 0.99, 0.999]), lr=tune.loguniform(1e-4, 1e-1), ) ) ``` Sorry for the wall of text, but as @kai also asked in the thread linked above for the trainer\_config, I post it below. → Could the issue be related to the fact that tune.loguniform() returns a Float? But how to overcome? > { ‘\_disable\_action\_flattening’: False, > ‘\_disable\_execution\_plan\_api’: True, > ‘\_disable\_preprocessor\_api’: False, > ‘\_enable\_rl\_module\_api’: False, > ‘\_enable\_rl\_trainer\_api’: False, > ‘\_fake\_gpus’: False, > ‘\_rl\_trainer\_hps’: RLTrainerHPs(), > ‘\_tf\_policy\_handles\_more\_than\_one\_loss’: False, > ‘action\_space’: None, > ‘actions\_in\_input\_normalized’: False, > ‘always\_attach\_evaluation\_results’: False, > ‘auto\_wrap\_old\_gym\_envs’: True, > ‘batch\_mode’: ‘truncate\_episodes’, > ‘callbacks’: \, > ‘checkpoint\_trainable\_policies\_only’: True, > ‘clip\_actions’: False, > ‘clip\_param’: 0.3, > ‘clip\_rewards’: None, > ‘compress\_observations’: False, > ‘create\_env\_on\_driver’: False, > ‘custom\_eval\_function’: None, > ‘custom\_resources\_per\_worker’: { }, > ‘disable\_env\_checking’: False, > ‘eager\_max\_retraces’: 20, > ‘eager\_tracing’: False, > ‘enable\_async\_evaluation’: False, > ‘enable\_connectors’: True, > ‘enable\_tf1\_exec\_eagerly’: False, > ‘entropy\_coeff’: 0.0, > ‘entropy\_coeff\_schedule’: None, > ‘env’: \, > ‘env\_task\_fn’: None, > ‘evaluation\_config’: None, > ‘evaluation\_duration’: 10, > ‘evaluation\_duration\_unit’: ‘episodes’, > ‘evaluation\_interval’: None, > ‘evaluation\_num\_workers’: 0, > ‘evaluation\_parallel\_to\_training’: False, > ‘evaluation\_sample\_timeout\_s’: 180.0, > ‘exploration\_config’: { ‘type’: ‘StochasticSampling’}, > ‘explore’: True, > ‘export\_native\_model\_files’: True, > ‘extra\_python\_environs\_for\_driver’: { }, > ‘extra\_python\_environs\_for\_worker’: { }, > ‘fake\_sampler’: False, > ‘framework’: ‘tf’, > ‘gamma’: { ‘grid\_search’: [ 0.9, > 0.99, > 0.999]}, > ‘grad\_clip’: None, > ‘horizon’: -1, > ‘ignore\_worker\_failures’: False, > ‘in\_evaluation’: False, > ‘input’: ‘sampler’, > ‘input\_config’: { }, > ‘is\_atari’: None, > ‘keep\_per\_episode\_custom\_metrics’: False, > ‘kl\_coeff’: 0.2, > ‘kl\_target’: 0.01, > ‘lambda’: 1.0, > ‘local\_tf\_session\_args’: { ‘inter\_op\_parallelism\_threads’: 8, > ‘intra\_op\_parallelism\_threads’: 8}, > ‘log\_level’: ‘INFO’, > ‘log\_sys\_usage’: True, > ‘logger\_config’: None, > ‘logger\_creator’: None, > **‘lr’: \,** > ‘lr\_schedule’: None, > ‘max\_requests\_in\_flight\_per\_sampler\_worker’: 2, > ‘metrics\_episode\_collection\_timeout\_s’: 60.0, > ‘metrics\_num\_episodes\_for\_smoothing’: 100, > ‘min\_sample\_timesteps\_per\_iteration’: 0, > ‘min\_time\_s\_per\_iteration’: None, > ‘min\_train\_timesteps\_per\_iteration’: 0, > ‘model’: { ‘\_disable\_action\_flattening’: False, > ‘\_disable\_preprocessor\_api’: False, > ‘\_time\_major’: False, > ‘\_use\_default\_native\_models’: -1, > ‘attention\_dim’: 64, > ‘attention\_head\_dim’: 32, > ‘attention\_init\_gru\_gate\_bias’: 2.0, > ‘attention\_memory\_inference’: 50, > ‘attention\_memory\_training’: 50, > ‘attention\_num\_heads’: 1, > ‘attention\_num\_transformer\_units’: 1, > ‘attention\_position\_wise\_mlp\_dim’: 32, > ‘attention\_use\_n\_prev\_actions’: 0, > ‘attention\_use\_n\_prev\_rewards’: 0, > ‘conv\_activation’: ‘relu’, > ‘conv\_filters’: None, > ‘custom\_action\_dist’: None, > ‘custom\_model’: \, > ‘custom\_model\_config’: { }, > ‘custom\_preprocessor’: None, > ‘dim’: 84, > ‘fcnet\_activation’: ‘tanh’, > ‘fcnet\_hiddens’: [ 256, > 256], > ‘framestack’: True, > ‘free\_log\_std’: False, > ‘grayscale’: False, > ‘lstm\_cell\_size’: 256, > ‘lstm\_use\_prev\_action’: False, > ‘lstm\_use\_prev\_action\_reward’: -1, > ‘lstm\_use\_prev\_reward’: False, > ‘max\_seq\_len’: 20, > ‘no\_final\_linear’: False, > ‘post\_fcnet\_activation’: ‘relu’, > ‘post\_fcnet\_hiddens’: [], > ‘use\_attention’: False, > ‘use\_lstm’: False, > ‘vf\_share\_layers’: False, > ‘zero\_mean’: True}, > ‘multiagent’: { ‘count\_steps\_by’: ‘env\_steps’, > ‘observation\_fn’: None, > ‘policies’: { ‘default\_policy’: ( None, > None, > None, > None)}, > ‘policies\_to\_train’: None, > ‘policy\_map\_cache’: -1, > ‘policy\_map\_capacity’: 100, > ‘policy\_mapping\_fn’: \}, > ‘no\_done\_at\_end’: -1, > ‘normalize\_actions’: True, > ‘num\_consecutive\_worker\_failures\_tolerance’: 100, > ‘num\_cpus\_for\_driver’: 1, > ‘num\_cpus\_per\_trainer\_worker’: 1, > ‘num\_cpus\_per\_worker’: 1, > ‘num\_envs\_per\_worker’: 1, > ‘num\_gpus’: 0, > ‘num\_gpus\_per\_trainer\_worker’: 0, > ‘num\_gpus\_per\_worker’: 0, > ‘num\_sgd\_iter’: 2, > ‘num\_trainer\_workers’: 0, > ‘num\_workers’: 1, > ‘observation\_filter’: ‘NoFilter’, > ‘observation\_space’: None, > ‘off\_policy\_estimation\_methods’: { }, > ‘offline\_sampling’: False, > ‘ope\_split\_batch\_by\_episode’: True, > ‘optimizer’: { }, > ‘output’: None, > ‘output\_compress\_columns’: [ ‘obs’, > ‘new\_obs’], > ‘output\_config’: { }, > ‘output\_max\_file\_size’: 67108864, > ‘placement\_strategy’: ‘PACK’, > ‘policies’: { ‘default\_policy’: \}, > ‘policy\_states\_are\_swappable’: False, > ‘postprocess\_inputs’: False, > ‘preprocessor\_pref’: ‘deepmind’, > ‘recreate\_failed\_workers’: False, > ‘remote\_env\_batch\_wait\_ms’: 0, > ‘remote\_worker\_envs’: False, > ‘render\_env’: False, > ‘replay\_sequence\_length’: None, > ‘restart\_failed\_sub\_environments’: False, > ‘rl\_module\_class’: None, > ‘rl\_trainer\_class’: None, > ‘rollout\_fragment\_length’: ‘auto’, > ‘sample\_async’: False, > ‘sample\_collector’: \, > ‘sampler\_perf\_stats\_ema\_coef’: None, > ‘seed’: None, > ‘sgd\_minibatch\_size’: 5, > ‘shuffle\_buffer\_size’: 0, > ‘shuffle\_sequences’: True, > ‘simple\_optimizer’: -1, > ‘soft\_horizon’: -1, > ‘sync\_filters\_on\_rollout\_workers\_timeout\_s’: 60.0, > ‘synchronize\_filters’: True, > ‘tf\_session\_args’: { ‘allow\_soft\_placement’: True, > ‘device\_count’: { ‘CPU’: 1}, > ‘gpu\_options’: { ‘allow\_growth’: True}, > ‘inter\_op\_parallelism\_threads’: 2, > ‘intra\_op\_parallelism\_threads’: 2, > ‘log\_device\_placement’: False}, > ‘train\_batch\_size’: 10, > ‘use\_critic’: True, > ‘use\_gae’: True, > ‘validate\_workers\_after\_construction’: True, > ‘vf\_clip\_param’: 10.0, > ‘vf\_loss\_coeff’: 1.0, > ‘vf\_share\_layers’: -1, > ‘worker\_cls’: None, > ‘worker\_health\_probe\_timeout\_s’: 60, > ‘worker\_restore\_timeout\_s’: 1800} ---
**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\ **Post date:** [June 20, 2023, 7:20pm UTC](https://discuss.ray.io/t/correct-usage-of-tune-sampling-in-algorithmconfig-dicts/10254/2 "2023-06-20T19:20:32Z")
In order to make sure that tune properly detects the search dimension (for example `gamma=tune.grid_search([0.9, 0.99, 0.999])`), you should convert the AlgorithmConfig to a dict before passing it into tune. tune is not 100% compatible with the AlgorithmConfig API. So you should assemble an AlgorithmConfig and give it a couple of search spaces like you did above. Then make sure that these search spaces don’t have implications on other parameters. For example, if you search over “\_enable\_rl\_module\_api”, enabling/disabling can impact other fields than the “\_enable\_rl\_module\_api” field. After you have made sure that this is not the case for your fields (very few parameters are impacted by this. HPs like gamma or num\_sgd\_iter are not impacted.), convert the config into a dict with `to_dict()`. You can check if the resulting dict has all the search spaces you defined previously. Please let us know if anything unexpected happens in this step. Then pass the dict to tune. RLlib will convert this dict back to an AlgorithmConfig object under the hood.