# ERROR when SEARCH ALGORITHM is used in SCHEDULER (pb2/ asha)) during Tuning

**URL:** <https://discuss.ray.io/t/error-when-search-algorithm-is-used-in-scheduler-pb2-asha-during-tuning/9198>\
**Category:** Configure Algorithm, Training, Evaluation, Scaling\
**Created:** [February 2, 2023, 12:04am UTC](https://discuss.ray.io/t/error-when-search-algorithm-is-used-in-scheduler-pb2-asha-during-tuning/9198 "2023-02-02T00:04:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alberto-Espinosa](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/alberto-espinosa/32/3820_2.png) [@Alberto-Espinosa](https://discuss.ray.io/u/Alberto-Espinosa)\
**Post date:** [February 2, 2023, 12:04am UTC](https://discuss.ray.io/t/error-when-search-algorithm-is-used-in-scheduler-pb2-asha-during-tuning/9198/1 "2023-02-02T00:04:25Z")

</div>

inspired on **PPO+PB2** [example](https://docs.ray.io/en/latest/tune/examples/includes/pb2_ppo_example.html)  
I set in **run.config** section the _sample\_from_s as in example.

I mean, the following code section:

```auto
"lambda": sample_from(lambda spec: random.uniform(0.9, 1.0)),
"clip_param": sample_from(lambda spec: random.uniform(0.1, 0.5)),
"lr": sample_from(lambda spec: random.uniform(1e-3, 1e-5)),
"train_batch_size": sample_from(lambda spec: random.randint(1000,

```

But as in ray 2.2.0 I cannot construct a algorithm from a dictionary, but using, for example `ppo.PPOConfig.from_dict(my_config_dict)` which throw an error like `ValueError: invalid literal for int() with base 10: 'grid_search'`

I tested the code with a custom environment and with a gym environment, using DQN and PPO algorithms, creating algorithm using `from_dict(my_config_dict)` and as previous version, directly with a configuration as dictionary.

The error appears only when include the search Algorithms and/or the `sample_from` method

Here my script.  
Have to say that because copyrights I cannot share my environment implementation.

I’m really grateful guys for your extraordinary work developing ray.  
I cannot wait to contribute if I could

```auto

import ray
from ray.tune.schedulers.pb2 import PB2
from ray.tune.schedulers import ASHAScheduler
from ray.tune import run, sample_from
import random
from cerri_memory.settings import *
from cerri_memory.environment import Environment
from ray import tune
import ray
from ray.rllib.algorithms import dqn, ppo

##########################################
# SETTINGS SECTION
##########################################
env_name = "cerri_env" # "Walker2d-v4" | "cerri_env"
algo_name = "PPO" # "DQN" | "PPO"
scheduler = "pb2" # "PB2" | "ASHAScheduler"
include_samplings = True
include_fcnet_hiddens_config = True
use_config_as_dict = True
##########################################

##########################################
# CREATING CONFIGURATION
##########################################
if env_name == "cerri_env":
    env_config = {
                "algo_name": "DQN",
                "abst_experiment_pkl_path": "/SERIALIZED_EXPERIMENT/",
                "stage": "Tuning",
                "verbose": verbose,
                "sort_training": sort_training,
                "day_repetitions_limit": day_repetitions_limit,
                "num_episodes_for_avg_reward": num_episodes_for_avg_reward,
                "validation_type": validation_type, 
                "finishing_episode_mode": finishing_episode_mode, 
                "validate_action_during_testing": validate_action_during_testing,
            }

else:
    env_config = {}

FC_SIZE = ray.tune.grid_search([[256, 256], [1024], [128, 64, 32]])
CONFIG = {
            "env": env_name,
            "env_config": env_config,
            "framework_str" : "tf2",
            "disable_env_checking":True,
        }

if include_fcnet_hiddens_config:
    CONFIG["model"] = {
        "fcnet_hiddens": FC_SIZE
    }

##########################################
# SET ALGORITHM'S CONFIGURATION
##########################################
if algo_name == "DQN":
    default_config_dict = dqn.DQNConfig().to_dict()

    if include_samplings:
        CONFIG["v_min"] = sample_from(lambda spec: random.randint(0, 0))
        CONFIG["v_max"] = sample_from(lambda spec: random.randint(1, 1))
    
    hyp_bounds = {
        "v_min": [0, 0],
        "v_max": [1, 1],
    }
else:
    default_config_dict = ppo.PPOConfig().to_dict()
    hyp_bounds = {}

if include_samplings:
    CONFIG["lr"]= sample_from(lambda spec: random.uniform(1e-3, 1e-5))
hyp_bounds["lr"] = [1e-3, 1e-5]

###########################################
# UPDATES DEFAULT CONFIG AND CREATE ALGORITHM
##########################################
for k, v in CONFIG.items():
    default_config_dict[k] = v

if use_config_as_dict is False:
    if algo_name == "DQN":
        algorithm_obj = dqn.DQNConfig.from_dict(default_config_dict)
    else:
        algorithm_obj = ppo.PPOConfig.from_dict(default_config_dict)
else:
    algorithm_obj = default_config_dict

# IF CUSTOM ENVIRONMENT, THEN REGISTER IT
if env_name == "cerri_env":
    environment = Environment(default_config_dict)
    tune.register_env("cerri_env", lambda config: environment)

##########################################
# SET SCHEDULER SECTION
##########################################
pb2 = PB2(
            time_attr="episode_reward_mean",
            metric="episode_reward_mean",
            mode="max",
            perturbation_interval=500,
            quantile_fraction=0.25, # copy bottom % with top %
            # Specifies the hyperparam search space
            hyperparam_bounds=hyp_bounds
            )

asha_scheduler = ASHAScheduler(
            time_attr='training_iteration',
            metric='episode_reward_mean',
            mode='max',
            max_t=100,
            grace_period=10,
            reduction_factor=3,
            brackets=1)

if scheduler == "pb2":
    scdlr = pb2
elif scheduler == "ASHAScheduler":
    scdlr = asha_scheduler

analysis = ray.tune.tune.run(
                algo_name,
                scheduler=scdlr,
                verbose=1,
                num_samples=4,
                stop={"timesteps_total": 500},
                config=algorithm_obj,
            )

```

---

<div class="post-metadata">

**Author:** ![Huaiwei\_Sun](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/huaiwei_sun/32/1316_2.png) [@Huaiwei\_Sun](https://discuss.ray.io/u/Huaiwei_Sun)\
**Post date:** [February 2, 2023, 5:31am UTC](https://discuss.ray.io/t/error-when-search-algorithm-is-used-in-scheduler-pb2-asha-during-tuning/9198/2 "2023-02-02T05:31:08Z")

</div>

cc: @avnishn @kourosh for thoughts

---

<div class="post-metadata">

**Author:** ![kourosh](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kourosh/32/2130_2.png) [@kourosh](https://discuss.ray.io/u/kourosh)\
**Post date:** [February 10, 2023, 8:52am UTC](https://discuss.ray.io/t/error-when-search-algorithm-is-used-in-scheduler-pb2-asha-during-tuning/9198/3 "2023-02-10T08:52:36Z")

</div>

Hey @Alberto-Espinosa , why do you use config dictionaries, instead of directly using config objects? Instead of creating a flat dict and then converting it to a config object you can directly construct the config object. If you can reproduce the issue in a simplified script with a few lines of code, I would be able to provide better guidelines.