Hi everyone,
I was running this script which builds a Python List of ray.tune.Experiment on line number 453 and calls ray.tune.run_experiments() on line number 574.
But, while running the script, I am getting the below error from ray.tune.Experiment.from_json() function at line number 356.
Warning: grid search will not be performed; using lr and optimizer from config...
Metal device set to: Apple M1 Pro
systemMemory: 16.00 GB
maxCacheSize: 5.33 GB
Using config:
run: PPO
checkpoint_freq: 5
checkpoint_at_end: True
stop
timesteps_total: 250000
config
opt_type: adam
lr: 0.0005
gamma: 0.98
num_workers: 5
num_envs_per_worker: 2
num_gpus: 1
observation_filter: NoFilter
sample_batch_size: 205
train_batch_size: 4000
sgd_minibatch_size: 200
num_sgd_iter: 30
batch_mode: truncate_episodes
fetch_lstm_gates: False
model
use_lstm: True
max_seq_len: 10000
slot1: lstm
slot2: avg
sum_instead: False
max_instead: True
straight_through: True
env_config
check_up: 1
check_down: -1
maze_length_upper_bound: None
pos_enc: False
wave_encoding_len: None
task_switch_after: None
intermediate_checks: False
intermediate_indicators: True
reset_intermediate_indicators: True
per_step_reset: True
num_indicators_components: 1
frac_correct_components_for_check: 1
final_intermediate_indicator: True
check_reward: 0.1
reward_per_correct_component: False
allow_left: False
force_final_decision: True
force_right: True
timeout: 150
timeout_reward: 0
maze_length: 100
indicator_pos: 0
flipped_indicator_pos: None
correlated_indicator_pos: None
success_reward: 4.0
fail_reward: -3.0
persistent_reward: 0.0
callbacks
on_episode_end: <function prep_for_saving_config_and_callback_data.<locals>.<lambda> at 0x2a6972790>
Trial_Name: T-LN_EXAMPLE_model-AMRL-Max_RUN1
local_dir: /Users/sid/rl_attention/AMRL_results/data
env: tmaze-v0
Traceback (most recent call last):
File "/Users/sid/miniconda3/envs/rl/lib/python3.9/site-packages/ray/tune/experiment/experiment.py", line 304, in from_json
exp = cls(name, run_value, **spec)
TypeError: __init__() got an unexpected keyword argument 'checkpoint_freq'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/sid/rl_attention/maze_runner.py", line 529, in <module>
main()
File "/Users/sid/rl_attention/maze_runner.py", line 523, in main
all_experiments = get_experiments(yaml_path_env_path_tups, save_dirs_successful, args)
File "/Users/sid/rl_attention/maze_runner.py", line 455, in get_experiments
new_experiment = get_experiment_dict_from_yaml(yaml_path, run_num, clust=args.clust, existing_save_dirs=save_dirs_successful,
File "/Users/sid/rl_attention/maze_runner.py", line 437, in get_experiment_dict_from_yaml
return tune.Experiment.from_json(experiment_name, d)
File "/Users/sid/miniconda3/envs/rl/lib/python3.9/site-packages/ray/tune/experiment/experiment.py", line 306, in from_json
raise TuneError(
ray.tune.error.TuneError: Failed to load the following Tune experiment specification:
{'checkpoint_at_end': True,
'checkpoint_freq': 5,
'config': {'batch_mode': 'truncate_episodes',
'callbacks': {'on_episode_end': <function prep_for_saving_config_and_callback_data.<locals>.<lambda> at 0x2a6972790>},
'env': 'tmaze-v0',
'env_config': {'allow_left': False,
'check_down': -1,
'check_reward': 0.1,
'check_up': 1,
'correlated_indicator_pos': None,
'fail_reward': -3.0,
'final_intermediate_indicator': True,
'flipped_indicator_pos': None,
'force_final_decision': True,
'force_right': True,
'frac_correct_components_for_check': 1,
'indicator_pos': 0,
'intermediate_checks': False,
'intermediate_indicators': True,
'maze_length': 100,
'maze_length_upper_bound': None,
'num_indicators_components': 1,
'per_step_reset': True,
'persistent_reward': 0.0,
'pos_enc': False,
'reset_intermediate_indicators': True,
'reward_per_correct_component': False,
'success_reward': 4.0,
'task_switch_after': None,
'timeout': 150,
'timeout_reward': 0,
'wave_encoding_len': None},
'fetch_lstm_gates': False,
'gamma': 0.98,
'lr': 0.0005,
'model': {'max_instead': True,
'max_seq_len': 10000,
'slot1': 'lstm',
'slot2': 'avg',
'straight_through': True,
'sum_instead': False,
'use_lstm': True},
'num_envs_per_worker': 2,
'num_gpus': 1,
'num_sgd_iter': 30,
'num_workers': 5,
'observation_filter': 'NoFilter',
'opt_type': 'adam',
'sample_batch_size': 205,
'sgd_minibatch_size': 200,
'train_batch_size': 4000},
'local_dir': '/Users/sid/rl_attention/AMRL_results/data',
'stop': {'timesteps_total': 250000},
'trial_name_creator': <function get_experiment_dict_from_yaml.<locals>.<lambda> at 0x2a6972670>}.
Please check that the arguments are valid. Experiment creation failed with the following error:
__init__() got an unexpected keyword argument 'checkpoint_freq'
I also logged the Python Dictionary I am passing as spec parameter to the ray.tune.Experiment.from_json() function. Please find below the same.
{'run': 'PPO', 'checkpoint_freq': 5, 'checkpoint_at_end': True, 'stop': {'timesteps_total': 250000}, 'config': {'opt_type': 'adam', 'lr': 0.0005, 'gamma': 0.98, 'num_workers': 5, 'num_envs_per_worker': 2, 'num_gpus': 0, 'observation_filter': 'NoFilter', 'sample_batch_size': 205, 'train_batch_size': 4000, 'sgd_minibatch_size': 200, 'num_sgd_iter': 30, 'batch_mode': 'truncate_episodes', 'fetch_lstm_gates': False, 'model': {'use_lstm': True, 'max_seq_len': 10000, 'slot1': 'lstm', 'slot2': 'avg', 'sum_instead': False, 'max_instead': True, 'straight_through': True}, 'env_config': {'check_up': 1, 'check_down': -1, 'maze_length_upper_bound': None, 'pos_enc': False, 'wave_encoding_len': None, 'task_switch_after': None, 'intermediate_checks': False, 'intermediate_indicators': True, 'reset_intermediate_indicators': True, 'per_step_reset': True, 'num_indicators_components': 1, 'frac_correct_components_for_check': 1, 'final_intermediate_indicator': True, 'check_reward': 0.1, 'reward_per_correct_component': False, 'allow_left': False, 'force_final_decision': True, 'force_right': True, 'timeout': 150, 'timeout_reward': 0, 'maze_length': 100, 'indicator_pos': 0, 'flipped_indicator_pos': None, 'correlated_indicator_pos': None, 'success_reward': 4.0, 'fail_reward': -3.0, 'persistent_reward': 0.0}}, 'local_dir': '/home/siddhantsahu/Desktop/rl/POMDP-RL/AMRL_results/data', 'env': 'tmaze-v0'}
I was wondering what might be missing here? What is correct way to create an object of ray.tune.Experiment with the help of it’s from_json() method?