Error when loading and restoring a trained algorithm from a checkpoint using a APPO Algorithm

RLlib
1

Hello everybody,
I trained a RL Model using a .train() method and I saved the results in a folder with the .save() method.

Then I try to restore with:

from ray.rllib.algorithms.algorithm import Algorithm
algo_2 = Algorithm.from_checkpoint(checkpoint_path_2)

but the error I got is:

TypeError                                 Traceback (most recent call last)
Cell In [14], line 2
      1 from ray.rllib.algorithms.algorithm import Algorithm
----> 2 algo_2 = Algorithm.from_checkpoint(checkpoint_path_2)

File ~\Anaconda3\lib\site-packages\ray\rllib\algorithms\algorithm.py:270, in Algorithm.from_checkpoint(checkpoint, policy_ids, policy_mapping_fn, policies_to_train)
    263 if checkpoint_info["checkpoint_version"] < version.Version("1.0"):
    264     raise ValueError(
    265         "`checkpoint_info['checkpoint_version']` in `Algorithm.from_checkpoint"
    266         "()` must be 1.0 or later! You are using a checkpoint with "
    267         f"version v{checkpoint_info['checkpoint_version']}."
    268     )
--> 270 state = Algorithm._checkpoint_info_to_algorithm_state(
    271     checkpoint_info=checkpoint_info,
    272     policy_ids=policy_ids,
    273     policy_mapping_fn=policy_mapping_fn,
    274     policies_to_train=policies_to_train,
    275 )
    277 return Algorithm.from_state(state)

File ~\Anaconda3\lib\site-packages\ray\rllib\algorithms\algorithm.py:2857, in Algorithm._checkpoint_info_to_algorithm_state(checkpoint_info, policy_ids, policy_mapping_fn, policies_to_train)
   2851 worker_state["filters"] = {
   2852     pid: filter
   2853     for pid, filter in worker_state["filters"].items()
   2854     if pid in policy_ids
   2855 }
   2856 # Remove policies from multiagent dict that are not in `policy_ids`.
-> 2857 policies_dict = state["config"]["multiagent"]["policies"]
   2858 policies_dict = {
   2859     pid: spec for pid, spec in policies_dict.items() if pid in policy_ids
   2860 }
   2861 state["config"]["multiagent"]["policies"] = policies_dict

TypeError: 'PPOConfig' object is not subscriptable
2

Hi @Giovanni_Febbraro,

Welcome to the forums.

state[“config”] is not a dictionary it is an AlgorithmConfig object.