Error restoring a Policy

RLlib Checkpointing, Restoring
1

Hi!

I’m trying to restore the trained policy of a tune experiment to use in an real scenario where the obs space comes from instrumentation and not from a simulated scenarios as I used to training the policy.

During training I saved the algorithm with the policies inside the directory as the documentation suggest. I found without difficulties the path to policy with the two files (.pkl and .json), but when I tried to restore the policy a ValueError appears:

Traceback (most recent call last):
  File "c:\Users\grhen\Documents\GitHub\EP_RLlib\GENERAL_QMIX_init_evaluation.py", line 6, in <module>   
    policy = Policy.from_checkpoint(checkpoint_path)
  File "C:\Users\grhen\anaconda3\envs\dwelling_DRL\lib\site-packages\ray\rllib\policy\policy.py", line 345, in from_checkpoint
    return Policy.from_state(state)
  File "C:\Users\grhen\anaconda3\envs\dwelling_DRL\lib\site-packages\ray\rllib\policy\policy.py", line 365, in from_state
    raise ValueError(
ValueError: No `policy_spec` key was found in given `state`! Cannot create new Policy.

My code is very simple:

from ray.rllib.policy.policy import Policy

checkpoint_path = "C:/Users/grhen/ray_results/ajuste_modelo_general_QMIX_5/QMIX_EPEnv_24cb3_00005_5_gamma=0.9900,lr=0.0100,mixing_embed_dim=32_2023-09-17_09-38-46/checkpoint_001400/policies/default_policy"

policy = Policy.from_checkpoint(checkpoint_path)

print(policy)
print(policy.get_weights())

How can I fix this error? How to do to specify a policy_spec in the algorithm config to be save in the checkpointing proccess? Is it something in my code that I used for training and checkpointing or is it a bug?