How can I restore a trained model for predictions without having the environment?
cfg = { "env": MyEnv }
trainer = ppo.APPOTrainer(config=cfg)
trainer.restore("APPOTrainer_MyEnv_2022-07-23_15-23-47xehu_jbw\\checkpoint_010895\\checkpoint-10895")
If I don’t specify the environment I get an error
@evo11x that should work by using a proper configuration of your trainer:
from ray.rllib.agents.ppo.appo import DEFAULT_CONFIG
config = DEFAULT_CONFIG.copy()
config["env"] = MyEnv
trainer = ppo.APPOTrainer(config=config)
trainer.restore(checkpoint_path)
However, it is adviced to use tune to run your experiments. With tune you also have the ExperimentAnalysis class to load the expriment configuration from your experiment and use this to build your trainer.
For a better understanding of your problem I suggest you to also always add the exact error messages to your posts. That helps us to find the source code that errors out.
@Lars_Simon_Zehnder Thanks! This is the error which I get if I remove the env from config
ValueError: `observation_space` not provided in PolicySpec for default_policy and env does not have an observation space OR no spaces received from other workers' env(s) OR no `observation_space` specified in config!
I want to use the trained model for prediction without having the environment class, is this possible?
Hi @evo11x,
If you are not providing an environment then you need to provide the observation and action spaces in the config.
It uses those to figure out the shapes for the model.
RLlib 2.0 will have support Connectors, which makes restoring policy checkpoints much easier.
for a sneak peak, check out the examples here ray/rllib/examples/connectors at master · ray-project/ray · GitHub
Checkout the cartpole server example’s config here:
You need to set:
config = {
# Indicate that the Algorithm we setup here doesn't need an actual env.
"env": None,
"observation_space": <obs-space>,
"action_space": <action-space>,
# ...
}