TF eager error (Executing eagerly)

RLlib Configure Algorithm, Training, Evaluation, Scaling
1

Hello,

I have trained an agent in customized environment and now when I restore the check points, I get this warning and it can not load the check points appropriately:

WARNING eager_tf_policy_v2.py:695 – Cannot restore an optimizer’s state for tf eager! Keras is not able to save the v1.x optimizers (from tf.compat.v1.train) since they aren’t compatible with checkpoints.

It should be noted that when I train the agent I got this warning:

INFO algorithm_config.py:2492 – Executing eagerly (framework=‘tf2’), with eager_tracing=tf2. For production workloads, make sure to set eager_tracing=True in order to match the speed of tf-static-graph (framework=‘tf’). For debugging purposes, eager_tracing=False is the best choice.

However, I had selected framework=‘tf2’ and eager_tracing=True in configuration dict.

Could you help me?

2

Hey @AI360 , could you give us a small reproduction script that shows the error?
Also, we would need your RLlib/Ray version information and python version.
Thanks! :slight_smile:

4

Hello,

class GDGR(gym.Env):
def init(self,env_config):
…
def reset(self):
…
def step(self, action):
…

…
.
ray.init(num_gpus=1,num_cpus = 4)
config = DEFAULT_CONFIG.copy()

.
.
.

register_env(“GDGR”, lambda config …)
.
.
.
config[“soft_horizon”] = 2048
config[“framework”] = “tf2”
.
.
.
.
Trainer = ppo.PPOTrainer(config=config)

.
.
.
.
Training process …

For loading:

I used this:

algo = Algorithm.from_checkpoint(path)

Python 3.10
Ray 2.2
tensorflow = 2.11.0

Thanks,