Dreamer V3 - Rllib, TensorFlow Error

1

Hello,

I am trying implemant Dreamer V3 to cartpole. My code suppose to work, but i got this error;

“ValueError: One of the parameters (<KerasVariable shape=(4, 256), dtype=float32, path=dreamer_model/vector_encoder/dense/kernel>) in the registered optimizer is not a tf.Variable!”

I think that is related DreamerV3Config. The error is caused by the relationship between DreamerV3Config and TensorFlow. But this seems very strange. DreamerV3Config is provided by rllib, so this should not contain errors.

Do you have any ideas how I can solve this problem?

Here is my code;

from ray.rllib.algorithms.dreamerv3.dreamerv3 import DreamerV3Config
from ray.tune.logger import pretty_print

config = (
DreamerV3Config()
.environment(“CartPole-v1”)
.training(
model_size=“XS”,
training_ratio=1024,
)
)

algo = config.build()

for i in range(10):
result = algo.train()
print(pretty_print(result))

if i % 5 == 0:
checkpoint_dir = algo.save().checkpoint.path
print(f"Checkpoint saved in directory {checkpoint_dir}")

3

Since tf 2.16 if you want to use keras 2 you have to set the environment variable TF_USE_LEGACY_KERAS=1 or you can add the following lines of code:

import os
os.environ[“TF_USE_LEGACY_KERAS”]=”1”

Important: you have to set this variable before any import of tensorflow, otherwise it will not work.