Hi community, Does anyone have experience in using a custom TF model with tf.keras.layers.Embedding? My input (i.e. tf.keras.layers.Input) to the model contains some (categorical) integer values which denote, e.g., a type or a label. I want to embed such integer values with a tf.keras.layers.Embed…

Hey @klausk55 , I think it’s pretty straight forward. One of the problems, though, would be that RLlib automatically adds a one-hot preprocessor :face_with_raised_eyebrow: . We can try adding a small example to the lib, see whether we can make this work …

The embedding model code should be something like this, but I’ll still have to figure out how to avoid the pre-processor (something we would love to get rid of completely soon anyways :slight_smile: ). from ray.rllib.policy.sample_batch import SampleBatch from ray.rllib.utils.framework import try_i…

This is already using the new native keras model support :slight_smile:

Sorry @sven1977 , but where occurs here the problem of a one-hot preprocessor? Do you mean that an integer input would be preprocessed into an one-hot-vector?

[image] Custom TF model with tf.keras.layers.Embedding RLlib This is already using the new native keras model support slight_smile Ah interesting… is the use of TFModelV2 obsolete?

Not yet (and won’t be for a long time), but yes, we would like to completely obsolete this API and just accept plain keras or torch models instead (WIP). On preprocessors: RLlib “provides” (enforces) default preprocessors on some input spaces (basically flattens everything it sees, including one-h…

[image] Custom TF model with tf.keras.layers.Embedding RLlib Yes, exactly, I haven’t checked yet. I think there is a way to disable that. Okay, but as far as I know this only happens for Discrete and MultiDiscrete obs spaces, right? To be honest, I have so far just t…

Ah, yes, makes perfect sense! Didn’t think about int Boxes :smiley:

Custom TF model with tf.keras.layers.Embedding

RLlib

sven1977 May 4, 2021, 5:14pm 6

Yes, exactly, I haven’t checked yet. I think there is a way to disable that.

Topic		Replies	Views
RLLIB Custom_keras_model RLlib	1	752	July 24, 2022
[RLlib] PPO custom model only get flattened observations RLlib	5	1674	May 5, 2021
Apply preprocessor in custom model RLlib	19	2600	May 13, 2024
Impala Deep Residual (Custom) Model RLlib	2	441	November 23, 2022
Example of custom tensorflow or pytorch neural network RLlib	5	722	December 10, 2021

Custom TF model with tf.keras.layers.Embedding

Related topics