Adding custom metrics to policy

RLlib
#1

Hi,

I am writing a custom policy and want to add some metrics like this.

self.stats = {
                "mean_q": tf.reduce_mean(q_t_selected)
}

The problem is, if it is an int or a float value, for example

some_value = 10
some_list = [0, 0.5, 0.99]
self.stats = {
                "some_value ": some_value 
                "some_list ": some_list 
}

I get an error like this

File "/home/abc/miniconda3/envs/env_hdqn/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3814, in _as_graph_element_locked
raise TypeError("Can not convert a %s into a %s." %
 TypeError: Can not convert a float into a Tensor or Operation.`


File "/home/abc/miniconda3/envs/env_hdqn/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 309, in __init__
raise TypeError('Fetch argument %r has invalid type %r, '
TypeError: Fetch argument 0.0 has invalid type <class 'float'>, must be a string or Tensor. (Can not convert a float into a Tensor or Operation.)

I have been successful at adding a Tensor to this dict, but not an int or float.

I have looked at the custom_metrics_and_callbacks and this post here but this seems a little complicated for something I want. Is there a simpler way? Why is adding a value to the dict throwing this error?

Thanks you.

#2

I was able to solve this issue by casting the custom metric into its appropriate type. So e.g.

some_value = 10
some_string = 'abc'
some_list = [0, 0.5, 0.99]
self.stats = {
                "some_value ": some_value, 
                "some_list ": some_list,
                "some_string ": some_string,
}

I first tried using this method as the error mentioned it must be a string or Tensor.

"some_value ": tf.strings.as_string(some_value)

which at least solved the issue but the resulting stats were byte encoded strings e.g. b'some_value'
'[b''0.000000'' b''0.500000'' b''0.99'']'

Eventually, I saw in another rllib policy that values are being casted, so I did as below

some_value = 10
some_string = 'abc'
some_list = [0, 0.5, 0.99]
self.stats = {
                "some_value ": tf.cast(some_value, tf.int64),
                "some_list ": tf.cast(some_list, tf.float64),,
                "some_string ": tf.cast(some_string, tf.string),,
}

which gave me the stats I was looking for.

1 Like