# Confused by output of \`compute\_log\_likelihoods\`

**URL:** <https://discuss.ray.io/t/confused-by-output-of-compute-log-likelihoods/5565>\
**Category:** RLlib\
**Created:** [March 28, 2022, 7:03am UTC](https://discuss.ray.io/t/confused-by-output-of-compute-log-likelihoods/5565 "2022-03-28T07:03:36Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![jwarley](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jwarley/32/2328_2.png) [@jwarley](https://discuss.ray.io/u/jwarley)\
**Post date:** [March 28, 2022, 7:03am UTC](https://discuss.ray.io/t/confused-by-output-of-compute-log-likelihoods/5565/1 "2022-03-28T07:03:36Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

**tl;dr:** the action probabilities returned by `compute_log_likelihoods` do not resemble the actual behavior of the agent when sampling actions.

**Long:** I’m trying to extract a tabular policy from a trained DQN agent by querying the action probabilities at every possible state. After I restore the trained agent, calling `compute_log_likelihoods` on its policy over the whole action space gives me something resembling a uniform distribution, which is unexpected since an optimal policy in my case should be deterministic. Indeed, when I run the restored agent, it performs well and is clearly running something close to an optimal policy. So the output of `compute_log_likelihoods` clearly doesn’t describe what I think it does. What does it describe and how do I compute the probability of an action?

Here is the all code I’m running:

> <https://gist.github.com/jwarley/3f7501ccc4a86e6ee9ffa5ad71ecc995>

Many thanks!