# Roll out CQL policy

**URL:** <https://discuss.ray.io/t/roll-out-cql-policy/4216>\
**Category:** RLlib\
**Created:** [November 23, 2021, 12:49pm UTC](https://discuss.ray.io/t/roll-out-cql-policy/4216 "2021-11-23T12:49:05Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [November 24, 2021, 6:47pm UTC](https://discuss.ray.io/t/roll-out-cql-policy/4216/6 "2021-11-24T18:47:51Z")

</div>

Hi @fksvensson ,

I guess that the Python version might be the problem. I would use `pyenv` (if you are on MacOS-X you can use `brew` to install it. Pull Python 3.9 and then use

```auto
pyenv install 3.9.4
mkdir ray-1.8.0 
cd ray-1.8.0
pyenv local 3.9.4
python -m venv -venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install numpy tensorflow
python -m pip install "ray[default]" 

```

and you should be good to go. ThatÄs the setup I use, so I can choose python versions and install with these versions the packages in a virtual env.

Regarding your problem:

1. Have you registered your input reader via `register_input("custom_input", input_creator)`? before requesting it in your `config`?
2. Have you tested your input reader? Does it really read in the samples and returns a `SampleBatch` as needed?
3. How did you write your ouputs? Did you use a specific ouput writer you coded yourself or did you use the default?
4. Do you actually need the input or can you deploy your policy via running the environment in the background?

I actually do not understand why you include all the training parameters when you want to do evaluation. Evaluation can be done quite easily by running

```auto
rllib rollout \
    ~/ray_results/default/DQN_CartPole-v0_0upjmdgr0/checkpoint_1/checkpoint-1 \
    --run DQN --env CartPole-v0 --steps 10000

```

So you use your checkpoint then the policy (`DQN` here) then your `env` and the number of steps you want to evaluate. The workers sample experiences from the environment using the trained policy - there is no need for an input sampler. You can also do custom evaluation as shown in this [example](https://github.com/ray-project/ray/blob/master/rllib/examples/custom_eval.py) here.

If you still need something else, you can post your code here and we can take a look at it.

Best, Simon

---

_[View the full topic](https://discuss.ray.io/t/roll-out-cql-policy/4216)._