# Offline data example

**URL:** <https://discuss.ray.io/t/offline-data-example/9721>\
**Category:** Offline RL\
**Created:** [March 10, 2023, 4:00pm UTC](https://discuss.ray.io/t/offline-data-example/9721 "2023-03-10T16:00:28Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![asdfg](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@asdfg](https://discuss.ray.io/u/asdfg)\
**Post date:** [March 10, 2023, 4:00pm UTC](https://discuss.ray.io/t/offline-data-example/9721/1 "2023-03-10T16:00:28Z")

</div>

Hello,

Is there a complete example of how to use Offline data using Tensorflow? The example shown below only seems to work with PyTorch. Also are there any examples using any of the offline RL algorithms such as BC or CRR?

```auto
from ray.rllib.algorithms.dqn import DQNConfig
from ray.rllib.offline.estimators import (
    ImportanceSampling,
    WeightedImportanceSampling,
    DirectMethod,
    DoublyRobust,
)
from ray.rllib.offline.estimators.fqe_torch_model import FQETorchModel

config = (
    DQNConfig()
    .environment(env="CartPole-v1")
    .framework("torch")
    .offline_data(input_="/tmp/cartpole-out")
    .evaluation(
        evaluation_interval=1,
        evaluation_duration=10,
        evaluation_num_workers=1,
        evaluation_duration_unit="episodes",
        evaluation_config={"input": "/tmp/cartpole-eval"},
        off_policy_estimation_methods={
            "is": {"type": ImportanceSampling},
            "wis": {"type": WeightedImportanceSampling},
            "dm_fqe": {
                "type": DirectMethod,
                "q_model_config": {"type": FQETorchModel, "polyak_coef": 0.05},
            },
            "dr_fqe": {
                "type": DoublyRobust,
                "q_model_config": {"type": FQETorchModel, "polyak_coef": 0.05},
            },
        },
    )
)

algo = config.build()
for _ in range(100):
    algo.train()

```

Thanks!

---

<div class="post-metadata">

**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\
**Post date:** [April 13, 2023, 11:29pm UTC](https://discuss.ray.io/t/offline-data-example/9721/2 "2023-04-13T23:29:02Z")

</div>

Hi @asdfg,

The follwing code is working for me:

```auto
from ray.rllib.algorithms.dqn import DQNConfig
from ray.rllib.offline.estimators import (
    ImportanceSampling,
    WeightedImportanceSampling,
    DirectMethod,
    DoublyRobust,
)
from ray.rllib.offline.estimators.fqe_torch_model import FQETorchModel

config = (
    DQNConfig()
    .environment(env="CartPole-v1")
    .framework("torch")
    .offline_data(input_=<path>)
    .evaluation(
        evaluation_interval=1,
        evaluation_duration=10,
        evaluation_num_workers=1,
        evaluation_duration_unit="episodes",
        off_policy_estimation_methods={
            "is": {"type": ImportanceSampling},
            "wis": {"type": WeightedImportanceSampling},
            "dm_fqe": {
                "type": DirectMethod,
                "q_model_config": {"type": FQETorchModel, "polyak_coef": 0.05},
            },
            "dr_fqe": {
                "type": DoublyRobust,
                "q_model_config": {"type": FQETorchModel, "polyak_coef": 0.05},
            },
        },
    )
)

algo = config.build()
for _ in range(100):
    algo.train()

```

What version are you on?  
Please try with the latest release.

---

<div class="post-metadata">

**Author:** ![asdfg](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@asdfg](https://discuss.ray.io/u/asdfg)\
**Post date:** [April 14, 2023, 1:35am UTC](https://discuss.ray.io/t/offline-data-example/9721/3 "2023-04-14T01:35:18Z")

</div>

Hello,

This does not seem to work with TF for me.

---

<div class="post-metadata">

**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\
**Post date:** [April 14, 2023, 9:47pm UTC](https://discuss.ray.io/t/offline-data-example/9721/4 "2023-04-14T21:47:27Z")

</div>

@asdfg . Got it. We are not testing this with `tf` because it’s not supported.  
Thanks for reporting this. We are not supporting to support this for `tf` for now.  
I’m opening a PR that throws a more informative error here.

---

<div class="post-metadata">

**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\
**Post date:** [April 14, 2023, 9:54pm UTC](https://discuss.ray.io/t/offline-data-example/9721/5 "2023-04-14T21:54:23Z")

</div>

> <https://github.com/ray-project/ray/pull/34417>
>
> \## Why are these changes needed?
> 
> User is running into issues when trying to r…un DirectMethod OPE with "tf".
> https://discuss.ray.io/t/offline-data-example/9721/2
> Error is not informative