# Trainable class from SACConfig.build()?

**URL:** https://discuss.ray.io/t/trainable-class-from-sacconfig-build/9249
**Category:** Ray Tune stopping condition & comparisons
**Created:** [February 5, 2023, 5:50am UTC](https://discuss.ray.io/t/trainable-class-from-sacconfig-build/9249 "2023-02-05T05:50:13Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Teenforever](https://avatars.discourse-cdn.com/v4/letter/t/e9a140/32.png) [@Teenforever](https://discuss.ray.io/u/Teenforever)
#### Post date: [February 5, 2023, 5:50am UTC](https://discuss.ray.io/t/trainable-class-from-sacconfig-build/9249/1 "2023-02-05T05:50:13Z")

</div>

I am in the process of migrating a Tensorflow SAC trainer written from scratch to RLLIB, but struggling with some ideas in RLLIB. I try to use Ray Tune for a SAC algorithm and plugged in some config parameters like:

```auto
register_env("Env", Env)
sac_config = SACConfig().environment("Env")
sac_config = sac_config.training(
    clip_actions = True,
    gamma = 0.99,
    optimization_config = {
        "actor_learning_rate": 0.01,
        "critic_learning_rate": 0.01,
        "entropy_learning_rate": 0.01,
    },
    policy_model_config = policy_model_config,
    q_model_config = q_model_config,
    store_buffer_in_checkpoints = True,
    target_network_update_freq = 1,
    tau = 0.005,
    train_batch_size = 256,
    twin_q = True,
)

```

the I run:

`sac = sac_config.build()`

I first thought `sac` would be a trainable class that Tune can take in under trainable class API, since build function in Algorithm\_config ([ray/algorithm\_config.py at master · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/master/rllib/algorithms/algorithm_config.py)) spits out an Algorithm, and Algorithm seems to be a subclass of Trainable ([ray/algorithm.py at master · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/master/rllib/algorithms/algorithm.py)). So as the next step I start Tune:

```auto
tuner = tune.Tuner(sac, 
    param_space = param_space, #for now, param_space is just a grid search of tau 
    tune_config = TuneConfig(
        scheduler = ASHAScheduler(metric = "agg_reward", mode = "max"), #agg_reward is reported in step of environment
        num_samples = 1,
        ),
    run_config = RunConfig(
        stop = {"training_iteration": 100},
        local_dir = ospath.join(ospath.dirname( __file__ ),"ray_results"),
        verbose = 1,
        log_to_file = True,
        checkpoint_config = CheckpointConfig(
            checkpoint_frequency = 10
            ),
        ),
    )

```

Then the error pops up:  
` raise TuneError("Improper 'run' - not string nor trainable.")`

If I register sac as a trainable via `register_trainable("sac_trainable", sac())`, it says `TypeError: 'SAC' object is not callable`.

I suspect Tune is telling me that `sac` is not a trainable class, but if not, what is the type of object that `build` blurts out? Could someone please point out what went wrong in the above and how I can make Tune work? Thank you!

---

<div class="post-metadata">

### Author: ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [February 5, 2023, 12:23pm UTC](https://discuss.ray.io/t/trainable-class-from-sacconfig-build/9249/2 "2023-02-05T12:23:14Z")

</div>

Hi @Teenforever

> [@Teenforever](#):
>
> `register_trainable("sac_trainable", sac())`,

I think this is telling you it should look like this:  
`register_trainable("sac_trainable", sac)`

In your case I think the more canonical approach would be to put this in the sac\_config:  
`tau=tune.grid_search([...])`

Do not build but run like this:

```auto
tuner = tune.Tuner("SAC",
    param_space = sac_config, 
   ... 

```

---

<div class="post-metadata">

### Author: ![Teenforever](https://avatars.discourse-cdn.com/v4/letter/t/e9a140/32.png) [@Teenforever](https://discuss.ray.io/u/Teenforever)
#### Post date: [February 6, 2023, 1:55pm UTC](https://discuss.ray.io/t/trainable-class-from-sacconfig-build/9249/3 "2023-02-06T13:55:36Z")

</div>

Works like a breeze, thanks @mannyv .

For whoever in the future wondering if it’s necessary to convert the sac\_config to a dict using .dict() - I tried and this isn’t necessary. In fact, if one tries to convert the config to a dict and if the config contains gridsearch (and probably other tune instructions), it blurts out an error.
