# Does Ray Serve support local model hot update/reload?

**URL:** <https://discuss.ray.io/t/does-ray-serve-support-local-model-hot-update-reload/6718>\
**Category:** Ray Serve\
**Created:** [July 5, 2022, 2:53am UTC](https://discuss.ray.io/t/does-ray-serve-support-local-model-hot-update-reload/6718 "2022-07-05T02:53:21Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Japson](https://avatars.discourse-cdn.com/v4/letter/j/e19adc/32.png) [@Japson](https://discuss.ray.io/u/Japson)\
**Post date:** [July 5, 2022, 2:53am UTC](https://discuss.ray.io/t/does-ray-serve-support-local-model-hot-update-reload/6718/1 "2022-07-05T02:53:21Z")

</div>

That is, I can smoothly update the deployed models to the latest version without having to stop the online service. And supports rollback operations

Just like Tensorflow Serving

---

<div class="post-metadata">

**Author:** ![Sihan\_Wang](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sihan_wang/32/2570_2.png) [@Sihan\_Wang](https://discuss.ray.io/u/Sihan_Wang)\
**Post date:** [July 5, 2022, 3:35am UTC](https://discuss.ray.io/t/does-ray-serve-support-local-model-hot-update-reload/6718/2 "2022-07-05T03:35:22Z")

</div>

Hi Japson,

Currently ray serve doesn’t have the functionality to support the model load/rollback/version control.

(To unblock you) For new version model update, can you directly trigger another deployment (new model) and shit traffic and remove the previous deployment (old model)?

---

<div class="post-metadata">

**Author:** ![shrekris](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@shrekris](https://discuss.ray.io/u/shrekris)\
**Post date:** [July 5, 2022, 6:48pm UTC](https://discuss.ray.io/t/does-ray-serve-support-local-model-hot-update-reload/6718/3 "2022-07-05T18:48:27Z")

</div>

Hi @Japson, as @Sihan_Wang mentioned, you can deploy your updated model to new Serve deployments on a fresh Ray cluster, and then you can shift traffic to it once the deployments are running.

Additionally, Serve [does a rolling update](https://docs.ray.io/en/releases-1.13.0/serve/core-apis.html#updating-a-deployment) when you update your live deployments on existing Ray cluster. In this case

1. Serve tears down some replicas from your old deployment.
2. Serve replaces them with replicas from your new deployment.
3. Serve repeats this with some more replicas from your old deployment until all replicas have updated.

During this time, your service is still live, but some requests may be handled by outdated replicas. This might also be a viable option.