Ray Serve does not currently provide a built-in API to trigger a lightweight config update (such as updating user_config and calling reconfigure) programmatically from within a running deployment or via Python code alone. Lightweight updates are only supported via updating the config file and running serve deploy, which is idempotent and only updates the changed config fields without restarting replicas, as described in the documentation. There is no watcher or observer pattern on the user_config dict; simply mutating it in Python will not trigger reconfigure, and calling serve.run imperatively will cause a full redeploy, not a lightweight update. This limitation is confirmed in both the documentation and community discussions, and there is no official API to trigger such updates from within a deployment as of Ray 2.48.0.
The only supported way to perform lightweight updates is to update the config file and redeploy using the CLI. Programmatic support for this feature has been requested by users, but as of now, it is not available, and there is no code example for in-place lightweight updates triggered from within a deployment or via a Python API. For more details, see the official docs and related discussions. Would you like a step-by-step breakdown of how the config-based update works, or more information on possible workarounds?
Sources:
- Ray Serve In-Place Updates Documentation
- Ray GitHub Issue: Programmatic Lightweight Update
- Ray Community Discussion: Local Model Hot Update/Reload
Hint: Mention @RunLLM in the post for followups.