How to redeploy ray-serve app

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I am deploying my app like this:

# Start Ray
ray start --head --dashboard-host=0.0.0.0

# Deploy my app (Ray Serve)
serve deploy my_app.yaml

How do I re-deploy my app? I tried:

serve deploy my_app.yaml

But this doesn’t deploy the latest code…? (See image)

Is there a way to undeploy? Something like serve UNdeploy my_app.yaml?

foo
foo758×407 45.1 KB

Cross posted in slack: Slack

2

More info

When I run the:serve deploy my_app.yaml, I can see some state change in the dashboard but it looks like the new code is not loaded:

bar
bar1419×837 110 KB

3

What does your Serve config look like?

If the Serve config’s runtime_env or import_path doesn’t change between the serve deploy calls, then the app won’t be redeployed because Serve won’t detect a code change. One workaround is to add an env var to the runtime_env and change it between deploys to force the runtime_env to reinstall.

In general though, the runtime_env should change when there’s a code change. This guards against version inconsistencies across nodes. See the warning in this docs section to learn more.

4

shrekris

What does your Serve config look like?

Something like this:

proxy_location: EveryNode
http_options:
  host: 0.0.0.0
  port: 9631
applications:
  - name: my_app
    route_prefix: /my_app
    import_path: my_stuff.apps:app
    runtime_env: {}
    deployments:
      - name: ThingOne
      - name: ThingTwo

I am including my code & my dependencies using a custom docker image, like this:

FROM rayproject/ray:latest-py310-gpu
USER root

# Poetry
RUN curl -sSL https://install.python-poetry.org | POETRY_VERSION=1.5.1 python3 -
ENV PATH="${PATH}:${HOME}/.local/bin:/root/.local/bin"
RUN poetry config virtualenvs.create false

WORKDIR /app

# Copy my code & requirements
COPY . /app

# Because we want to install korok itself
RUN poetry install -n --with dev,tf

This means that generally my ray-config stays pretty static.

The workaround that I have been using is to redeploy using:

serve shutdown --yes
serve deploy my_app.yaml
5

Hi @Jacob_Summers, like Shreyas mentioned if runtime_env, import_path, or args doesn’t change, the code won’t be reloaded. This is because Serve supports lightweight config updates, which allow you to update configuration values in-place without having to tear down and restart your application. Also, in a production setting, we typically expect the runtime env to change if your code has changed.

One workaround for your use case is to set an environment variable in your runtime env, and modify it each time you want to reload your code, e.g:

proxy_location: EveryNode
http_options:
  host: 0.0.0.0
  port: 9631
applications:
  - name: my_app
    route_prefix: /my_app
    import_path: my_stuff.apps:app
    runtime_env:
      env_vars:
        CODE_VERSION: "1" # update when you update code
    deployments:
      - name: ThingOne
      - name: ThingTwo
6

Thanks for the reply @cindy_zhang ! This clarifies things immensely.

One question - Is using packaging your code and providing it in the runtime_env preferable to packaging in side a docker container?