Hello,
I have one application with single model defined in a python file let’s say inference:app where app I defined as a variable Model.bind()
, everytime I try to deploy a python file using serve deploy config.yaml
and when I check status it always throwing module not found error the model I want to deploy. My question is how to deploy using config.yaml a local python file which contains the model deployment code ? I am sharing my yaml file little bit later.
I tried following this documentation but it doesn’t worked.
https://docs.ray.io/en/latest/serve/api/doc/ray.serve.Application.html
My yaml looks like this :
proxy_location: ...
http_options:
host: ...
port: ...
request_timeout_s: ...
keep_alive_timeout_s: ...
grpc_options:
port: ...
grpc_servicer_functions: ...
logging_config:
log_level: ...
logs_dir: ...
encoding: ...
enable_access_log: ...
applications:
- name: ...
route_prefix: ...
import_path: inference:app
runtime_env: ...
deployments:
- name: ...
num_replicas: ...
...
My another question is this why it always replace the default application deployed to Ray serve , my ray version is : 2.9.3