Hello Team,
I have my rayserve application running inside a docker container. It’s a single node application with both the ray head and workers running inside the same container.
I have an entrypoint.sh file containing the commands to start the ray server and the deployments (shown below)
python -m rayserve.data_helpers.download_resources
ray start --head --node-ip-address=0.0.0.0 --include-dashboard=true --dashboard-host=0.0.0.0
serve build rayserve.deployer:deployment_graph -o config.yaml
serve deploy config.yaml
python -m rayserve.wait
Details of the commands:
download_resources → Downloads all the model files from S3 into a folder from which they are loaded into memory when the deployment objects are called
ray start --head … → Starts the ray server
serve build … → Creates a config file
serve deploy config.yaml → Starts the workers from the config file
Since serve deploy starts the deployments in the background , I invoke a wait.py file which basically does this
import time
if __name__=="__main__":
while True:
time.sleep(0.00001)
Since serve deploy runs everything in the background, I am not able to see any application logs in the console of my container (Screenshot given below)
How do you suggest I overcome this?
And is this the recommended way of deploying the application?
Please help