How severe does this issue affect your experience of using Ray?
- High: It blocks me to complete my task.
Hello,
I’m encountering an issue with deploying a Ray Serve application inside a Docker container preventing access to my service from outside the Docker container.
Environment:
- Ray version: 2.10
Issue Description:
Here is my docker image:
FROM python:3.10.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["sh", "-c", "ray start --head --dashboard-host 0.0.0.0 && python3 testing.py"]
Here is my code:
import cv2
import numpy as np
import base64
from fastapi import FastAPI
from pydantic import BaseModel
from ray import serve
import ray
from globals import AppConfig
from Utils.utils import OpenVINOModel, output_dict
ray.init(address="auto", namespace="example")
serve.start(detached=True, http_options={"host": "0.0.0.0"})
app = FastAPI()
class InputImg(BaseModel):
nid_img: str
@serve.deployment()
@serve.ingress(app)
class OrganicOrienter:
def __init__(self, config):
self.vino_client = OpenVINOModel(config)
self.img_size = config["input_image_size"]
self.confidence_threshold = config["Confidence_Threshold"]
def decode_img(self, img):
img = base64.b64decode(img)
img = np.frombuffer(img, dtype=np.uint8)
img = cv2.imdecode(img, cv2.IMREAD_COLOR)
return img
@app.post("/verify")
async def orient_img(self, data: InputImg):
img = self.decode_img(data.nid_img)
img = cv2.resize(img, (self.img_size, self.img_size))
img = img.transpose(2, 0, 1)
img = img.reshape(1, 3, self.img_size, self.img_size) / 255.0
outputs = self.vino_client.run(img)
scores = next(iter(outputs.values()))[0]
orientation = np.argmax(scores)
orientation = {"orientation": float(orientation)}
return orientation
config = AppConfig("Config/config.json")
orienter_app = OrganicOrienter.bind(config.config["Orienter"])
serve.run(orienter_app)
Here is the relevant output upon running my image:
(facematch) yousefvalify@Userk:/mnt/c/Users/User/Desktop/FaceVerification$ docker run -it --rm -p 8000:8000 -p 8265:8265 --shm-size=2gb face
Enable usage stats collection? This prompt will auto-proceed in 10 seconds to avoid blocking cluster startup. Confirm [Y/n]: y
Usage stats collection is enabled. To disable this, add `--disable-usage-stats` to the command that starts the cluster, or run the following command: `ray disable-usage-stats` before starting the cluster. See https://docs.ray.io/en/master/cluster/usage-stats.html for
more details.
Local node IP: 172.17.0.2
--------------------
Ray runtime started.
--------------------
Next steps
To add another node to this Ray cluster, run
ray start --address='172.17.0.2:6379'
To connect to this Ray cluster:
import ray
ray.init()
To submit a Ray job using the Ray Jobs CLI:
RAY_ADDRESS='http://172.17.0.2:8265' ray job submit --working-dir . -- python my_script.py
See https://docs.ray.io/en/latest/cluster/running-applications/job-submission/index.html
for more information on submitting Ray jobs to the Ray cluster.
To terminate the Ray runtime, run
ray stop
To view the status of the cluster, use
ray status
To monitor and debug Ray, view the dashboard at
172.17.0.2:8265
If connection to the dashboard fails, check your firewall settings and network configuration.
2024-04-02 13:22:37,377 INFO worker.py:1567 -- Connecting to existing Ray cluster at address: 172.17.0.2:6379...
2024-04-02 13:22:37,389 INFO worker.py:1743 -- Connected to Ray cluster. View the dashboard at 172.17.0.2:8265
(ProxyActor pid=376) INFO 2024-04-02 13:22:45,888 proxy 172.17.0.2 proxy.py:1160 - Proxy starting on node 024b1ebaa45b456a87671a03285107332046399ff125e54ba4a0129a (HTTP port: 8000).
The new client HTTP config differs from the existing one in the following fields: ['host', 'location']. The new HTTP config is ignored.
(ServeController pid=332) INFO 2024-04-02 13:22:46,138 controller 332 deployment_state.py:1582 - Deploying new version of Deployment(name='OrganicOrienter', app='default') (initial target replicas: 1).
(ServeController pid=332) INFO 2024-04-02 13:22:46,247 controller 332 deployment_state.py:1910 - Adding 1 replica to Deployment(name='OrganicOrienter', app='default').
2024-04-02 13:22:55,195 INFO api.py:575 -- Deployed app 'default' successfully.
How i make request:
import requests
import base64
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
class Base64Encoder:
@staticmethod
def run(file_name):
with open(file_name, "rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
return encoded_string.decode("utf-8")
def send_request(encoded_image1):
payload = {"nid_img": encoded_image1}
response = requests.post("http://localhost:8000/verify/", json=payload)
return response.json()
if __name__ == "__main__":
image_path1 = "img1.jpeg"
encoded_image1 = Base64Encoder.run(image_path1)
print(send_request(encoded_image1))