Unable to make Request from outside Docker Container

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.


I’m encountering an issue with deploying a Ray Serve application inside a Docker container preventing access to my service from outside the Docker container.


  • Ray version: 2.10

Issue Description:

Here is my docker image:

FROM python:3.10.12-slim


COPY requirements.txt ./

RUN pip install --no-cache-dir -r requirements.txt

COPY . .


CMD ["sh", "-c", "ray start --head --dashboard-host && python3 testing.py"]

Here is my code:

import cv2
import numpy as np
import base64
from fastapi import FastAPI
from pydantic import BaseModel
from ray import serve
import ray

from globals import AppConfig
from Utils.utils import OpenVINOModel, output_dict

ray.init(address="auto", namespace="example")
serve.start(detached=True, http_options={"host": ""})

app = FastAPI()

class InputImg(BaseModel):
    nid_img: str

class OrganicOrienter:
    def __init__(self, config):
        self.vino_client = OpenVINOModel(config)
        self.img_size = config["input_image_size"]
        self.confidence_threshold = config["Confidence_Threshold"]

    def decode_img(self, img):
        img = base64.b64decode(img)
        img = np.frombuffer(img, dtype=np.uint8)
        img = cv2.imdecode(img, cv2.IMREAD_COLOR)
        return img
    async def orient_img(self, data: InputImg):
        img = self.decode_img(data.nid_img)
        img = cv2.resize(img, (self.img_size, self.img_size))
        img = img.transpose(2, 0, 1)
        img = img.reshape(1, 3, self.img_size, self.img_size) / 255.0
        outputs = self.vino_client.run(img)
        scores = next(iter(outputs.values()))[0]

        orientation = np.argmax(scores)

        orientation = {"orientation": float(orientation)}

        return orientation

config = AppConfig("Config/config.json")

orienter_app = OrganicOrienter.bind(config.config["Orienter"])


Here is the relevant output upon running my image:

(facematch) yousefvalify@Userk:/mnt/c/Users/User/Desktop/FaceVerification$ docker run -it --rm -p 8000:8000 -p 8265:8265 --shm-size=2gb face
Enable usage stats collection? This prompt will auto-proceed in 10 seconds to avoid blocking cluster startup. Confirm [Y/n]: y
Usage stats collection is enabled. To disable this, add `--disable-usage-stats` to the command that starts the cluster, or run the following command: `ray disable-usage-stats` before starting the cluster. See https://docs.ray.io/en/master/cluster/usage-stats.html for 
more details.

Local node IP:

Ray runtime started.

Next steps
  To add another node to this Ray cluster, run
    ray start --address=''

  To connect to this Ray cluster:
    import ray

  To submit a Ray job using the Ray Jobs CLI:
    RAY_ADDRESS='' ray job submit --working-dir . -- python my_script.py

  See https://docs.ray.io/en/latest/cluster/running-applications/job-submission/index.html
  for more information on submitting Ray jobs to the Ray cluster.

  To terminate the Ray runtime, run
    ray stop

  To view the status of the cluster, use
    ray status

  To monitor and debug Ray, view the dashboard at

  If connection to the dashboard fails, check your firewall settings and network configuration.
2024-04-02 13:22:37,377 INFO worker.py:1567 -- Connecting to existing Ray cluster at address:
2024-04-02 13:22:37,389 INFO worker.py:1743 -- Connected to Ray cluster. View the dashboard at 
(ProxyActor pid=376) INFO 2024-04-02 13:22:45,888 proxy proxy.py:1160 - Proxy starting on node 024b1ebaa45b456a87671a03285107332046399ff125e54ba4a0129a (HTTP port: 8000).
The new client HTTP config differs from the existing one in the following fields: ['host', 'location']. The new HTTP config is ignored.
(ServeController pid=332) INFO 2024-04-02 13:22:46,138 controller 332 deployment_state.py:1582 - Deploying new version of Deployment(name='OrganicOrienter', app='default') (initial target replicas: 1).
(ServeController pid=332) INFO 2024-04-02 13:22:46,247 controller 332 deployment_state.py:1910 - Adding 1 replica to Deployment(name='OrganicOrienter', app='default').
2024-04-02 13:22:55,195 INFO api.py:575 -- Deployed app 'default' successfully.

How i make request:

import requests
import base64
import time
from concurrent.futures import ThreadPoolExecutor, as_completed

class Base64Encoder:
    def run(file_name):
        with open(file_name, "rb") as image_file:
            encoded_string = base64.b64encode(image_file.read())
            return encoded_string.decode("utf-8")

def send_request(encoded_image1):
    payload = {"nid_img": encoded_image1}
    response = requests.post("http://localhost:8000/verify/", json=payload)
    return response.json()

if __name__ == "__main__":
    image_path1 = "img1.jpeg"
    encoded_image1 = Base64Encoder.run(image_path1)


Hi @Yousef_Ibrahim, I don’t really see an issue with your setup. When you see that “Deployed app ‘default’ successfully.” message, your application should be up and able to accept requests. Is your container still running at the time when you submit the request? Did you open port 8000 on your container?

I tried when a simplified version

# fake.py
import ray
from faker import Faker
from ray import serve

ray.init(address="auto", namespace="example")
serve.start(detached=True, http_options={"host": ""})

def create_fake_email():
    return Faker().email()

app = create_fake_email.bind()

# Dockerfile
FROM rayproject/ray:2.10.0-py310

RUN pip install Faker==18.13.0

COPY fake.py /home/ray/fake.py


CMD ["sh", "-c", "ray start --head --dashboard-host && serve run fake:app"]

Running this to build the image

docker build . -t test_request_from_outside_container:latest

Running this to start the container

docker run -it --rm -p 8000:8000 test_request_from_outside_container:latest

You should be able to submit request in the other terminal window

(ray) ubuntu@devbox:~$ curl localhost:8000