Cannot access ray serve app outside Docker

1

Hi guys! I’ve packaged up my ray application into a docker image and serve my application locally with serve run module:entrypoint. Within the docker, I can request the endpoints and get back my outputs but outside the docker and still on the same machine, I’m unable to ping the endpoint.

My dockerfile looks like:

FROM python:3.8

ARG GITHUB_TOKEN

WORKDIR /src

COPY requirements.txt /src/requirements.txt

RUN pip install --no-cache-dir --upgrade -r /src/requirements.txt

COPY . /src

EXPOSE 8000
EXPOSE 8265
CMD ["serve", "run", "module:entrypoint", "--host", "0.0.0.0", "--port", "8000"]

Does anyone know how to solve this? If I just remote into the container with docker run myimage -p 8000:8000 -p 8265:8265 and then run the app, I’m still unable to reach it outside the docker. This is the error I get when trying to ping it:

 File "/home/ubuntu/nlp-executor/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "python3.8/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "python3.8/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 277, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer
2

Hi @nabs, welcome to the Ray Discuss forums! In the future, I recommend posting Ray Serve questions to the Ray Serve Discuss channel or the Ray Slack instead of leaving it uncategorized. That way, it’ll get more attention and responses from Serve users.

The serve run issue is a known bug. In Ray 2.0.0, serve run ignores the host and port arguments, so the host isn’t actually being set to 0.0.0.0. This should be resolved in the next release.

For now, serve deploy should let you test Serve in Docker. This comment offers some other possible mitigations.