Testing with Ray + Serve + FastApi + PyTest

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

I want to test a Serve program with the FastApi TestClient and PyTest.

a) There is no response from the test program (test_waiter.py) when run with the while-time-loop (in waiter.py).

b) If the while-time-loop is removed, pytest responds with a 404 status for different paths. See the output below.

c) There is a connection to Serve because entering http://localhost:8000/api/spider in the browser gives:

"Merde! Garcon, Garcon. There is a spider in my soup!"

What am I doing wrong? The code is:

# waiter.py
import time
import subprocess
import ray
from fastapi import FastAPI
from ray import serve

subprocess.run(["ray", "start", "--head", "--address", "127.0.0.1", "--port", "8787", "--disable-usage-stats"])

app = FastAPI()
ray.init(namespace="serve", ignore_reinit_error=True)
serve.start()

@serve.deployment(route_prefix="/api")
@serve.ingress(app)
class Garcon:
    @app.get("/soup")
    def root(self):
        return "Garcon, Soup of the day please."

    @app.get("/spider")
    def root(self):
        return "Merde! Garcon, Garcon. There is a spider in my soup!"

    @app.get("/fly")
    def root(self):
        return "Mademoiselle, Do not worry. The spider catches the flies. Bon appetit."

Garcon.deploy()

while True:
    time.sleep(2)
    pass

# test_waiter.py
from fastapi.testclient import TestClient
from waiter import app

client = TestClient(app)

def test_path_type():
    response = client.get("http://127.0.0.1:8000/api", headers={'content-type': 'application/json'})
    assert response.status_code == 200

if __name__ == "__main__":
    import sys
    import pytest

    sys.exit(pytest.main(["-v", "-s", __file__]))

_________________________ test_path_type _________________________

    def test_path_type():
        response = client.get("http://127.0.0.1:8000/api", headers={'content-type': 'application/json'})
>       assert response.status_code == 200
E       assert 404 == 200
E        +  where 404 = <Response [404]>.status_code

test_waiter.py:9: AssertionError

_________________________ test_path_type _________________________

    def test_path_type():
        response = client.get("http://127.0.0.1:8000/api/fly", headers={'content-type': 'application/json'})
>       assert response.status_code == 200
E       assert 404 == 200
E        +  where 404 = <Response [404]>.status_code

test_waiter.py:9: AssertionError

_________________________ test_path_type _________________________

    def test_path_type():
        response = client.get("http://127.0.0.1:8000/api/spider", headers={'content-type': 'application/json'})
>       assert response.status_code == 200
E       assert 404 == 200
E        +  where 404 = <Response [404]>.status_code

test_waiter.py:9: AssertionError
2

There’s no response from the test program when running with the while-time-loop because when test_waiter.py runs from waiter import app, it runs the waiter.py file. However, the waiter.py file never exits since it loops forever. This causes test_waiter.py to hang on the import statement.

When you remove the while-time-loop, waiter.py runs and deploys the deployments. Note that these deployments will still be live when the test_waiter.py file runs, so there’s no need to include the while-time-loop.

The test also needs to be refactored a bit. The URL "http://127.0.0.1:8000/api" doesn’t point to any one endpoint. It needs to be replaced with a URL pointing to a specific endpoint. Any of these should do:

  • "http://127.0.0.1:8000/api/soup"
  • "http://127.0.0.1:8000/api/spider"
  • "http://127.0.0.1:8000/api/fly"

I did, however, also get 404 errors when using the TestClient to connect to these endpoints. It may be an issue with the interaction between TestClient and Serve’s FastAPI integration. I was able to get a similar test working though:

# test_waiter.py
import requests
from waiter import app

def test_path_type():
    assert requests.get("http://127.0.0.1:8000/api/soup").json() == "Garcon, Soup of the day please."

if __name__ == "__main__":
    import sys
    import pytest

    sys.exit(pytest.main(["-v", "-s", __file__]))

Out of curiosity, what’s the use-case for using TestClient?

3

a) I should have been more clear but the pytest output displays results of 3 endpoints.

b) Using requests like this works:

def test_path_type():
    response = requests.get("http://127.0.0.1:8000/api/fly", headers={'content-type': 'application/json'})
    assert response.status_code == 200

c) Want to use the TestClient because it appears to be a cleaner way to perform tests for Serve + FastApi.

Does look like an integration problem between TestClient and Serve’s FastAPI integration :frowning:

4

Ah, I didn’t catch that the pytest output included all 3 endpoints. Thanks for clarifying!

I’m glad to hear that requests works. If possible, this might be a good workaround when writing tests for now.

As for the integration problem with TestClient, do you mind posting an issue, so we can track it and patch it?