I am not sure what is underlying issue here.
I have same code working when I am running with Local(same MBP) kubernetes deployment, but with kubernetes deployed on EKS I am seeing this pretty ambiguous error(at least I can’t figure this out with errors).
Here is the code -
from fastapi import FastAPI, HTTPException
from fastapi.responses import JSONResponse
from fastapi import status
from pydantic import BaseModel
import ray
from ray import serve
import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
app = FastAPI()
ray.init(address="auto", namespace="ray", log_to_driver=True, logging_level=logging.INFO)
serve.start(detached=True)
class InRequest(BaseModel):
text: str
@serve.deployment(route_prefix='/api', num_replicas=1)
@serve.ingress(app)
class InspectFlair:
@app.post('/entities')
def inspect(self, request: InRequest):
if len(request.text) == 0:
raise HTTPException(status_code=400, detail="'text' not found!")
try:
data = request.text
logger.error(f"ERR - Request: {request}, data: {data}")
flair_handle = serve.get_deployment("FlairTokenizer").get_handle()
future = flair_handle.remote(data)
response = ray.get(future)
logger.error(f"ERR - Response: {response}")
return JSONResponse(status_code=status.HTTP_200_OK, content=response)
except ValueError:
return {'error': ValueError}, 400
InspectFlair.deploy()
- Curl call response - Duplicate Content-Length is clearly visible but no errors.
❯ curl -ki -H "Content-Type:application/json" -d '{"text":"I live in california"}' http://127.0.0.1:8000/api/entities
HTTP/1.1 200 OK
date: Tue, 25 Jan 2022 23:01:06 GMT
server: uvicorn
content-length: 30
content-length: 30
content-type: application/json
{"predictions":["california"]}%
- POSTMAN response for same request - Received error.
- Stacktrace from head-node suggests that this is some invalid http method.
INFO: Started server process [422]
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ray/anaconda3/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 132, in data_received
self.parser.feed_data(data)
File "httptools/parser/parser.pyx", line 212, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserInvalidMethodError: Invalid method encountered
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ray/anaconda3/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 132, in data_received
self.parser.feed_data(data)
File "httptools/parser/parser.pyx", line 212, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserInvalidMethodError: Invalid method encountered
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ray/anaconda3/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 132, in data_received
self.parser.feed_data(data)
File "httptools/parser/parser.pyx", line 212, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserInvalidMethodError: Invalid method encountered
WARNING: Invalid HTTP request received.
Traceback (most recent call last):
File "/home/ray/anaconda3/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 132, in data_received
self.parser.feed_data(data)
File "httptools/parser/parser.pyx", line 212, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserInvalidMethodError: Invalid method encountered
I checked with uvicorn as well as FastAPI and this seems to be generating from uvicorn but not sure how to avoid this or is there any other issue.
I am following recommended way to integrate with FastAPI via native integration. Do we have any option to disable header originating from ray.serve? or is there any other solution to this?