Ray serve : Request Object has no attribute data

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hello !

I am trying to implement a serving for a NLP model I trained, i’m inspiring myself of this for the serving part : Code Example

I have made the following code :

from transformers import CamembertForSequenceClassification,CamembertTokenizer
import ray
from ray import serve
import requests
import argparse
import torch

args=argparse.Namespace()
use_gpu = torch.cuda.is_available()
args.device = torch.device("cuda" if use_gpu else "cpu")

client=serve.start()

class predict_class:
    def __init__(self,args):
        self.args=args
        self.model=CamembertForSequenceClassification.from_pretrained("/home/amaury/Documents/project_a1/camembert-v1")
        self.tokenizer=CamembertTokenizer.from_pretrained("camembert-base")
        self.model.to(args.device)

    def __call__(self,flask_request):
        input=flask_request.data.decode("utf-8")
        
        tokenized=self.tokenizer(input)
        text=torch.tensor([tokenized]).to(args.device)

        result=self.model(text)
        return(result)

client.create_backend("classpredict", predict_class, args, ray_actor_options={"num_gpus": 1})
client.create_endpoint("classpredict",backend="classpredict", route="/classpredict",methods=["GET","POST"])

r=requests.post("http://127.0.0.1:8000/classpredict",data="hello")
r.content

I’m getting the following error : AttributeError: ‘Request’ object has no attribute ‘data’

I do not get why or what it is that i’m doing differently that might be linked to this.
Thanks in advance for your help ! sorry if this was already brought up, i did not find anything regarding this.

2

Hello!

The Ray Serve API changed a little bit since that example code was posted. As of Ray version 1.2.0, Ray Serve now uses a Starlette request object instead of a Flask request object. Here’s the migration guide: Ray Serve Request Migration Guide - Google Docs

For your specific error, you’ll want to use await request.body() instead of request.data.

1 Like
3

Ah, Many thanks I did not know that,

Indeed the error went away !

Thanks for the help :slight_smile: