Define Custom Logger in Ray server application with multiple apps

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

It seems like Ray Serve is not able to log the details in the custom log file I am defining, can anyone help

Here is my logger initialization

from ray import serve
import logging

logger = logging.getLogger("ray.serve")
logger.setLevel(logging.WARNING)
logger.addHandler(logging.FileHandler("/app/logs/app.log"))

And here is my main inference function

    async def __call__(self, request):
        image_path = await request.body()
        im = Image.open(image_path).convert("RGB")
        # prepare image for the model
        encoded_inputs = self.processor(
            im, return_tensors="pt", padding="max_length", truncation=True
        )

        # make sure all keys of encoded_inputs are on the same device as the model
        for k, v in encoded_inputs.items():
            encoded_inputs[k] = v.to(self.model.device)

        # forward pass
        outputs = self.model(**encoded_inputs)
        logits = outputs.logits
        predicted_class_idx = logits.argmax(-1).item()
        del logits
        del outputs
        del im 
        torch.cuda.empty_cache()
        gc.collect()
        # Finally class index to actual class
        logger.info("Request completed")
        return self.idx2label[predicted_class_idx]

The target is to log the custom information during inference in a separate logging file, not in the default location of ray serve logs /tmp/ray/session.......

2

@shrekris any help on this topic, would love to hear your response

3

So, Now I am able to get the logs from tmp dir to the custom dir I have in system with mapping volumes in docker compose

    volumes:
      - /mnt/efs/fs1/logs/:/tmp/ray/

But It contains lot of information what I want to exclude and also , Few things I want to log custom in the inference service, Is there any way to change this configuration

4

Hi wgetdd.deb, do you mind sharing what you change to have the correct log path? So that i can help to check why you have unnecessary information.