How to log to stdout from Ray Serve

Shravani · June 20, 2025, 12:58pm

We are using KubeRay in production as our inferencing solution. The logs are available for the deployment inside the metrics server of the head pod and ofcourse inside the filesystem of the pod at /tmp/ray/session_*/logs.

However, we have a different team transporting our logs and we want to print the logs for the server to the pod’s stdout so that it can be exported. Is there any way to do that?

This is the python script we are using for the serve application,

github.com/ray-project/kuberay

ray-operator/config/samples/vllm/serve.py

release-1.3

import os

from typing import Dict, Optional, List
import logging

from fastapi import FastAPI
from starlette.requests import Request
from starlette.responses import StreamingResponse, JSONResponse

from ray import serve

from vllm.engine.arg_utils import AsyncEngineArgs
from vllm.engine.async_llm_engine import AsyncLLMEngine
from vllm.entrypoints.openai.cli_args import make_arg_parser
from vllm.entrypoints.openai.protocol import (
    ChatCompletionRequest,
    ChatCompletionResponse,
    ErrorResponse,
)
from vllm.entrypoints.openai.serving_chat import OpenAIServingChat

This file has been truncated. show original

Shravani · June 23, 2025, 10:52am

FYI, I tried,
RAY_LOG_TO_STDERR
It was able to log to stdout, but the json format with this is not working

Topic		Replies	Views
Log inside function in class decorated by deployment does not appear in console Ray Serve	2	15	March 12, 2025
Logging to stdout for ray serve Ray Serve	1	892	November 29, 2022
Define Custom Logger in Ray server application with multiple apps Ray Serve	3	367	November 9, 2023
Configuring Ray Serve Logging: JSON Formatting, Stream Handling, and CLI Output Ray Serve	1	66	November 6, 2024
Ray serve logs in k8s pod Ray Serve	0	84	May 23, 2024

How to log to stdout from Ray Serve

Related topics