Logging configuration with Ray Serve

Ray Libraries (Data, Train, Tune, Serve)
1

Hey,

I’ve been browsing through documentation and the python ray codebase, but can’t seem to find a solution.

I’m using ray serve (in the CLI, so ray serve my_script:my_model ) and I want all my logs to be fully in JSON format, unfortunately that seems a lot harder than expected. Whenever any of Ray’s functionality internally calls configure_component_logger, I don’t manage to undo some of the streamhandler’s prefixes.

Small toy example:

@serve.deployment
@serve.ingress(app)
class MyModel:
    def __init__(self):
         pass

    @app.get(
        "/health",
        summary="Health Check",
        description="Check whether the service is up and running.",
    )
    async def health(self):
        # Doing this in the init does not work to get a JSON formatted output, I have to do it in
        # the method...
        _logger = logging.getLogger("ray.serve")
        _logger.handlers.clear()
        handler = logging.StreamHandler()
        handler.setFormatter(get_formatter(log_format=LOG_FORMAT))
        _logger.addHandler(handler)

        _logger.info("I'm healthy!")

        return {"status": "healthy"}

my_model = MyModel.bind()

get_formatter is code I can’t share, but it creates a formatter (with structlog) that structures the logs into JSONs. Unfortunately two things break my approach here:

  • Updating the handler in the init does not work, between the init and the method (here health), something happens that undoes my handler configuration. I suspect in RayServeWrappedReplicate here.
  • Even when I put the logging configuration in the method itself, just before doing the logging, my stdout logs get an unwanted prefix:
(ServeReplica:default_MyModel pid=382) {"event": "I'm healthy!", "level": "info"}

How can I configure Ray Serve logging so that messages are fully JSON, without having to reconfigure the logger inside the non-init endpoints? I’m happy to have the information of the pid and model/actor in my log, but it should be attributes in the json.