Hey,
I’ve been browsing through documentation and the python ray codebase, but can’t seem to find a solution.
I’m using ray serve (in the CLI, so ray serve my_script:my_model ) and I want all my logs to be fully in JSON format, unfortunately that seems a lot harder than expected. Whenever any of Ray’s functionality internally calls configure_component_logger, I don’t manage to undo some of the streamhandler’s prefixes.
Small toy example:
@serve.deployment
@serve.ingress(app)
class MyModel:
def __init__(self):
pass
@app.get(
"/health",
summary="Health Check",
description="Check whether the service is up and running.",
)
async def health(self):
# Doing this in the init does not work to get a JSON formatted output, I have to do it in
# the method...
_logger = logging.getLogger("ray.serve")
_logger.handlers.clear()
handler = logging.StreamHandler()
handler.setFormatter(get_formatter(log_format=LOG_FORMAT))
_logger.addHandler(handler)
_logger.info("I'm healthy!")
return {"status": "healthy"}
my_model = MyModel.bind()
get_formatter is code I can’t share, but it creates a formatter (with structlog) that structures the logs into JSONs. Unfortunately two things break my approach here:
- Updating the handler in the init does not work, between the init and the method (here
health), something happens that undoes my handler configuration. I suspect in RayServeWrappedReplicate here. - Even when I put the logging configuration in the method itself, just before doing the logging, my stdout logs get an unwanted prefix:
(ServeReplica:default_MyModel pid=382) {"event": "I'm healthy!", "level": "info"}
How can I configure Ray Serve logging so that messages are fully JSON, without having to reconfigure the logger inside the non-init endpoints? I’m happy to have the information of the pid and model/actor in my log, but it should be attributes in the json.