No Logging Options for Serve Deployment and/or Ray generated Pool Actors


I am looking for an option(s) for controlling the logging level for a serve deployment and pool actors (generated under the Ray Pool API) with Python and have not been able to do so, in either the Ray source code or existing documentation/issues. Specifying the logging level at initialization should pass the level to any actors using the encapsulated resources, though this doesn’t actually appear to be the case. A minimum example code for reproduction of the serve deployment is included below (though similar behavior occurs with pool actors as described/exhibited). Any insights would be appreciated; thank you in advanced.

Versions / Dependencies

Windows 10
Ubuntu 18.04
Python 3.8
Ray 1.10.0

Reproduction script

import logging
import ray
from ray import serve
import time
ray.init(configure_logging=True, logging_level=logging.ERROR)
class ModelServer:
    def __init__(self): print('Initializing')
    def __call__(self): print('Called')

model = ModelServer.get_handle()

What happened

 pid=47948) 2022-02-22 15:30:47,910     INFO -- Using RayInternalKVStore for controller checkpoint and recovery.
 pid=47948) 2022-02-22 15:30:48,017     INFO -- Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:mQCdAu:SERVE_PROXY_ACTOR-node:' on node 'node:' listening on ''
2022-02-22 15:30:48,655 INFO -- Started Serve instance in namespace '0d3370fc-79eb-44c2-9487-469ea0e95e50'.
2022-02-22 15:30:48,670 INFO -- Updating deployment 'ModelServer'. component=serve deployment=ModelServer
 pid=2228) INFO:     Started server process [2228]
 pid=47948) 2022-02-22 15:30:48,769     INFO -- Adding 1 replicas to deployment 'ModelServer'. component=serve deployment=ModelServer
2022-02-22 15:30:49,437 INFO -- Deployment 'ModelServer' is ready at ``. component=serve deployment=ModelServer
 pid=40024) Initializing
 pid=40024) Called
 pid=47948) 2022-02-22 15:30:49,542     INFO -- Removing 1 replicas from deployment 'ModelServer'. component=serve deployment=ModelServer

What you expected to happen

 pid=40024) Initializing
 pid=40024) Called

Hi @Yatagarasu, thanks for reporting this! I agree, I would expect that setting the logging level as you did would hide the INFO logs. @sangcho can I confirm with you that this is how the configure_logging and logging_level APIs are supposed to work?

If so, this looks like a bug in Ray Serve (and possibly also in Ray Pool), and we can make a Github issue for this to track it.

Thank you for your reply; now opened as an issue:

1 Like