I am looking for an option(s) for controlling the logging level for a serve deployment and pool actors (generated under the Ray Pool API) with Python and have not been able to do so, in either the Ray source code or existing documentation/issues. Specifying the logging level at initialization should pass the level to any actors using the encapsulated resources, though this doesn’t actually appear to be the case. A minimum example code for reproduction of the serve deployment is included below (though similar behavior occurs with pool actors as described/exhibited). Any insights would be appreciated; thank you in advanced.
Versions / Dependencies
Windows 10
Ubuntu 18.04
Python 3.8
Ray 1.10.0
Reproduction script
import logging
import ray
from ray import serve
import time
ray.init(configure_logging=True, logging_level=logging.ERROR)
@serve.deployment(route_prefix="/ModelServer")
class ModelServer:
def __init__(self): print('Initializing')
def __call__(self): print('Called')
serve.start()
ModelServer.deploy()
model = ModelServer.get_handle()
ray.get(model.remote())
serve.shutdown()
What happened
pid=47948) 2022-02-22 15:30:47,910 INFO checkpoint_path.py:16 -- Using RayInternalKVStore for controller checkpoint and recovery.
pid=47948) 2022-02-22 15:30:48,017 INFO http_state.py:98 -- Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:mQCdAu:SERVE_PROXY_ACTOR-node:127.0.0.1-0' on node 'node:127.0.0.1-0' listening on '127.0.0.1:8000'
2022-02-22 15:30:48,655 INFO api.py:475 -- Started Serve instance in namespace '0d3370fc-79eb-44c2-9487-469ea0e95e50'.
2022-02-22 15:30:48,670 INFO api.py:249 -- Updating deployment 'ModelServer'. component=serve deployment=ModelServer
pid=2228) INFO: Started server process [2228]
pid=47948) 2022-02-22 15:30:48,769 INFO deployment_state.py:920 -- Adding 1 replicas to deployment 'ModelServer'. component=serve deployment=ModelServer
2022-02-22 15:30:49,437 INFO api.py:261 -- Deployment 'ModelServer' is ready at `http://127.0.0.1:8000/ModelServer`. component=serve deployment=ModelServer
pid=40024) Initializing
pid=40024) Called
pid=47948) 2022-02-22 15:30:49,542 INFO deployment_state.py:940 -- Removing 1 replicas from deployment 'ModelServer'. component=serve deployment=ModelServer
Hi @Yatagarasu, thanks for reporting this! I agree, I would expect that setting the logging level as you did would hide the INFO logs. @sangcho can I confirm with you that this is how the configure_logging and logging_level APIs are supposed to work?
If so, this looks like a bug in Ray Serve (and possibly also in Ray Pool), and we can make a Github issue for this to track it.