Hello,
Please help me to get logging levels to work after importing ray.
I need to see INFO and sometimes DEBUG. I set logging level in main, but it does not work after importing ray.
It is about main module, not tune tasks (childs)
Example:
logging.debug(“debug log message”)
logging.info(“info log message”)
logging.warning(“warning log message”)
logging.error(“error log message”)
logging.critical(“critical log message”)
Output:
WARNING:root:warning log message
ERROR:root:error log message
CRITICAL:root:critical log message
1 Like
cc @amogkam Can you address his question?
@viotemp1 I think you just need to set the logging_level arg in your ray.init to the appropriate value you want.
Hello,
I always do that.
ray_log_level = logging.INFO
if not ray.is_initialized():
print(“ray init”)
ray.init(
log_to_driver=True, # False to print TF messages # True False
configure_logging=True,
logging_level=ray_log_level,
include_dashboard=False,
num_cpus=multiprocessing.cpu_count(),
num_gpus=1,
# resources={“my_resource”: 1},
)
What I wanted to point out is that before running ray.init my info messages are not shown.
logging.debug(“debug log message”)
logging.info(“info log message”)
logging.warning(“warning log message”)
logging.error(“error log message”)
logging.critical(“critical log message”)
I’ll prepare a small notebook these days to be more clear.
Hello,
Checked again today removing one by one the imports and I found out that the issue is not directly from ray, but from hebo.
Sample notebook: test_logging_levels
I’ll check a bit on hebo to see what’s wrong.
Regards,
Hello, the issue is indeed from HEBO (gpy_wgp.py).
I commented the following lines there and everything is fine now.
#import logging
#logging.disable(logging.WARNING)
#import warnings
#warnings.filterwarnings(‘ignore’, category = RuntimeWarning)
Regards,
@amogkam I seem to have a similar issue. If I add a logger the same way how it’s done in the Ray repo, then setting the level doesn’t seem to work:
# main script
import logging
import ray
from ray import tune
from test_env import SimpleCorridor
if __name__ == "__main__":
# ray.init(logging_level=logging.DEBUG)
ray.init(logging_level=logging.INFO)
config = {
"env": SimpleCorridor,
"evaluation_interval": 1,
"env_config": {
"corridor_length": 5,
},
"evaluation_num_episodes": 10,
"num_gpus": 0.5,
"lr": tune.grid_search([1e-4, 1e-4]),
"num_workers": 2,
"num_envs_per_worker": 1,
"framework": "torch",
}
stop = {"training_iteration": 10}
results = tune.run('APPO', config=config, stop=stop, verbose=1)
# test_env.py
import logging
import gym
import numpy as np
from gym.spaces import Box, Discrete
logger = logging.getLogger(__name__)
class SimpleCorridor(gym.Env):
def __init__(self, config):
logger.debug("debug log message")
logger.info("info log message")
logger.warning("warning log message")
logger.error("error log message")
logger.critical("critical log message")
self.end_pos = config["corridor_length"]
self.cur_pos = 0
self.action_space = Discrete(2)
self.observation_space = Box(
0.0, self.end_pos, shape=(1, ), dtype=np.float32)
def reset(self):
self.cur_pos = 0
return [self.cur_pos]
def step(self, action):
assert action in [0, 1], action
if action == 0 and self.cur_pos > 0:
self.cur_pos -= 1
elif action == 1:
self.cur_pos += 1
done = self.cur_pos >= self.end_pos
return [self.cur_pos], 1.0 if done else -0.1, done, {}
Output:
....
(pid=510610) warning log message
(pid=510610) error log message
(pid=510610) critical log message
(pid=510606) warning log message
(pid=510606) error log message
(pid=510606) critical log message
(pid=510603) warning log message
(pid=510603) error log message
(pid=510603) critical log message
(pid=510612) warning log message
(pid=510612) error log message
(pid=510612) critical log message
....