Did a default smoothing behavior recently get added?

RLlib
1

1. Severity of the issue: (select one)
Low: Annoying but doesn’t hinder my work.

2. Environment:

  • Ray version: 3.0.0
  • Python version: 3.12
  • OS: Ubuntu

3. What happened vs. what you expected:
When releasing a custom metric through the following code…

bt_dict = get_BT_skill(self.league, win_ar)
        for k, v in bt_dict.items():
            algorithm.metrics.log_value(("BradleyTerry", k), v)

(Full code here.)

…my metric outputs are a lot smoother than I’d expect over multiple iterations. I ran this code three weeks prior under slightly different environment settings, and noticed a much spikier set of Bradley-Terry scores.

I checked the official repo’s MetricsLogger, but it doesn’t seem to have suddenly added a default EMA value, or anything else I’d expect to see here. Am I looking in the wrong place?

2

The smoothing of your custom metric is likely due to the default behavior of Ray’s MetricsLogger, which applies an Exponential Moving Average (EMA) with a default coefficient of 0.01 when no window or reduction method is specified. This can make metrics appear smoother and less spiky across iterations, especially if you log values without explicitly setting the reduce or window parameters. This behavior is documented in the MetricsLogger API and confirmed in the test_metrics_logger.py, where the default EMA coefficient is shown to be 0.01.

If you want less smoothing (i.e., spikier metrics), you should explicitly set the reduce parameter to "mean" and specify a small window (e.g., window=1) when calling log_value. This will override the default EMA and give you the raw or windowed mean values, which are more sensitive to recent changes and will appear spikier in your plots. Would you like a code example for this?

Sources:

Hint: Mention @RunLLM in the post for followups.

3

Ah, I found this commit. There was supposed to be a default EMA value before two weeks ago, but, in practice, there wasn’t one.