Is there an elegant way to stack DefaultCallbacks objects? I have one that mutates sample batch in on_postprocess_trajectory() and another that logs custom metrics in on_episode_end(). Right now I merge the code into one class, which is not satisfying (very not OOP). If I pass them in as a list, I…

Inheritance would work perfectly. However, I was looking for a solution via configuration. Main reason is that if a new callbacks class is written, new inheritance code (also for combinations of different callbacks!) also need to be written. Here is a merge function that can take any number of callb…

Personally I do not see an issue with merging them together but another alternative would be to have one inherit from the other. class MyCallbacks(DefaultCallbacks): def on_postprocess_trajectory(self, # etc class AnotherCallbacks(MyCallbacks): def on_episode_end(self, #if needed #super…

Hey @mannyv and @RickLan , I’m currently also thinking about making the callbacks API better by replacing it with an events-based system, where anyone (user, algos, exploration components, models, etc…) can subscribe to built-in events (such as “episode_step”, “learn_on_batch”, etc…), but also defin…

Hi @RickLan , Yes there are multiple ways to address this. I like the forced inheritance method because it encourages the developer to reason about the chain of callbacks and conflicts that might exist between them. Imagine if two callbacks are mutating state in incompatible ways. I think adding cla…

Hey @mannyv , totally agree. The (hidden) order would be the biggest weakness of such an even based system. E.g. for reward-tampering callbacks (e.g. curiosity adding an internal reward, then the user doesn’t know, whether this already happened or not). However, the ordering conflicts that you descr…

[image] mannyv: This said, I think both are perfectly workable solutions and like many things comes down to preference. A third way: both also works together :slight_smile:

[image] sven1977: making the callbacks API better by replacing it with an events-based system I’m not a software design guru, but I always like better tools :slight_smile: Edit: I probably don’t know enough about RLlib. From my point of view, as an user/researcher, if the documentation expl…

[image] sven1977: (I think this is limited right now to scalars and video types ??) Yes! Image support would be nice. I hacked it here: [RLlib] Writing to tensorboard during custom evaluation - #3 by RickLan

I would love some better tensorboard logging. E.g. array of images would be nice (instead of being converted to video) for visualising autoencoder reconstruction, etc. For the events, maybe it makes sense to give them very descriptive name, trace once at startup (similar to trajectory_view ), and p…

Yeah, this is a great idea! We should do the same for the trajectory view API: Actually print out some results from the pre-passes. At least for logging level INFO.

@smorad , This is a great idea. Of course you know someone, probably one of us actually, will come along and try to dynamically add and remove event listeners.

Stacking callback objects [Solved. Code included.]

RLlib

RickLan April 29, 2021, 11:19am 9

Yes! Image support would be nice. I hacked it here: [RLlib] Writing to tensorboard during custom evaluation - #3 by RickLan

Topic		Replies	Views
Persisting values across callbacks RLlib	2	655	May 8, 2023
Logging from DefaultCallbacks to LoggerCallback gives weird behavior RLlib	1	387	June 9, 2022
Example code uses RLlib's DefaultCallbacks, but tune.run expects tune.Callback Ray Tune	1	1430	July 14, 2022
Population-Based Training Replays Do Not Execute Callbacks Ray Tune	0	296	July 20, 2021
Multiple callbacks in Tune Ray Tune	1	65	November 8, 2024

Stacking callback objects [Solved. Code included.]

Related topics