It's very Inconvenient that ObjectRef must be cloudpickled for later outband remote task cancelation

I known that there exist a reason for cloudpickle to save ObjectRef for later outband task cancelation, but it’s just not a convenient way to cancel task, from the point of view of a ray user.

Could you share more details about why it’s not convenient?

If I want to kill a remote task out of band, I have to cloudpickle ObjectRef in the first place and save it somewhere for later use which I feel inconvenient.

Is it correct that “out of band task” means you’d like to cancel a task after you lose the reference?

I think the primary problem is the “out-of-band task cancellation” is not an officially supported path, and cloudpickle is a workaround to achieve this. It basically is against the reference counting model Ray currently has.

I feel like the right approach is to use a high level lib that supports out-of-band tasks. @yic isn’t workflow lib the good use case for this? I can imagine it should support it.

Good point! True, I need another layer of abstraction.