Ray tune capturing children tasks

hi all,

I’m trying to launch remote tasks from within ray.tune, in the step method of Ray.Trainable. The children are not captured by the parent, and hence require separate resources.

My use case is using a library to do a heavy computation, using the cores allocated to the ray tune task, and when that finishes launching several smaller tasks in parallel, also on the same cores since they are idle. The reason I need to run them in the step function is to be able to return metrics collected using this later parallel execution from the step function, which can be used for guiding hyperparam search.

I have 1.4.0, and I’ve come accross this: `placement_group=None` is not working when trying not to use the same placement group from children actors. · Issue #15944 · ray-project/ray · GitHub, where the comment says you can pass False to the placement_group of the child remote tasks, which I do, but still doesn’t work as expected.

Is there a workaround to this for the latest ray release? (1.4.0)

Hi Taylan,

I think the issue is outdated. Here we updated the semantics: [Placement group] Fix capturing child tasks doesn't work. by rkooo567 · Pull Request #16049 · ray-project/ray · GitHub

TLDR: Can you try passing placement_group="default" instead?

This fix is merged 10 days ago, 1.4.0 released 18 days ago - I guess this change will make it to 1.4.1

I’ll try your suggestion with the nightly builds.