Multi-processing within a grid search


I am running an HP grid search with Tune trying to optimize F(a, b, c).

The thing is that F is an aggregation of N runs of another function G(a, b, c, K) for K in some list L. So I need first to compute all the values of G.

I tried to run the for loop in parallel using p_tqdm but ran into multiprocessing and pool issues… So for now, I worked around by not doing parallelization.

Would you advise a particular solution for parallelization of “sub-functions” when using Tune?

Thank you ! Tune and Ray are really amazing!

Hi @Selim_Amrouni, generally you can use the core Ray abstractions to run parallel functions from Tune trainables. If you’re looking into running separate Tune processes (basically “Tune within Tune”) you might run into some problems there (we haven’t tested this and there are some knobs to turn to make it work).

Can you share more of your problem (e.g. pseudo/actual code) so we can make more sense of it?