[Tune, Serve] Passing a handle to grid search cause trials to get stuck in running and pending mode

Created an issue for this on GitHub (issue), but perhaps there is an alternative flow to achieve this.

In the below example we use tune.with_parameters() to pass a Ray Serve handle to a simulation function. We want to use Ray Tune to grid search simulation parameters and use the handle to call a ML model prediction function from within the simulation function.

The result of running the code below is explained further in the GitHub issue, but in essence a number trials are started and finished / TERMINATED, but the next trials seems to get stuck in RUNNING and PENDING.

The below code works in Ray 1.13.0 with tune.run() instead of Tuner and tuner.fit().

Wanted to lift all code to 2.0.0, but this seems to be a showstopper, if there is no other preferred workflow?

import ray
from ray import serve, tune
from ray.tune import Tuner

def simulation(config, handle=None):

class Model:
  def __init__(self):

  async def __call__(self, request):

def main():
  handle = serve.run(Model.bind())
  tuner = Tuner(tune.with_parameters(simulation, handle=handle), 
                param_space = {'seed': tune.grid_search(list(range(50)))})

if __name__ == "__main__":

Hey @jharaldson, thanks for posting this with a really clean repro! I’ve responded to the Github issue, we can continue the conversation there. :slight_smile: