Yes, this is expected. Using tune.choice with num_samples=24 will randomly sample 24 parameter combinations, possibly with repeats, not exhaustively enumerate all unique combinations. For a true grid search (i.e., every possible combination exactly once), you must use tune.grid_search for each parameter instead of tune.choice—this will ensure all combinations are covered without repetition, regardless of num_samples [source], [source], [source].
Would you like a code example or more detail on how to set up grid search in Ray Tune?
Sources:
- ray.tune.grid_search documentation
- Tune terminates after running a single configuration
- How to execute each experiment once
Hint: Mention @RunLLM in the post for followups.