# ExperimentAnalysis - Did you pass the correct \`metric\` parameter? **URL:** \ **Category:** Ray Tune\ **Created:** [March 6, 2023, 1:57pm UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643 "2023-03-06T13:57:06Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mblum](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mblum/32/4019_2.png) [@mblum](https://discuss.ray.io/u/mblum)\ **Post date:** [March 6, 2023, 1:57pm UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643/1 "2023-03-06T13:57:06Z")
**How severe does this issue affect your experience of using Ray?** - Medium: It contributes to significant difficulty to complete my task, but I can work around it. **Questions** I run experiments and reported metrics and checkpoints by calling: ```auto session.report( {"mrr": mrr, "mr": mr, "hits10": hits10, "hits5": hits5, "hits3": hits3, "hits1": hits1}, checkpoint=checkpoint) ``` Then I create an ExperimentAnalysis object: ```auto analysis = ExperimentAnalysis(experiment_checkpoint_path=RUN_NAME) ``` Then I call the method `get_best_logdir` as follows: ```auto best_run_name = analysis.get_best_logdir("mrr", mode="max") ``` and get the error message `Could not find best trial. Did you pass the correct `metric` parameter?`. However, if I restore the experiment with `Tuner.restore`, can access the `mrr` metric. Why is the `ExperimentAnalysis` not working as expected? ---
**Author:** ![justinvyu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/justinvyu/32/4309_2.png) [@justinvyu](https://discuss.ray.io/u/justinvyu)\ **Post date:** [March 7, 2023, 12:35am UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643/2 "2023-03-07T00:35:32Z")
Hi @mblum, Could you try using the new `ResultGrid` API instead? See here for a guide: [Analyzing Tune Experiment Results — Ray 2.3.0](https://docs.ray.io/en/latest/tune/examples/tune_analyze_results.html) ```python result_grid = Tuner.restore("/path/to/experiment").get_results() best_run_path = result_grid.get_best_result(metric="mrr", mode="max").uri ``` To figure out the existing issue, what is the `RUN_NAME` that you’re passing into `ExperimentAnalysis`? ---
**Author:** ![mblum](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mblum/32/4019_2.png) [@mblum](https://discuss.ray.io/u/mblum)\ **Post date:** [March 7, 2023, 8:20am UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643/3 "2023-03-07T08:20:37Z")
Hi @justinvyu, thank you very much for your help. I have tried out your code and got the same error: `RuntimeError: No best trial found for the given metric: mrr. This means that no trial has reported this metric, or all values reported for this metric are NaN. To not ignore NaN values, you can set the `filter_nan_and_inf` arg to False.` The `RUN_NAME` is the path to the Ray experiment (‘’/media/compute/homes/mblum/ray\_results/feature\_vs\_objective\_link\_pred\_2023-02-23\_10-36-24"). However, the path must be correct because the restoring seems to work correctly. Moreover, if I call `results._experiment_analysis._trial_dataframes`, the dataframes contain the `mrr` score. ```auto [1 rows x 19 columns], '/media/compute/homes/mblum/ray_results/feature_vs_objective_link_pred_2023-02-23_10-36-24/train_89bf9_00003_3_alpha=0.5905,batch_norm=False,batch_size=256,dim=100,dropout=0,eta=1,lr=0.0002,reg=False_2023-02-23_10-36-46': mrr time_this_iter_s should_checkpoint done timesteps_total episodes_total training_iteration trial_id experiment_id ... timestamp time_total_s pid hostname node_ip time_since_restore timesteps_since_restore iterations_since_restore warmup_time 0 tensor(0.0018) 288.402447 True True NaN NaN 1 89bf9_00003 7e67e64d429e43038068b31bdee51141 ... 1677145300 288.402447 8790 handy 129.70.137.103 288.402447 0 1 0.004892 ``` ---
**Author:** ![justinvyu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/justinvyu/32/4309_2.png) [@justinvyu](https://discuss.ray.io/u/justinvyu)\ **Post date:** [March 7, 2023, 6:01pm UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643/4 "2023-03-07T18:01:37Z")
@mblum The issue seems to be that your metric is passed to Tune as a `tensor` object, rather than a float. Could you try reporting `mrr.item()` (if you’re using PyTorch) instead? ---
**Author:** ![mblum](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mblum/32/4019_2.png) [@mblum](https://discuss.ray.io/u/mblum)\ **Post date:** [March 8, 2023, 9:36am UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643/5 "2023-03-08T09:36:26Z")
@justinvyu That’s what I am trying out right now. It seems to work. However, that’s bad news for the experiments I have to re-run. ---
**Author:** ![justinvyu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/justinvyu/32/4309_2.png) [@justinvyu](https://discuss.ray.io/u/justinvyu)\ **Post date:** [March 8, 2023, 5:39pm UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643/6 "2023-03-08T17:39:20Z")
Got it, you could also find the best result through the dataframe directly if you don’t want to re-run the experiments. Generally, you should report primitive types through `session.report(metrics)`, but it technically does allow all sorts of serializable objects to be reported. So, it’s hard to throw a warning during logging. We may want to add a try/catch to convert to a primitive type, when using it to order trials. ---
**Author:** ![mblum](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mblum/32/4019_2.png) [@mblum](https://discuss.ray.io/u/mblum)\ **Post date:** [March 10, 2023, 10:55am UTC](https://discuss.ray.io/t/experimentanalysis-did-you-pass-the-correct-metric-parameter/9643/7 "2023-03-10T10:55:11Z")
Thank you very much @justinvyu ! I will follow your suggestion. You are right, some warning or automatic type conversion would be great. However, making this more clear in the documentation would actually help a lot in the future.