# Ray.get doesn't return results even all the tasks are finished **URL:** \ **Category:** Ray Core\ **Created:** [April 30, 2023, 5:30pm UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450 "2023-04-30T17:30:38Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![zhiyuanpeng](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/zhiyuanpeng/32/4338_2.png) [@zhiyuanpeng](https://discuss.ray.io/u/zhiyuanpeng)\ **Post date:** [April 30, 2023, 5:30pm UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450/1 "2023-04-30T17:30:39Z")
**How severe does this issue affect your experience of using Ray?** - High: It blocks me to complete my task. ```auto @ray.remote(max_retries=-1) def build_inverted_list(file_path): vocab = set() inverted_list = defaultdict(set) item_attr = pd.read_parquet(file_path) # limit = 0 for _, row in item_attr.iterrows(): # if limit > 3000: # break # limit += 1 item_id = row[convert_dict["item_id"]] for token in standard_tokenizer(row[convert_dict["title"]]): if token: inverted_list[token].add(item_id.encode()) vocab.add(token) assert len(vocab) == len(inverted_list) return (inverted_list, vocab) # I only copy the related code here for concise files = sorted(Path(db_path).glob("*")) lists_vocabs = [build_inverted_list.remote(file_path) for file_path in files] print("collect inverted lists ...") lists_vocabs = ray.get(lists_vocabs) print("collect inverted lists finished ...") ``` I have 181 files under my `db_path` folder. When I uncomment the code about `limit` above to only read 3000 rows of each file, the code works as excepted, so the code should have no bugs. But when I read all the rows (about 800K per file), the code stucks at the `lists_vocabs = ray.get(lists_vocabs)` for 20h even the dashboard shows that all the tasks are done within about 2.5 h. I run the code on GCP n1-standard-96. Env: python 3.8, and latest ray 2.4.0. Anyone knows how to solve or debug this issue? Thanks. ---
**Author:** ![Jules\_Damji](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jules_damji/32/4058_2.png) [@Jules\_Damji](https://discuss.ray.io/u/Jules_Damji)\ **Post date:** [May 1, 2023, 5:00pm UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450/2 "2023-05-01T17:00:57Z")
@zhiyuanpeng Does the dashboard show any OOM errors, or do the logs show any OOM errors? cc: @rickyyx @sangcho ---
**Author:** ![zhiyuanpeng](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/zhiyuanpeng/32/4338_2.png) [@zhiyuanpeng](https://discuss.ray.io/u/zhiyuanpeng)\ **Post date:** [May 1, 2023, 5:51pm UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450/3 "2023-05-01T17:51:31Z")
Hi Jules, thanks for your reply. I am rerunning the code and will let you know later. ---
**Author:** ![zhiyuanpeng](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/zhiyuanpeng/32/4338_2.png) [@zhiyuanpeng](https://discuss.ray.io/u/zhiyuanpeng)\ **Post date:** [May 1, 2023, 10:52pm UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450/4 "2023-05-01T22:52:14Z")
Hi Jules @Jules_Damji, I copy the error message from dashboard.err: Traceback (most recent call last): 2 File “/home/xx/install/miniconda3/envs/py38/lib/python3.8/site-packages/ray/\_private/gcs\_utils.py”, line 124, in check\_health 3 resp = stub.CheckAlive(req, timeout=timeout) 4 File “/home/xx/install/miniconda3/envs/py38/lib/python3.8/site-packages/grpc/\_channel.py”, line 946, in **call** 5 return \_end\_unary\_response\_blocking(state, call, False, None) 6 File “/home/xx/install/miniconda3/envs/py38/lib/python3.8/site-packages/grpc/\_channel.py”, line 849, in \_end\_unary\_response\_blocking 7 raise \_InactiveRpcError(state) 8grpc.\_channel.\_InactiveRpcError: \<\_InactiveRpcError of RPC that terminated with: 9 status = StatusCode.DEADLINE\_EXCEEDED 10 details = “Deadline Exceeded” 11 debug\_error\_string = “UNKNOWN:Deadline Exceeded {created\_time:“2023-05-01T18:07:09.933881436+00:00”, grpc\_status:4}” ---
**Author:** ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)\ **Post date:** [May 2, 2023, 10:11am UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450/5 "2023-05-02T10:11:59Z")
That seems unrelated actually. What’s the version of Ray are you using? Also how did you verify all the tasks are done? ---
**Author:** ![zhiyuanpeng](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/zhiyuanpeng/32/4338_2.png) [@zhiyuanpeng](https://discuss.ray.io/u/zhiyuanpeng)\ **Post date:** [May 2, 2023, 10:11pm UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450/6 "2023-05-02T22:11:33Z")
@sangcho I check the dashboard and find all the tasks are done. I run the code on GCP n1-standard-96. Env: python 3.8, and latest ray 2.4.0. ---
**Author:** ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)\ **Post date:** [May 11, 2023, 1:49am UTC](https://discuss.ray.io/t/ray-get-doesnt-return-results-even-all-the-tasks-are-finished/10450/7 "2023-05-11T01:49:10Z")
Hmm that’s odd. Is there any useful information from the event view from the dashboard? [Ray Dashboard — Ray 3.0.0.dev0](https://docs.ray.io/en/master/ray-core/ray-dashboard.html#overview) I’d also love to pair debug if that works for you