# Ray Data LLM APIs

**URL:** https://discuss.ray.io/c/ray-data/llm-batch-inference/28.md

[Latest](https://discuss.ray.io/latest.md) · [Categories](https://discuss.ray.io/categories.md) · [Tags](https://discuss.ray.io/tags.md)

---

## [About the Ray Data LLM APIs category](https://discuss.ray.io/t/about-the-ray-data-llm-apis-category/22204)

<div class="topic-metadata">

**Author:** [@christina](https://discuss.ray.io/u/christina)\
**Replies:** 0\
**Last updated:** [April 2, 2025, 6:24pm UTC](https://discuss.ray.io/t/about-the-ray-data-llm-apis-category/22204 "2025-04-02T18:24:36Z")

</div>

Ray Data has a LLM module that enables efficient batch inference with large language models (LLMs) using Ray Data. It integrates with inference engines like vLLM and OpenAI-compatible APIs, allowing users to process LLM …

---

## [PrepareImageUDF Error "The actor is temporarily unavailable" for ray.data.llm multimodal batch inference](https://discuss.ray.io/t/prepareimageudf-error-the-actor-is-temporarily-unavailable-for-ray-data-llm-multimodal-batch-inference/23249)

<div class="topic-metadata">

**Author:** [@Erickek](https://discuss.ray.io/u/Erickek)\
**Replies:** 4\
**Last updated:** [October 16, 2025, 5:57pm UTC](https://discuss.ray.io/t/prepareimageudf-error-the-actor-is-temporarily-unavailable-for-ray-data-llm-multimodal-batch-inference/23249 "2025-10-16T17:57:35Z")

</div>

tested: ds = ray.data.from\_items( \[ \[{ "role": "system", "content": \[ { "type": "text", "text": "You are a helpful assistant."}…

---

## [Ray data concurrency \> 1 fails](https://discuss.ray.io/t/ray-data-concurrency-1-fails/23082)

<div class="topic-metadata">

**Author:** [@anaykulkarni](https://discuss.ray.io/u/anaykulkarni)\
**Replies:** 1\
**Last updated:** [September 8, 2025, 6:43pm UTC](https://discuss.ray.io/t/ray-data-concurrency-1-fails/23082 "2025-09-08T18:43:58Z")

</div>

Hi guys, I’m new to Ray and I’m facing issues with concurrency while using ray data’s map\_batches. I’m trying to launch 8 vLLM instances in parallel. At first it tries to launch 8 actors, then the model loading starts of…

---

## [LLM Batch Model loading using runai\_streamer is very slow](https://discuss.ray.io/t/llm-batch-model-loading-using-runai-streamer-is-very-slow/22922)

<div class="topic-metadata">

**Author:** [@sologymming](https://discuss.ray.io/u/sologymming)\
**Replies:** 1\
**Last updated:** [August 13, 2025, 6:14pm UTC](https://discuss.ray.io/t/llm-batch-model-loading-using-runai-streamer-is-very-slow/22922 "2025-08-13T18:14:38Z")

</div>

1. Severity of the issue: (select one) None: I’m just curious or want clarification. Low: Annoying but doesn’t hinder my work. Medium: Significantly affects my productivity but can find a workaround. \[V\] High: Com…

---

## [Does map\_batches avoid saturating the inference engine?](https://discuss.ray.io/t/does-map-batches-avoid-saturating-the-inference-engine/22538)

<div class="topic-metadata">

**Author:** [@shevateng0](https://discuss.ray.io/u/shevateng0)\
**Replies:** 1\
**Last updated:** [May 25, 2025, 11:30pm UTC](https://discuss.ray.io/t/does-map-batches-avoid-saturating-the-inference-engine/22538 "2025-05-25T23:30:12Z")

</div>

I am experimenting with Ray Data for batch inference. Ray runs in standalone mode as I only use one GPU node. map\_batches() is used to perform batch inference. I want to know if batch inference via Ray will slow down th…

---

## [Does RayData Support multi-node vllm inference](https://discuss.ray.io/t/does-raydata-support-multi-node-vllm-inference/22530)

<div class="topic-metadata">

**Author:** [@DriverSong](https://discuss.ray.io/u/DriverSong)\
**Replies:** 2\
**Last updated:** [May 23, 2025, 8:14pm UTC](https://discuss.ray.io/t/does-raydata-support-multi-node-vllm-inference/22530 "2025-05-23T20:14:14Z")

</div>

1. Severity of the issue: (select one) None: I’m just curious or want clarification. Low: Annoying but doesn’t hinder my work. Medium: Significantly affects my productivity but can find a workaround. High: Comple…