# How can I integrate Apache Kafka with Ray Serve?

**URL:** https://discuss.ray.io/t/how-can-i-integrate-apache-kafka-with-ray-serve/5789
**Category:** Ray Serve
**Created:** [April 14, 2022, 2:02pm UTC](https://discuss.ray.io/t/how-can-i-integrate-apache-kafka-with-ray-serve/5789 "2022-04-14T14:02:03Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![youssefbenfarhat](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/youssefbenfarhat/32/2395_2.png) [@youssefbenfarhat](https://discuss.ray.io/u/youssefbenfarhat)
#### Post date: [April 14, 2022, 2:02pm UTC](https://discuss.ray.io/t/how-can-i-integrate-apache-kafka-with-ray-serve/5789/1 "2022-04-14T14:02:03Z")

</div>

Hello guys,  
I’m still beginner with Ray Serve and I want to ask you if it is possible to use Apache Kafka with Ray Serve.

I found this blog but I find it a little bit hard to understand the steps [Anyscale - Serverless Kafka Stream Processing with Ray](https://www.anyscale.com/blog/serverless-kafka-stream-processing-with-ray).

Thank you so much.

---

<div class="post-metadata">

### Author: ![shrekris](https://avatars.discourse-cdn.com/v4/letter/s/ce73a5/32.png) [@shrekris](https://discuss.ray.io/u/shrekris)
#### Post date: [April 14, 2022, 5:12pm UTC](https://discuss.ray.io/t/how-can-i-integrate-apache-kafka-with-ray-serve/5789/2 "2022-04-14T17:12:22Z")

</div>

I believe that this is possible. That blog walks through how to build a Ray application that uses Kafka, but you should be able to do something similar with Serve. Instead of using Ray [tasks](https://docs.ray.io/en/master/ray-core/tasks.html) and [actors](https://docs.ray.io/en/releases-1.11.0/ray-core/actors.html) (which are decorated with `@ray.remote`), you can use Serve [deployments](https://docs.ray.io/en/releases-1.11.0/serve/core-apis.html) (which are decorated with `@serve.deployment`).

In general, Ray Serve accepts any arbitrary Python code, so if you can access Kafka with Python, Ray Serve should be able to support it.

You might be able to follow that blog post using the `@serve.deployment` decorator instead of `@ray.remote`. However, I would recommend first going through the [Ray Serve end-to-end tutorial](https://docs.ray.io/en/releases-1.11.0/serve/end_to_end_tutorial.html) to get more context on Serve.

---

<div class="post-metadata">

### Author: ![phi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/phi/32/2315_2.png) [@phi](https://discuss.ray.io/u/phi)
#### Post date: [May 10, 2022, 5:46pm UTC](https://discuss.ray.io/t/how-can-i-integrate-apache-kafka-with-ray-serve/5789/3 "2022-05-10T17:46:03Z")

</div>

You can also try Rayvens which integrates with Kafka.

> **[rayvens/examples at main · project-codeflare/rayvens](https://github.com/project-codeflare/rayvens/tree/main/examples)**
>
> main/examples

---

<div class="post-metadata">

### Author: ![Gaurav\_Sanas](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/gaurav_sanas/32/3444_2.png) [@Gaurav\_Sanas](https://discuss.ray.io/u/Gaurav_Sanas)
#### Post date: [November 17, 2022, 12:38pm UTC](https://discuss.ray.io/t/how-can-i-integrate-apache-kafka-with-ray-serve/5789/4 "2022-11-17T12:38:26Z")

</div>

Hi, I am trying to integrate my kafka consumer with Ray.

[Anyscale - Serverless Kafka Stream Processing with Ray](https://www.anyscale.com/blog/serverless-kafka-stream-processing-with-ray) I also read this document this is bit difficult to understand. Do we any other resource which is easy to understand.

---

<div class="post-metadata">

### Author: ![Rostyslav\_N](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rostyslav_n/32/4857_2.png) [@Rostyslav\_N](https://discuss.ray.io/u/Rostyslav_N)
#### Post date: [July 29, 2023, 3:13am UTC](https://discuss.ray.io/t/how-can-i-integrate-apache-kafka-with-ray-serve/5789/5 "2023-07-29T03:13:55Z")

</div>

In case you are still interested in this topic, I have written an article about Ray Serve and Kafka integration: [Integrate Distributed Ray Serve Deployment with Kafka](https://towardsdatascience.com/integrate-distributed-ray-serve-deployment-with-kafka-181403f4e194)
