# RayTune Downloading Data from S3

**URL:** <https://discuss.ray.io/t/raytune-downloading-data-from-s3/13701>\
**Category:** Kubernetes\
**Created:** [February 12, 2024, 2:40pm UTC](https://discuss.ray.io/t/raytune-downloading-data-from-s3/13701 "2024-02-12T14:40:01Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![EthanMarx](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/ethanmarx/32/5357_2.png) [@EthanMarx](https://discuss.ray.io/u/EthanMarx)\
**Post date:** [February 12, 2024, 2:40pm UTC](https://discuss.ray.io/t/raytune-downloading-data-from-s3/13701/1 "2024-02-12T14:40:01Z")

</div>

- High: It blocks me to complete my task.

Hello,

I am launching a multi replica ray cluster with kubernetes to support a `RayTune` job. Specifically, I am using `TorchTrainer` to wrap a `lightning` module.

Currently, my lightning module downloads my training data from an S3 bucket to a directory named based off the `worker_id` of the ray worker. However, this means that I am using up extra memory downloading multiple instances of my dataset for each ray worker on the node.

I am wondering what the best practices are for this scenario? Is there some sort of “pre tune” hook I can run on each kubernetes replica that will download the dataset to one directory all my trials can access?

appreciate the help!