# Loading Geotiff Images Into Ray Dataset

**URL:** <https://discuss.ray.io/t/loading-geotiff-images-into-ray-dataset/13997>\
**Category:** Ray Data\
**Created:** [March 13, 2024, 10:13pm UTC](https://discuss.ray.io/t/loading-geotiff-images-into-ray-dataset/13997 "2024-03-13T22:13:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kaylahardie](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kaylahardie/32/5839_2.png) [@kaylahardie](https://discuss.ray.io/u/kaylahardie)\
**Post date:** [March 13, 2024, 10:13pm UTC](https://discuss.ray.io/t/loading-geotiff-images-into-ray-dataset/13997/1 "2024-03-13T22:13:18Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I’m trying to migrate a geospatial ML inference pipeline to use ray.  
The first step in our pipeline is loading a bunch of geotiff files that represent satellite images. **Where I’m stuck is trying to figure out the best way to get the geotiffs in the ray dataset**

So far:  
I tried using ray.data.read\_images but it relies on pil/pillow and pil says that they support tifs ([Image file formats - Pillow (PIL Fork) 10.2.0 documentation](https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#tiff)) but pil doesn’t seem to be very robust at opening geotiffs ([is it possible to open a geotiff file in python without using gdal? - Geographic Information Systems Stack Exchange](https://gis.stackexchange.com/questions/283141/is-it-possible-to-open-a-geotiff-file-in-python-without-using-gdal)). I tried opening 4 different geotiffs and only 1 worked with pil and it wasn’t clear why.  
I tried using gdal to open the geotiff and then store it in a python object and use ray’s from\_items loader but it had trouble serializing the gdal opened image.

Do you have any tips on how to load geotiffs into a ray dataset?

---

<div class="post-metadata">

**Author:** ![sjl](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sjl/32/6672_2.png) [@sjl](https://discuss.ray.io/u/sjl)\
**Post date:** [March 14, 2024, 5:58pm UTC](https://discuss.ray.io/t/loading-geotiff-images-into-ray-dataset/13997/2 "2024-03-14T17:58:43Z")

</div>

One possibly way would be to read the geotiff files in with `ray.data.read_binary_files()` ([docs](https://docs.ray.io/en/latest/data/api/doc/ray.data.read_binary_files.html#ray.data.read_binary_files)), then decode each file with a function passed to `Dataset.map()` ([docs](https://docs.ray.io/en/latest/data/api/doc/ray.data.Dataset.map.html#ray.data.Dataset.map)). This decode function would likely use PIL / gdal, essentially taking 1 file and translating to the actual object format you want to use in the dataset.

```auto
ds = ray.data.read_binary_files(file_paths).map(decode_fn)

```

---

<div class="post-metadata">

**Author:** ![kaylahardie](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kaylahardie/32/5839_2.png) [@kaylahardie](https://discuss.ray.io/u/kaylahardie)\
**Post date:** [March 20, 2024, 9:06pm UTC](https://discuss.ray.io/t/loading-geotiff-images-into-ray-dataset/13997/3 "2024-03-20T21:06:49Z")

</div>

Thanks sjl!  
This helped me. I used ray’s read\_binary\_files api like you suggested and then I used gdals virtual file system ([GDAL Virtual File Systems (compressed, network hosted, etc...): /vsimem, /vsizip, /vsitar, /vsicurl, ... — GDAL documentation](https://gdal.org/user/virtual_file_systems.html)) to have the geotiffs load from memory instead of disk.

---

<div class="post-metadata">

**Author:** ![Sheece](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sheece/32/6814_2.png) [@Sheece](https://discuss.ray.io/u/Sheece)\
**Post date:** [November 22, 2024, 1:06pm UTC](https://discuss.ray.io/t/loading-geotiff-images-into-ray-dataset/13997/4 "2024-11-22T13:06:58Z")

</div>

Hi @kaylahardie can you please share code/recipy how you loaded the geotif?i am thinking ill convert each band to numpy → save nupy array → and load them
