# Directory structure dataset help

**URL:** <https://discuss.ray.io/t/directory-structure-dataset-help/11255>\
**Category:** Ray Data\
**Created:** [July 4, 2023, 1:42am UTC](https://discuss.ray.io/t/directory-structure-dataset-help/11255 "2023-07-04T01:42:32Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ya\_boi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/ya_boi/32/4690_2.png) [@ya\_boi](https://discuss.ray.io/u/ya_boi)\
**Post date:** [July 4, 2023, 1:42am UTC](https://discuss.ray.io/t/directory-structure-dataset-help/11255/1 "2023-07-04T01:42:32Z")

</div>

Hi everybody,

I have a challenge, I have grown accustomed to the Pytorch dataset framework where to create a dataset I create a custom Pytorch dataset object. For my dataset which is generated using Nvidia omniverse, I have a directory structure holding all my data, I will have a folder of rgb images and a folder of segmentation images and a folder of jsons ect. When I build the dataset the pytorch way I initialize by creating a table of paths so when **getitem** is called I will lookup the paths for that item and load each piece of data accordingly. But I was swayed when learning of the parallelism, sharding, and abstractions (local filesystem or s3 for example) of the ray datasets. However, I don’t see a clear way to transform my directory-structured dataset into a ray dataset. If anybody has any suggestions on how to go about doing this I would deeply appreciate it!