I am trying to read a huge parquet dataset which is partitioned. Is it possible to use ray.data.read_parquet to read this dataset only to read a few selected partitions?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
|
Why isn't `ray.data.read_api._get_reader` parallelized?
|
0 | 191 | December 5, 2023 | |
|
Ray Data read Parquet loads all the data in one go
|
4 | 603 | October 21, 2023 | |
|
Hive Partitioned Datasets
|
0 | 461 | July 3, 2023 | |
| Ray Dataset Cannot Read Parquet File | 1 | 655 | August 1, 2022 | |
| Ray.data.filter() much slower than without filter | 8 | 260 | March 6, 2025 |