# Dataset support concurrency in one block when using map\_batches **URL:** \ **Category:** Uncategorized\ **Created:** [September 5, 2022, 12:09pm UTC](https://discuss.ray.io/t/dataset-support-concurrency-in-one-block-when-using-map-batches/7440 "2022-09-05T12:09:48Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Basasuya](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@Basasuya](https://discuss.ray.io/u/Basasuya)\ **Post date:** [September 5, 2022, 12:09pm UTC](https://discuss.ray.io/t/dataset-support-concurrency-in-one-block-when-using-map-batches/7440/1 "2022-09-05T12:09:48Z")
Hi, after using ray dataset in large-scale inference. I find **block number** is the max parallelism in map\_batches if we assume resource is unlimited. The batch\_size parameter in map\_batches can only determine the solving batch size in one BlockWorker (the transform inner block is serial), which means can not accelerate the code through tuning batch\_size. I wish the max parallelism in map\_batches is determined by batch\_size. each batch can be solved in concurrent. Is this reasonable? reference from: > > ---
**Author:** ![matthewdeng](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/matthewdeng/32/1446_2.png) [@matthewdeng](https://discuss.ray.io/u/matthewdeng)\ **Post date:** [September 11, 2022, 1:11am UTC](https://discuss.ray.io/t/dataset-support-concurrency-in-one-block-when-using-map-batches/7440/2 "2022-09-11T01:11:27Z")
Hey @Basasuya, can you share more about your use-case and if you’re running into some performance issues because of this? If you have a few large blocks and as a result your parallelization is limited by the number of blocks as opposed to resources, one option is to [repartition](https://docs.ray.io/en/latest/data/api/dataset.html#ray.data.Dataset.repartition) your dataset first so that you have more blocks that are each smaller. ---
**Author:** ![Basasuya](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@Basasuya](https://discuss.ray.io/u/Basasuya)\ **Post date:** [September 15, 2022, 1:09pm UTC](https://discuss.ray.io/t/dataset-support-concurrency-in-one-block-when-using-map-batches/7440/3 "2022-09-15T13:09:53Z")
I have hundreds of files, and each file would be large (serveral GB), my code is like below: ```python dataset = ray.data.read_binary_files(paths=input_path).window(blocks_per_window=5) .map_batches(parse_db_line, batch_size=None, compute=ActorPoolStrategy(5,5,1) .map_batches(predictor, batch_size=None, compute=ActorPoolStrategy(5,5,1), num_gpus=1, num_cpus=1) .write_json(output_path) ``` I think one solution is using repartition\_each\_window, which is the alltoall stage? maybe we can support a onetoone stage repartition in the future which would run faster ---
**Author:** ![jianxiao](https://avatars.discourse-cdn.com/v4/letter/j/e480ec/32.png) [@jianxiao](https://discuss.ray.io/u/jianxiao)\ **Post date:** [September 19, 2022, 11:28pm UTC](https://discuss.ray.io/t/dataset-support-concurrency-in-one-block-when-using-map-batches/7440/4 "2022-09-19T23:28:06Z")
@Basasuya As your understanding, the parallelism is determined by the number of blocks for a Dataset. There is no plan to support parallelization for batches within a block. However, there are a few things to call out: - In processing a batch, the execution is vectorized (by leveraging e.g. the compute kernel of Arrow), so there are parallelization at hardware level already - As mentioned above by @matthewdeng , you can use repartition() to increase the number of blocks. You’re right repartition() is an all-to-all operation, but if you don’t request for shuffle (by default it’s disabled) during repartition, it’s actually quite efficient, likely only involving block splitting (large block split into smaller ones) - If possible, you may shard your input files into smaller ones (e.g. if you are using Spark upstream, that should be doable to repartition before writing out to files) - If not possible, the good news is that we do have a plan to build a feature called Dynamic Block Splitting, which means you can produce multiple (small) blocks from a single input file as you read them into Dataset. It’s estimated to arrive in Ray 2.1 release, stay tuned! ---
**Author:** ![Basasuya](https://avatars.discourse-cdn.com/v4/letter/b/f08c70/32.png) [@Basasuya](https://discuss.ray.io/u/Basasuya)\ **Post date:** [October 1, 2022, 1:35pm UTC](https://discuss.ray.io/t/dataset-support-concurrency-in-one-block-when-using-map-batches/7440/5 "2022-10-01T13:35:00Z")
@jianxiao @matthewdeng thank you for reply I think repartition\_each\_window is useful for me. Looking forward to using Dynamic Block Splitting in future release 🙂