There are various examples how Ray can read and write data from Amazon S3, for example ds.repartition(1).write_json("s3://bucket/inference-results") How to configure Ray with S3 credentials? I don’t run Ray in AWS, I run it locally on my laptop and want to read data from my Amazon S3 and also writ…

What’s the error message that you get? You should be able to get this working with aws configure and making sure you have read/write permissions to your S3 bucket.

If i use Ray cluster, with few worker nodes, then i also need to run aws configure as well? I thought there is some configuration option in Ray itself to configure S3 access… Thanks

Yeah, you need to either run aws configure or set the relevant AWS environment variables to access S3. cc @Clark_Zinzow

How to configure Ray to access Amazon S3

Ray Data

Clark_Zinzow January 7, 2022, 7:27am 5

@rliaw Answered in this thread in Core posted by the same user: How to access Amazon S3 - #2 by Clark_Zinzow

Topic		Replies	Views
How to access Amazon S3 Ray Core	3	4529	June 19, 2023
Downloading working directory from private S3 storage Ray Core	5	460	February 5, 2025
Unable to locate credentials with S3 Remote URIs Ray Core	3	706	March 1, 2024
AWS Permission issue while creating a cluster Ray Clusters	3	2066	April 14, 2022
RayTune S3 Access Error Ray Tune	0	260	February 14, 2024

How to configure Ray to access Amazon S3

Related topics