How severe does this issue affect your experience of using Ray? High: It blocks me to complete my task. Hi, I want to connect Ray to a private S3 storage and read working_dir files from there. To download the working_dir I’ve set the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_ENDPOINT_URL …

Hi stwerner97, welcome to the Ray community! :smiley: So there’s a few different things about authenticating with Ray, namely, Ray relies on environment variables, shared credentials files, or IAM roles for AWS authentication. It doesn’t explicitly pass credentials to boto3. Experiment Tracking — R…

Hi @christina , thanks for the response! :slight_smile: Yes, I can confirm that. I’ve tried this with a custom script that uses boto3 to download the working directory from S3 (which I want Ray to do). Yes, this also works. I have tried this. Adding the access keys as environment variables to the …

Hi stwerner97, Can you elaborate a bit more on your use case, like what exactly you mean by “other users to pass their credentials”? I think that would help me debug this a lot! Tyvm! :slight_smile:

Hi @christina , thanks for looking into this. I want developers (users) to be able to deploy their own Ray serve applications on an existing Ray cluster that is deployed on Kubernetes with KubeRay. This includes, for example, to serve an object detection model with custom pre- and post-processing c…

Downloading working directory from private S3 storage

Ray Core

stwerner97 January 24, 2025, 1:24pm 2

This issue seems related: Unable to locate credentials with S3 Remote URIs

Topic		Replies	Views
Unable to locate credentials with S3 Remote URIs Ray Core	3	706	March 1, 2024
How to access Amazon S3 Ray Core	3	4530	June 19, 2023
How to configure Ray to access Amazon S3 Ray Data	4	2462	January 7, 2022
How can I use aws secret managar in ray server yaml file for github secrets Ray Serve	1	207	April 8, 2024
Unable to download whl package from amazon Ray Core	3	454	December 13, 2021

Downloading working directory from private S3 storage

Related topics