SSL peer certificate or SSH remote key was not OK

@rentom from the error logs it looks like the problem may be in pyarrow. Can you try s3_filesystem.get_file_info(s3_path) to see if this problem comes up with just pyarrow (without ray data)?

You may be able to circumvent this by using fsspec/s3fs (pip install s3fs) and a pyarrow.fs.PyFileSystem:

from pyarrow.fs import PyFileSystem, FSSpecHandler
pa_fs = PyFileSystem(FSSpecHandler(fs)) 

(fs would be your s3fs.S3FileSystem instance).

Since s3fs uses boto under the hood, it should work out of the box.