What are the recommended way to read video files into the ray?

Ray Libraries (Data, Train, Tune, Serve) Ray Data
1

Hey there!
What are the recommended way to read video files into the ray? i.e. mp4 file
I have a mp4 videos to be preprocessed using ray data framework and I want to extract one frame per second from the raw mp4 files. Can you help me with where to start to do that?

2

Answered already on Slack but reposting here incase others are wondering:

If you’re running Ray on Anyscale, use the VideoDatasource API: Video API | Anyscale Docs.

If you’re running open source Ray, implement a custom datasource. Here’s a relevant guide: Advanced: Read and Write Custom File Types — Ray 2.9.3. You can use GitHub - dmlc/decord: An efficient video loader for deep learning with smart shuffling that's super easy to digest to iteratively read video frames.

1 Like