How to keep frame and detected boundingboxes in order for object tracker

Ray Serve
1

1. Severity of the issue: (select one)
[+] Medium: Significantly affects my productivity but can find a workaround.

2. Environment:

  • Ray version: 2.43.0
  • Python version: 3.10.6
  • OS: EulerOSv2R10

3. What happened vs. what you expected:

  • Expected: Keeping frame and detected boundingboxes in order for object tracker
  • Actual: It is my first time to learning Ray, I have some componets video input, preprocess, detection (e.g. DETR model), postprocess (e.g. filter box by score threshold) and tracker (i.e. video input -> preprocess -> detection -> postprocess -> tracker ). I want to use them by adding @ray.remote or @serve.deployment. I find that @ray.remote have num_returns = streaming, could I use it to make the comming input boundingboxes for tracker in order by frame index? Or should I create a priority queue to sort frame index (with recording the smallest next frame index) to select one by one? Could anyone support a brief method for reference? Thanks in advance.
2

Hi @czjghost, welcome to the community and thanks for your post! num_returns = streaming will not ensure frame order out of the box, it only means that the output of the remote call can be streamed. As a result, you will need to implement frame order yourself so that the tracker component can track objects across frames correctly. A priority queue seems reasonable, or buffering the frames by frame index and waiting until the expected next frame is in the buffer.

3

okay, I got, thanks for your reply