Handle Partially Corrupted Videos

[phuturesh]

🚀 The feature

In the scanning procedure, a video corruption may stop scanning, and it will fall on an Error in the decoding procedure. I expect the scanning process to be more robust, allowing it to ignore such corruptions and continue scanning the remaining frames.

Motivation, pitch

As mentioned in #165, I will repeat my comment here for clarity:

Currently, the VideoDecoder may raise a RuntimeError or other exceptions when decoding videos that are partially corrupted. During the scanning process, it seems to just stop upon encountering the first error.

Ex. In a video containing 10,000 frames with corruption around the 4,000th frame, VideoDecoder._num_frames will only be about 4,000. The last 6,000 frames are discarded and we can not access them through VideoDecoder.

The thing is many real-world videos contain partial corruptions, yet various media players manage to ignore this and play them like there are no errors. Now the implementation of VideoDecoder may not properly align with this approach.

There may be a potential solution. In the scanning process we should somehow get the ``true'' num_frames. And in the decoding process, we can either

1. ignore a corrupted frame and return a 0 tensor.
2. or raise an Error, so programmers are to handle it. I prefer this one.

[phuturesh]

The above unexpected behavior seems to be a bug triggered when reading from bytes:

with open('xxx.mp4', "rb") as f:
    raw_video_bytes = f.read()
from torchcodec.decoders import VideoDecoder
decoder = VideoDecoder(raw_video_bytes, device='cuda')

The _num_frames is as expected when directly reading from a file path like:
decoder = VideoDecoder('xxx.mp4', device='cuda')

[scotts]

@phuturesh, can you point us to a video that triggers this behavior?