torchvision 0.15.2 + ffmpeg 6.0 build error: ‘AV_CODEC_CAP_INTRA_ONLY’ was not declared

[AlexanderZatserkovnyy]

🐛 Describe the bug

The build error is if it's compiled with ffmpeg 6:
...
vision-0.15.2/torchvision/csrc/io/decoder/stream.cpp:68:42: error: ‘AV_CODEC_CAP_INTRA_ONLY’ was not declared in this scope; did you mean ‘AV_CODEC_PROP_INTRA_ONLY’
...
The source of the problem is renamed macro AV_CODEC_CAP_INTRA_ONLY. The following trivial patch resolve the issue:
--- a/torchvision/csrc/io/decoder/stream.cpp 2023-09-28 11:34:50.636149910 +1000
+++ b/torchvision/csrc/io/decoder/stream.cpp 2023-09-28 11:32:02.002280926 +1000
@@ -65,7 +65,7 @@
// otherwise set sensible defaults
// with the special case for the different MPEG4 codecs
// that don't have threading context functions

• if (codecCtx_->codec->capabilities & AV_CODEC_CAP_INTRA_ONLY) {

• if (codecCtx_->codec->capabilities & AV_CODEC_PROP_INTRA_ONLY) {
  codecCtx_->thread_type = FF_THREAD_FRAME;
  codecCtx_->thread_count = 2;
  } else {

Versions

Used python 3.10, pytorch-2.0.1, caffe2-2.0.1
I don't use pip on the host, so:
python collect_env.py
Collecting environment information...
Traceback (most recent call last):
File "/home/avz/collect_env.py", line 611, in
main()
File "/home/avz/collect_env.py", line 594, in main
output = get_pretty_env_info()
File "/home/avz/collect_env.py", line 589, in get_pretty_env_info
return pretty_str(get_env_info())
File "/home/avz/collect_env.py", line 427, in get_env_info
pip_version, pip_list_output = get_pip_packages(run_lambda)
File "/home/avz/collect_env.py", line 399, in get_pip_packages
out = run_with_pip([sys.executable, '-mpip'])
File "/home/avz/collect_env.py", line 384, in run_with_pip
for line in out.splitlines()
AttributeError: 'NoneType' object has no attribute 'splitlines'

[pmeier]

Unfortunately, we are currently stuck on

vision/.github/scripts/setup-env.sh

Line 44 in 1a9ff0d

'ffmpeg<4.3'

[ganessh22]

@pmeier I faced the same error above when I had installed ffmpeg 6.0.0. I followed this and tried out compiling master with 4.2.9 but then I hit :

The installed version of ffmpeg is missing the header file 'bsf.h' which is required for GPU video decoding. Please install the latest ffmpeg from conda-forge channel: conda install -c conda-forge ffmpeg.

which tells to install version 6.0.0 via conda. Docs here as well say the same.

UPDATE : Was able to get past this by copying the bsf.h file from the git repository of FFMPEG to /usr/local/include/libavcodec/. Have gotten torchvision working now.

[samthephantom]

Might I ask a silly question?

Is it really necessary to compile torchvision with ffmpeg? I need the 0.15.1 version to work with torch 2.0.0+nv23.05, and I need the ffmpeg + nvdec which are used to decode video with hardware.

[zboszor]

ffmpeg has this more than 3 years old changelog entry:

2020-05-21 - 13b1bbff0b - lavc 58.86.101 - avcodec.h
  Deprecated AV_CODEC_CAP_INTRA_ONLY and AV_CODEC_CAP_LOSSLESS.

ffmpeg 6.0 removed those symbols.

[zboszor]

Here's the patch to make torchvision 0.16.0 build with ffmpeg 6.0:

diff --git a/torchvision/csrc/io/decoder/stream.cpp b/torchvision/csrc/io/decoder/stream.cpp
index 0d625ef211..8c91405058 100644
--- a/torchvision/csrc/io/decoder/stream.cpp
+++ b/torchvision/csrc/io/decoder/stream.cpp
@@ -63,15 +63,8 @@ int Stream::openCodec(std::vector<DecoderMetadata>* metadata, int num_threads) {
     codecCtx_->thread_count = num_threads;
   } else {
     // otherwise set sensible defaults
-    // with the special case for the different MPEG4 codecs
-    // that don't have threading context functions
-    if (codecCtx_->codec->capabilities & AV_CODEC_CAP_INTRA_ONLY) {
-      codecCtx_->thread_type = FF_THREAD_FRAME;
-      codecCtx_->thread_count = 2;
-    } else {
-      codecCtx_->thread_count = 8;
-      codecCtx_->thread_type = FF_THREAD_SLICE;
-    }
+    codecCtx_->thread_count = 8;
+    codecCtx_->thread_type = FF_THREAD_SLICE;
   }
 
   int ret;