2.58.2 libgio.so breaks Linux ldconfig

[pkgw]

Hoo boy. I just discovered a fun esoteric problem with the Linux binaries distributed with the new version 2.58.2.

This new version uses the Meson build system. It turns out that upon installation, Meson will automatically modify the RPATH data for ELF libraries and executables that it installs, doing the same sort of consistency work that the Conda build system does. The relevant file is called depfixer.py.

The problem is that the rewriting done during this process can create binaries that are OK according to the ELF spec, but are confusing to certain tools that are naive in how they parse ELF executables. Relevant discussion is here.

Unfortunately, one of these naive tools is ldconfig. And in the case of our glib package, what happens is that ldconfig decides that the SONAME of libgio-2.0.so is actually \n (i.e. a single newline character), which leads it to create a file called $PREFIX/lib/\n — the first time I've ever run across an instance of the fun Unix feature that it is legal to create files with newlines in their names. Note that if you readelf -d the resulting file, you'll get the right answer, because readelf is sufficiently smart. chrpath -l will give you the wrong answer because it is not.

Finally, if you're trying to build a package that depends on glib and calls ldconfig, this new file will confuse Conda's verifier that tries to compare the contents of the resulting tarball and the Conda file listing, resulting in a very very very hard-to-trace-down Exception('info/files').

I think the best solution here is to turn off Meson's RPATH munging. Unfortunately, I can't find a built-in way to turn it off, so I think we'll have to patch it.

CC @conda-forge/meson.

[pkgw]

Upon further investigation, the problem is only caused when Meson's parameter install_rpath for an executable is empty — it triggers a separate codepath that tries to delete the RPATH record altogether, and that's the thing that works in a way that causes ldconfig to have problems. So, it looks like we can avoid this problem by patching glib's build files to set install_rpath for the libraries that it installs. Pull request in the works.

[SoapZA]

I would like to get @jpakkane's opinion on this problem. There's a good reason Meson does this RPATH hackery - it avoids potentially extremely costly relinking operations (given how slow ld.bfd is). Nonetheless, we shouldn't try to break standard workflows.

[pkgw]

@SoapZA Hopefully mesonbuild/meson#4685 will be a good place for that discussion to happen. Hacking RPATH is something I can definitely get behind — conda is built on it — but this particular implementation seems to have unfortunate corner cases.

[kleisauke]

Instead of patching all the libraries that are affected by this, can't we just patch the meson-feedstock? See for example this patch:
https://gitlab.com/buildroot.org/buildroot/blob/master/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch

(comment mesonbuild/meson#314 (comment) might be relevant here)

[SoapZA]

@jpakkane Maybe we should add this in Meson instead?

[jpakkane]

Note that having an empty rpath entry in the ELF file is different from not having it at all. No, I don't know how, but this caused problems in some distro (Debian?) where it triggered something. First you probably want to examine is this will cause issues to you.

[pkgw]

I am planning to incorporate the buildroot patch into the pending 0.53.0 package of meson (conda-forge/meson-feedstock#26). In the broader ecosystem I can see how issues like the Debian corner case might crop up, but in Conda we are already munging RPATHs and forcing them to have specific, nonempty paths, so I don't believe we'll encounter any problems here.

[kleisauke]

All those install_rpath patches needs probably to be reverted after patching Meson. The fix_rpath function will still be called if install_rpath is non-empty.

Nevertheless, the gstreamer-orc-feedstock has a regression test for detecting this:
https://github.com/conda-forge/gstreamer-orc-feedstock/blob/master/recipe/meta.yaml#L29-L30

[pkgw]

@kleisauke Since the install_rpath patches add non-empty RPATHs, I believe that those kinds of patches can be reverted but won't need to be reverted. But, I just tried to build gstreamer-orc-feedstock with meson 0.53 — without adding in the buildroot patch — and got an early error:

Project name: orc
Project version: 0.4.31
Appending CFLAGS from environment: '-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/gstreamer-orc-0.4.31 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix'
Appending LDFLAGS from environment: ['-Wl,-O2', '-Wl,--sort-common', '-Wl,--as-needed', '-Wl,-z,relro', '-Wl,-z,now', '-Wl,--disable-new-dtags', '-Wl,--gc-sections', '-Wl,-rpath,$PREFIX/lib', '-Wl,-rpath-link,$PREFIX/lib', '-L$PREFIX/lib']
Appending CPPFLAGS from environment: '-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem $PREFIX/include'

meson.build:1:0: ERROR: Unable to determine dynamic linker

I'll take a look and see if I can figure out this issue and see if there's an issue with meson 0.53 in general, or what. (If you have the chance to also take a look that would be awesome!)

[kleisauke]

Ah I see, since it adds non-empty RPATHs this issue would not manifest.

It looks like the Unable to determine dynamic linker error is a meson 0.53 bug, see: mesonbuild/meson#6431

[pkgw]

Thanks @kleisauke , I didn't do any googling. Looks like we should monitor and hold off deploying 0.53.0 for the time being.

[kleisauke]

Indeed it's wiser to stick with version 0.52.1. You may also need to increase the Python requirement to >=3.5.2 in the meson feedstock when version 0.53.1 is released: mesonbuild/meson@23ef804.

[pkgw]

Meson 0.53.1 is out and we have a candidate package at conda-forge/meson-feedstock#27.