Support rpath to locate shared libraries?

[ryanking13]

Proposal

Introduce a new rpath.0 custom section into Emscripten WASM module that stores runtime path for the module and can be used to locate shared libraries per-module (or extend the current dylink.0 section to have such value?)

Background

ELF binaries (or shared libraries) has rpath or runtime path section that is used to locate its dependent shared libraries. In addition to LD_LIBRARY_PATH which is a system-wide path for searching shared libraries, rpath can be used per-module path for searching its dependencies.

For now, -rpath flag is ignored when it is passed to emcc.

emcc: warning: ignoring unsupported linker flag: `-rpath` [-Wlinkflags]

Reason for the proposal

Pyodide loads python packages with shared libraries in runtime. In order to ensure that each Python package is self-contained, Python utilizes rpaths to locate dependent dynamic libraries within a single package.

For instance, numpy Python package for linux has following runtime path:

$ objdump -x lapack_lite.cpython-312-x86_64-linux-gnu.so | grep 'PATH'
  RPATH                $ORIGIN/../../numpy.libs

and it is used to locate shared libraries in numpy.libs, which is not in LD_LIBRARY_PATH.

$ ldd lapack_lite.cpython-312-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffee37cd000)
        libscipy_openblas64_-99b71e71.so => /mnt/c/Users/siha/Downloads/numpy-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64/numpy/linalg/./../../numpy.libs/libscipy_openblas64_-99b71e71.so (0x00007ff9b7c08000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff9b786a000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff9b764b000)
        libgfortran-040039e1-0352e75f.so.5.0.0 => /mnt/c/Users/siha/Downloads/numpy-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64/numpy/linalg/./../../numpy.libs/libgfortran-040039e1-0352e75f.so.5.0.0 (0x00007ff9b719a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff9b6da9000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff9b9da5000)
        libquadmath-96973f99-934c22de.so.0.0.0 => /mnt/c/Users/siha/Downloads/numpy-2.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64/numpy/linalg/./../../numpy.libs/libquadmath-96973f99-934c22de.so.0.0.0 (0x00007ff9b6b6b000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ff9b694e000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff9b6736000)

it would be nice if this is also available in Emscripten.

Alternatives / Current workaround

Currently, Pyodide performs rpath-like behavior in the following ways.

0. We directly use loadDynamicLibrary JS function instead of using dlopen.
1. When calling loadDynamicLibrary, we pass fs parameter with custom findObject function.
   1.a. This function was removed in latest emscripten, but we are using a patched-version of it (Move dlopen file operations into native code. NFC #19310)
2. In out custom findObject function, we search shared libraries by looking at some pre-defined search path per module, which behaves like rpath.

References

• Pyodide Issue for adding rpath
• Previous discussion: https://github.com/emscripten-core/emscripten/pull/19310/files#r1214703517
• Partially related issue: Emscripten calls null GOT entry when side module loaded with allowUndefined: true #22052 (comment)

---

Above all, thank you for maintaining such an amazing project! We (Pyodide) benefit a lot from Emscripten :)

cc: @hoodmane @agriyakhetarpal for visibility

[sbc100]

Seems like a reasonable request yes. It would make sense to add this to the existing dylink.0. That section is deliberately designed to be extensible to include stuff like this.

[ryanking13]

HI @sbc100, what would be the steps I should take to move this forward? I'm not sure exactly what I need to do to add a subsection, but I'm guessing I would need to do the following.

0. Define the format of a new subsection type in dylink.0, such as WASM_DYLINK_RPATH. Maybe in WebAssembly/tool-conventions repository.
1. Update LLVM linker to generate a new subsection.
2. Update library_dylink.js to use this subsection when locating shared libraries.

[sbc100]

Yup those steps look correct to me!

[ryanking13]

Thanks. Let me prepare a PR in the WebAssembly/tool-conventions repository then.

[hoodmane]

@sbc100 can you merge llvm/llvm-project#126080?

[sbc100]

Done