dependencies: add pybind11 custom factory

[eli-schwartz]

This works with pkg-config and cmake without any special support. The custom factory adds further support for config-tool, via pybind11-config. This is useful because the config-tool will work out of the box when pybind11 is installed, but the pkg-config and cmake files are shoved into python's site-packages, which is an unfortunate distribution model and makes it impossible to use in an out of the box manner.

It's possible to manually set up the PKG_CONFIG_PATH to detect it anyway, but in case that does not happen, having the config-tool fallback is extremely useful.

Depends on pybind/pybind11#4526 (sort of, the version will be unknown without that PR)

[eli-schwartz]

/cc @rgommers for SciPy, @henryiii for pybind11

[codecov]

Codecov Report

Merging #11463 (21b6ef7) into master (162ac25) will increase coverage by 1.71%.
The diff coverage is n/a.

❗ Current head 21b6ef7 differs from pull request most recent head 62c269d. Consider uploading reports for the commit 62c269d to get more accurate results

@@            Coverage Diff             @@
##           master   #11463      +/-   ##
==========================================
+ Coverage   67.10%   68.82%   +1.71%     
==========================================
  Files         418      209     -209     
  Lines       91126    45574   -45552     
  Branches    20247     9434   -10813     
==========================================
- Hits        61154    31367   -29787     
+ Misses      25250    11784   -13466     
+ Partials     4722     2423    -2299     

see 211 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[rgommers]

@eli-schwartz I just realized that this is going to break the fix for in-tree venvs in scipy/scipy#18006. I think this implementation should use relpath/abspath logic like: https://github.com/scipy/scipy/blob/edd8fd25eb24d0ce55855ee27e76b596d639be92/scipy/meson.build#L90-L96

[eli-schwartz]

It works for me:

Determining dependency 'pybind11' with pkg-config executable '/usr/bin/pkg-config'
env[PKG_CONFIG_PATH]: 
Called `/usr/bin/pkg-config --modversion pybind11` -> 1
stderr:
Package pybind11 was not found in the pkg-config search path.
Perhaps you should add the directory containing `pybind11.pc'
to the PKG_CONFIG_PATH environment variable
Package 'pybind11', required by 'virtual:world', not found
-----------
pybind11-config binary missing from cross or native file, or env var undefined.
Trying a default pybind11-config fallback at pybind11-config
pybind11-config found: YES (/tmp/scipy/venvify/bin/pybind11-config) 2.11.0
Run-time dependency pybind11 found: YES 2.11.0
Running command: /tmp/scipy/venvify/bin/python -c 'import os
os.chdir(os.path.join("..", "tools"))
import pythran
try:
  incdir = os.path.relpath(pythran.get_include())
except Exception:
  incdir = pythran.get_include()
print(incdir)
'
--- stdout ---
../../../usr/lib/python3.10/site-packages/pythran

--- stderr ---


Pkg-config binary for 1 is cached.
Determining dependency 'openblas' with pkg-config executable '/usr/bin/pkg-config'

from build.ninja:

build scipy/fft/_pocketfft/pypocketfft.cpython-310-x86_64-linux-gnu.so.p/pypocketfft.cxx.o: cpp_COMPILER ../scipy/fft/_pocketfft/pypocketfft.cxx
 DEPFILE = scipy/fft/_pocketfft/pypocketfft.cpython-310-x86_64-linux-gnu.so.p/pypocketfft.cxx.o.d
 DEPFILE_UNQUOTED = scipy/fft/_pocketfft/pypocketfft.cpython-310-x86_64-linux-gnu.so.p/pypocketfft.cxx.o.d
 ARGS = -Iscipy/fft/_pocketfft/pypocketfft.cpython-310-x86_64-linux-gnu.so.p -Iscipy/fft/_pocketfft -I../scipy/fft/_pocketfft -I/usr/include/python3.10 -I/tmp/scipy/venvify/lib/python3.10/site-packages/pybind11/include -fvisibility=hidden -fvisibility-inlines-hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++14 -O2 -g -fPIC -pthread -DPOCKETFFT_PTHREADS

No warnings are triggered, because it's only caused by include_directories(), not by dependency(). The latter works at a lower level and assumes that the paths it gets are safe (for example dynamically calculated by the config-tool).

[rgommers]

No warnings are triggered, because it's only caused by include_directories(), not by dependency(). The latter works at a lower level and assumes that the paths it gets are safe (for example dynamically calculated by the config-tool).

Ah okay, thanks. I remembered checking before and dependency(include_directories: ...) triggers the check too. But only using the returned dependency object from dependency('pybind11') does not. Nice:)