Generate bindings in build.rs to fix cross-compilation #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The pre-generated bindings are not working on the platforms that is not the same platform these bindings are generated at: [/src/lapack.rs](https://github.com/numrs/lapack-sys/blob/6f42e18ff245e0b53527a24211af1f640b4b637d/src/lapack.rs) In particular, this is failing during pretty much any attempt at cross-compilation. And might cause problems diring native compilation on different platform as well. For example, when cross-compiling to aarch64-unknown-linux-gnu, the defaut C integer types are different from what's on x86_64: [build.log.txt](https://github.com/user-attachments/files/17705378/build.log.txt) The [preferred approach[(https://rust-lang.github.io/rust-bindgen/library-usage.html) nowadays is to generate bindings at build-time and then programmatically paste them into lib.rs. This way they come out with all the specifics required for the target platform. Here I: - add `build.rs` script which uses bindgen library to generate `$OUTPUT_DIR/bindings.rs` (in the `target/` directory) - add the functions starting with underscore to the allowlist (`.allowlist_function("^.*_$")`) -`include!()` the `$OUTPUT_DIR/bindings.rs`into `lib.rs` - remove the `generate.sh` script (the `build.rs` is now doing it's job) - remove pre-generated bindings
ivan-aksamentov
added a commit
to numrs/ndarray-linalg
that referenced
this pull request
Nov 11, 2024
Depends on blas-lapack-rs/lapack-sys#15 This replaces hardcoded platform-specific `char*` types with the opaque types from [`core::ffi`](https://doc.rust-lang.org/stable/core/ffi/type.c_char.html). This is necessary when cross-compiling. I believe when interacting with C and Fortran, both `lax` and `lapack-sys` should not assume integer and pointer types. Current setup is biased towards x86_64 platform and it could cause broken builds and undefined behavior on other platforms. For example, cross compilation build could fail with: error[E0308]: mismatched types --> /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lax-0.16.0/src/cholesky.rs:30:26 | 30 | $trf(uplo.as_ptr(), &n, AsPtr::as_mut_ptr(a), &n, &mut info); | ---- ^^^^^^^^^^^^^ expected `*const u8`, found `*const i8` | | | arguments to this function are incorrect ... 41 | impl_cholesky_!(c64, lapack_sys::zpotrf_); | ----------------------------------------- in this macro invocation | = note: expected raw pointer `*const u8` found raw pointer `*const i8` note: function defined here --> /workdir/.build/docker-aarch64-unknown-linux-gnu/aarch64-unknown-linux-gnu/release/build/lapack-sys-5fad11066d38a1a6/out/bindings.rs:12886:12 Full build log for aarch64-unknown-linux-gnu: [build.log.txt](https://github.com/user-attachments/files/17706011/build.log.txt) In this PR I change `i8` to `core::ffi::c_char` - this, for example, resolves to `i8` on x86 and to `u8` on ARM, but is also universal for any platform rust supports, removing bias towards x86.
This was referenced Nov 11, 2024
|
This does not just fail with cross compilation, it fails when compiling natively on non-x86 hosts too. I have had to postpone indefinitely a Rust signal processing project intended to be run on aarch64 because of this issue. |
|
Hello! Sorry for being slow. Now trying to catch up with what has happened. According to the discussion in the linked PR, it has been resolved. At least, as far as |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to: rust-ndarray/ndarray-linalg#384
The pre-generated bindings are not working on the platforms that is not the same platform these bindings are generated at: /src/lapack.rs
In particular, this is failing during pretty much any attempt at cross-compilation. And might cause problems during native compilation on different platforms as well. For example, when cross-compiling to aarch64-unknown-linux-gnu, the defaut C integer types are different from what's on x86_64: build.log.txt
The preferred approach nowadays is to generate bindings at build-time and then programmatically paste them into lib.rs. This way they come out with all the specifics required for the target platform.
Here I:
add
build.rsscript which uses bindgen library to generate$OUTPUT_DIR/bindings.rs(in thetarget/directory)add the functions starting with underscore to the allowlist (
.allowlist_function("^.*_$"))include!()the$OUTPUT_DIR/bindings.rsintolib.rsremove the
generate.shscript (thebuild.rsis now doing it's job)remove pre-generated bindings