Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing definition of OQS_init() and OQS_destroy() breaks compilation #152

Closed
mouse07410 opened this issue May 1, 2023 · 12 comments
Closed

Comments

@mouse07410
Copy link
Contributor

Describe the bug
Current master fails to build - commit fbd2538 is at fault.

Also, fails to clone qsc-key-encoder.git

$ ../oqsprov-build.sh 
make: *** No rule to make target `clean'.  Stop.
Building for source-based OpenSSL-3.2.x-dev...
-- The C compiler identification is AppleClang 14.0.3.14030022
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/local/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Creating Debug build with OQS-Debug env vars enabled
-- Build will store public keys in PKCS#8 structures
-- Build will include external encoding library for SPKI/PKCS#8
-- Found OpenSSL: /Users/ur20980/src/openssl/libcrypto.dylib (found suitable version "3.2.0", minimum required is "3.0")  
-- Building commit fbd2538 in /Users/ur20980/src/oqs-provider
CMake Warning at test/CMakeLists.txt:35 (message):
  OpenSSL source build directory
  '/Users/ur20980/src/oqs-provider/test/../openssl' not present.  Some
  dependent tests will be skipped.


-- Configuring done
-- Generating done
-- Build files have been written to: /Users/ur20980/src/oqs-provider/_build
/opt/local/bin/cmake -S/Users/ur20980/src/oqs-provider -B/Users/ur20980/src/oqs-provider/_build --check-build-system CMakeFiles/Makefile.cmake 0
/opt/local/bin/cmake -E cmake_progress_start /Users/ur20980/src/oqs-provider/_build/CMakeFiles /Users/ur20980/src/oqs-provider/_build//CMakeFiles/progress.marks
/opt/local/bin/gmake  -f CMakeFiles/Makefile2 all
gmake[1]: Entering directory '/Users/ur20980/src/oqs-provider/_build'
/opt/local/bin/gmake  -f oqsprov/CMakeFiles/encoder.dir/build.make oqsprov/CMakeFiles/encoder.dir/depend
gmake[2]: Entering directory '/Users/ur20980/src/oqs-provider/_build'
cd /Users/ur20980/src/oqs-provider/_build && /opt/local/bin/cmake -E cmake_depends "Unix Makefiles" /Users/ur20980/src/oqs-provider /Users/ur20980/src/oqs-provider/oqsprov /Users/ur20980/src/oqs-provider/_build /Users/ur20980/src/oqs-provider/_build/oqsprov /Users/ur20980/src/oqs-provider/_build/oqsprov/CMakeFiles/encoder.dir/DependInfo.cmake --color=
gmake[2]: Leaving directory '/Users/ur20980/src/oqs-provider/_build'
/opt/local/bin/gmake  -f oqsprov/CMakeFiles/encoder.dir/build.make oqsprov/CMakeFiles/encoder.dir/build
gmake[2]: Entering directory '/Users/ur20980/src/oqs-provider/_build'
[  4%] Creating directories for 'encoder'
cd /Users/ur20980/src/oqs-provider/_build/oqsprov && /opt/local/bin/cmake -Dcfgdir= -P /Users/ur20980/src/oqs-provider/_build/oqsprov/encoder-prefix/tmp/encoder-mkdirs.cmake
cd /Users/ur20980/src/oqs-provider/_build/oqsprov && /opt/local/bin/cmake -E touch /Users/ur20980/src/oqs-provider/_build/oqsprov/encoder-prefix/src/encoder-stamp/encoder-mkdir
[  8%] Performing download step (git clone) for 'encoder'
cd /Users/ur20980/src/oqs-provider/_build/oqsprov/encoder-prefix/src && /opt/local/bin/cmake -P /Users/ur20980/src/oqs-provider/_build/oqsprov/encoder-prefix/tmp/encoder-gitclone.cmake
Cloning into 'encoder'...
error: git-remote-https died of signal 11
fatal: remote transport reported error
Cloning into 'encoder'...
error: git-remote-https died of signal 11
fatal: remote transport reported error
Cloning into 'encoder'...
error: git-remote-https died of signal 11
fatal: remote transport reported error
-- Had to git clone more than once: 3 times.
CMake Error at /Users/ur20980/src/oqs-provider/_build/oqsprov/encoder-prefix/tmp/encoder-gitclone.cmake:39 (message):
  Failed to clone repository:
  'https://github.com/Quantum-Safe-Collaboration/qsc-key-encoder.git'

.  .  .

gmake[2]: Entering directory '/Users/ur20980/src/oqs-provider/_build'
[ 22%] Building C object oqsprov/CMakeFiles/oqsprovider.dir/oqsprov.c.o
cd /Users/ur20980/src/oqs-provider/_build/oqsprov && /opt/local/bin/clang -DOQSPROVIDER_VERSION_TEXT=\"0.5.0-dev\" -DOQS_PROVIDER_COMMIT="\" (fbd2538)\"" -DUSE_ENCODING_LIB -Doqsprovider_EXPORTS -I/Users/ur20980/src/openssl/include -I/Users/ur20980/src/oqs-provider/_build/install/include -isystem /opt/local/include/oqs -O3 -std=gnu18 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/opt/local/include -L/opt/local/lib -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -fPIC -Wunused-function -std=gnu11 -MD -MT oqsprov/CMakeFiles/oqsprovider.dir/oqsprov.c.o -MF CMakeFiles/oqsprovider.dir/oqsprov.c.o.d -o CMakeFiles/oqsprovider.dir/oqsprov.c.o -c /Users/ur20980/src/oqs-provider/oqsprov/oqsprov.c
clang: warning: argument unused during compilation: '-L/opt/local/lib' [-Wunused-command-line-argument]
/Users/ur20980/src/oqs-provider/oqsprov/oqsprov.c:555:4: error: call to undeclared function 'OQS_destroy'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
   OQS_destroy();
   ^
1 error generated.
gmake[2]: *** [oqsprov/CMakeFiles/oqsprovider.dir/build.make:79: oqsprov/CMakeFiles/oqsprovider.dir/oqsprov.c.o] Error 1
gmake[2]: Leaving directory '/Users/ur20980/src/oqs-provider/_build'
gmake[1]: *** [CMakeFiles/Makefile2:155: oqsprov/CMakeFiles/oqsprovider.dir/all] Error 2
gmake[1]: Leaving directory '/Users/ur20980/src/oqs-provider/_build'

To Reproduce
Steps to reproduce the behavior:

  1. git clone https://github.com/open-quantum-safe/oqs-provider.git --recurse-submodules
  2. cmake -DCMAKE_BUILD_TYPE=Debug -DOPENSSL_ROOT_DIR="$HOME/src/openssl" -DCMAKE_C_FLAGS="$CFLAGS -I/opt/local/include -L/opt/local/lib" -DCMAKE_VERBOSE_MAKEFILE:BOOL=True -S . -B _build
  3. cmake --build _build
  4. See error

Expected behavior
A normal clean build, like before.

Environment (please complete the following information):

  • OS: MacOS Ventura 13.3.1
  • OpenSSL version 3.1.0 and 3.2.0-dev

Additional context
IMHO, it's not really nice to git-clone other repositories in the mid-build. Let's either define them as submodules, or add as a pre-requisite to the README.

@baentsch
Copy link
Member

baentsch commented May 2, 2023

This error can only happen if liboqs is outdated: OQS_destroy is a recent addition. Please git pull in/for liboqs in your build script (or call scripts/fullbuild.sh -F re-fetching everything -- but yes, I'm aware of the deficiencies of that script).

IMHO, it's not really nice to git-clone other repositories in the mid-build. Let's either define them as submodules, or add as a pre-requisite to the README.

I'd agree. @bhess: Would you want to reconsider how to do this? Having a submodule isn't my first choice given how this can trip up people. If we'd go that way, I'd also suggest adding liboqs as a submodule (also fixing the original error report -- if someone thinks about adding --recurse-submodules....

@bhess
Copy link
Member

bhess commented May 2, 2023

I'd rather use submodules than documenting in the README, also for the liboqs dependency. This way we can also avoid issues like reported here when using outdated dependencies.

@mouse07410
Copy link
Contributor Author

mouse07410 commented May 2, 2023

In my case I already have liboqs installed system-wide for user by different packages, and I don't need or want oqs-provider to clone any build its own copy. No thanks. I'd rather quit using this package than let it define liboqs as a submodule. Next you'd want to pull the whole OpenSSL as a submodule too?

While I don't use qsc separately (so can't use the same argument), surprising error with cmake failing to git-clone it (while "standalone" git has no problem cloning this repo!) makes me very concerned.

A user who's smart enough to clone oqs-provider from GitHub in the first place, is likely smart enough to clone qsc-encoder as well, especially if the Build section of README tells him to. If you aren't sure - just try them. ;)

Update

When adding code patch that depends on a certain version of a certain library - the one sane requirement is to explicitly document that at least in the comments of the offending PR. And not doing this - is bad.

@baentsch
Copy link
Member

baentsch commented May 2, 2023

@mouse07410 Good argument that someone may not want to copy something (again) already present (for other purposes). We certainly never will pull openssl (one reason being oqsprovider being a submodule to it -- and I don't think recursion will go down well there :-)

@bhess Given the rather "unspectacular" reception of QSC encoding what about changing it to an optional component (not pulled by default but only "by documentation"/sample build scripts)? At least the IETF interop community stated pretty clearly they don't think this encoding is needed. If it is, let's reconsider, but for the time being, it's just causing head-scratching (also in this issue)...

@bhess
Copy link
Member

bhess commented May 2, 2023

Point taken @mouse07410.

And yes @baentsch, we can make -DUSE_ENCODING_LIB=OFF the default (build without encoding dependency).

@bhess
Copy link
Member

bhess commented May 2, 2023

While I don't use qsc separately (so can't use the same argument), surprising error with cmake failing to git-clone it (while "standalone" git has no problem cloning this repo!) makes me very concerned.

Could you please report the cmake version you are using?

@mouse07410
Copy link
Contributor Author

Could you please report the cmake version you are using?

Naturally:

$ cmake --version
cmake version 3.24.4

CMake suite maintained and supported by Kitware (kitware.com/cmake).
$ port installed cmake
The following ports are currently installed:
  cmake @3.24.4_0 (active)
$ 

The funny part is that in some cases(when building oqs-provider for the OpenSSL source master residing in ~/src/openssl) it consistently fails, while in other cases (building oqs-provider for the Macports-installed binaries of OpenSSL-3.1.0) it usually succeeds...

@bhess
Copy link
Member

bhess commented May 2, 2023

The only reference I find about segfaulting 'git clone' is related to older libcurl versions in git: https://bugzilla.redhat.com/show_bug.cgi?id=1873327

Running GIT_CURL_VERBOSE=1 cmake --build _build should give more debug output about the failing clone.
Not sure yet how the failing git clone is connected to the openssl dependency used.

@mouse07410
Copy link
Contributor Author

$ curl --version
curl 8.0.1 (x86_64-apple-darwin22.3.0) libcurl/8.0.1 OpenSSL/3.1.0 zlib/1.2.13 brotli/1.0.9 zstd/1.5.5 libidn2/2.3.4 libpsl/0.21.2 (+libidn2/2.3.4) nghttp2/1.52.0
Release-Date: 2023-03-20
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB PSL SSL threadsafe TLS-SRP UnixSockets zstd
$ 

I can't explain it, but this time it did not seem to even need to pull...
curl-debug.txt

@bhess
Copy link
Member

bhess commented May 3, 2023

I can't explain it, but this time it did not seem to even need to pull...
curl-debug.txt

If oqs-provider is updated after @baentsch's PR #153, this is expected (sets -DUSE_ENCODING_LIB=OFF as default, which doesn't pull anything during the build).

@mouse07410
Copy link
Contributor Author

this is expected (sets -DUSE_ENCODING_LIB=OFF as default)

No, I changed this default for this test. Though in general I intend to keep that default for "production" builds.

@mouse07410
Copy link
Contributor Author

OK, the main point of this issue has been addressed. PR should be amended with a comment regarding what version(s) of liboqs it applies to, and a warning that it breaks the provider is an older liboqs is used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants