glibcCross: use a libgcc built separately from gcc

[ghost]

Summary

This PR completely and finally solves the gcc<->glibc circular buildInputs problem, for cross compilation. The same technique can be applied to native builds in the future.

Closes #213453

Motivation

Prior to this PR, we had the following circular buildInputs problem:

1. gcc has glibc in its buildInputs

   • a compiled copy of glibc must be present before building gcc; if it isn't, gcc cripples itself (inhibit_libc) and refuses to build libgcc_s.so

2. glibc has libgcc_s.so in its buildInputs

   • glibc dlopen()s libgcc_s.so in order to implement POSIX thread cancellation. For security reasons glibc requires that the path to libgcc_s.so is hardwired into glibc at compile time, so it's technically not a true dynamic link -- it just pretends to be one.

3. libgcc_s.so is built in the same derivation as gcc

   • libgcc_s.so is built as part of the gcc build process

We must cut one of these three links in the loop.

Previous Attempts

Previously #238154 had attempted to cut link (1) by building gcc without glibc, and using the libgcc_s which emerges from that build. Unfortunately this just doesn't work. GCC's configure script extracts quite a lot of information from the glibc headers (which are a build artifact -- you can't just copy them out of the source tarball) and various ./configure-driven linking attempts. If glibc isn't around at build time you wind up with a libgcc_s.so that is missing various unwinder features (see #213453 for the most problematic one).

Musl "cuts" link (2), or rather never creates it in the first place. "Cancellation cleanup handling in musl has no relationship to C++ exceptions and unwinding... glibc implements cancellation as an exception". IMHO Musl made the smarter decision here. It is incredibly rare to find a codebase that uses both POSIX thread cancellation and C++ exceptions. I have never seen a codebase that uses both and expects them to be aware of each other, and I would be astonished if one existed. Glibc paid an immense cost in complexity for something nobody has ever used.

Changes Made

This PR cuts link (3): instead of building libgcc_s.so as part of gcc, we build it separately from gcc. Now there is a strict acyclic graph of buildInputs:

 gccWithoutTargetLibc
 |
 +--->glibc-nolibgcc
 |    |
 |    v
 +--->libgcc
 |    |
 |    v
 +--->glibc
 |    |
 |    v
 +--->gcc

In other words, there's a simple linear buildInputs chain glibc-nolibgcc -> libgcc -> glibc -> gcc where all four packages are compiled by (and therefore have as a (native)BuildInput) gccWithoutTargetLibc.

gccWithoutTargetLibc and glibc-nolibgcc are strictly bootstrapping artifacts; nothing else has them as a buildInput and they shouldn't appear in the closure of any final deployment packages. glibc-nolibgcc lacks libgcc_s.so, so it will segfault if you try to use it with POSIX thread cancellation. Fortunately all we need from it is (a) its headers (lib.getDev) and (b) to use it in the ./configure script for libgcc.

When translated over to the native bootstrap, xgcc takes the place of gccWithoutTargetLibc, and the "first glibc" (we build two of them) takes the place of glibc-nolibgcc. At that point our native and cross bootstrap have the same overall architecture, and it becomes possible to merge them (at last!)

[ghost]

@ofborg build tests.cross.mbuffer

[ghost]

tests.cross.mbuffer on x86_64-linux - Success

🎉

[ghost]

@ofborg eval

[ghost]

Figured out how to do this without a rebuild of native stdenv; base has been switched to master.

[Majiir]

Building pkgsCross.armv7l-hf-multiplatform.glibc fails (full build log):

       > In file included from ../../.././gcc/options.h:8,
       >                  from ../../.././gcc/tm.h:23,
       >                  from /build/gcc-12.3.0/libgcc/libgcc2.c:29:
       > /build/gcc-12.3.0/libgcc/../gcc/config/arm/arm-opts.h:29:10: fatal error: arm-isa.h: No such file or directory
       >    29 | #include "arm-isa.h"
       >       |          ^~~~~~~~~~~
       > compilation terminated.
       > make: *** [Makefile:501: _muldi3.o] Error 1

Bisect results:

• 32301623bc11 is good
• 36909cfb3d0e and 1fea17366181 fail with infinite recursion encountered
• 89e7eaacb38c fails with the arm-isa.h error above

[ghost]

Building pkgsCross.armv7l-hf-multiplatform.glibc fails (full build log):

Thanks for catching this, @Majiir.

• Fix is in 322acc7
• Regression test for armv7hf is in ac86ffc (so I don't miss this again)

[ghost]

36909cf and 1fea173 fail with infinite recursion encountered

I've squashed these two commits into the commit after them.

I originally separated them in order to try to change only one package per commit, but unfortunately that creates non-evaluable commits in the history which -- as you noticed -- is a pain when git bisecting.

[Majiir]

Build successful, and it fixes the mbuffer issue. 🎉 I tested it on my armv7l hardware (cross-compiled from x86_64) to be sure.

I'm also trying to cross-compile my whole armv7l system on this branch, but currently running into build failures on latest master (unrelated to this PR). I'll update if I get a successful build or run into any problems.

I spent some time reading through your commits. I'm not qualified to give these changes a critical review, but it generally looks good to me.

Thank you for your efforts on this and for looking after the core bits of nixpkgs!

[ghost]

I'm also trying to cross-compile my whole armv7l system on this branch

Let me know how it goes! If you run into any problems that manifest for cross but not for native let me know; there's a good chance I've seen them before. I mostly hang out in #tvl on hackint.org.

[lopsided98]

I get the arm-isa.h: No such file or directory error on armv6l and armv5tel; 322acc7 probably needs to be expanded to cover all 32-bit ARM architectures.