cmd/link, cmd/cgo: internal linking fails if CGO_CFLAGS enables LTO #43505
Comments
|
So what's going on here is that these cflags make gcc include symbols in object files that cmd/link/internal/loadelf doesn't like, causing it to abandon parsing the file and so not find symbols it needs to find. loadelf could be taught to ignore these symbols, I guess, or just skip all symbols it doesn't know how to handle, but really with my Ubuntu/Debian golang maintainer hat on, I'd rather there was some way of preventing the linker ever attempt internal linking if any cgo is used at all, at least during package builds. I don't want to disable internal linking entirely -- that would mean all go executables, even those not using cgo at all, would be dynamically linked, which seems a bit much of a change. I guess I can do this by patching internalpkg in cmd/link/internal/ld/lib.go to be empty but that also seems a bit much. I don't think there is an existing flag or environment variable that does this. |
|
We want to use internal linking for Go programs that do not explicitly use cgo. That includes Go programs that use the standard packages net and os/user, even though those packages do use cgo by default. So we do want internal linking to work for programs that use cgo, provided the only packages that use cgo are those in the standard library. But we could consider using external linking by default if |
toothrot
added
the
NeedsInvestigation
|
That would work. Can the linker assume that CGO_CFLAGS in its environment is the one that was used to build the object files it is processing though? The cflags are recorded in the _cgo_flags but that's only used by the gccgo support in cmd/go currently, I'm not sure the linker can read them from anywhere? |
|
It should be the case that if the linker sees But what about flags specified with |
It's certainly good enough for my use case!
I wondered about that, but any use of cgo outside the standard library triggers external linking anyway so this isn't an issue (I think we can be sure no-one will accidentally add "#cgo CFLAGS: -flto -ffat-lto-objects" to runtime/cgo ...) |
|
Good point. |
|
Change https://golang.org/cl/281314 mentions this issue: |
|
I think only accepting -g and -O is too narrow. On some platforms there are some flags we use. For example, -march=armv6 is used when we build ARMv6 release. On Darwin we pass some flag to specify the macOS version. Listing all allowed flags seems complicated. Compared to this, maybe -linkmode=external-if-cgo is more desirable. |
|
Change https://golang.org/cl/322614 mentions this issue: |
The linker now accepts unrecognized object files in external linking mode. These objects will simply be passed to the external linker. This permits using -flto which can generate pure byte code objects, whose symbol table the linker does not know how to read. The cgo tool now passes -fno-lto when generating objects whose symbols it needs to read. The cgo tool now emits matching types in different objects, so that the lto linker does not report a mismatch. This is based on https://golang.org/cl/293290 by Derek Parker. For #43505 Fixes #43830 Fixes #46295 Change-Id: I6787de213417466784ddef5af8899e453b4ae1ad Reviewed-on: https://go-review.googlesource.com/c/go/+/322614 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
seankhliao
changed the title
|
@ianlancetaylor @cherrymui looks like this was merged in for 1.17 but isn't assigned to the milestone. Can I also request a 1.16 backport? |
|
Milestones don't indicate when an issue is fixed, except for minor version milestones. Also, this issue isn't closed. Is there anything else we need to do here? I don't think there is, but maybe I'm missing something. I'm not comfortable with a 1.16 backport of https://golang.org/cl/322614. Changes in cgo have a long history of causing problems and being reverted and changed. It's not the kind of thing we want to in a minor release cycle. This also doesn't seem to qualify by our normal backport guidelines at https://golang.org/wiki/MinorReleases. It's not a regression in behavior from earlier releases. Fortunately 1.17 will be out soon. |
|
Change https://golang.org/cl/339370 mentions this issue: |
|
I just upgraded to 1.17 and am still seeing this during a build when LTO is enabled, so it does not seem to have been fixed in this release. |
|
Yes that's why this bug is still open :) I just pinged my simpler CL to fix this -- it seems it didn't assigned reviewers for some reason. |
|
Sorry to necrobump this but I was wondering if this fix will be incorporated in go |
bump 6.3 branch to 6.3.5 bump 6.2 branch to 6.2.5 bump 5.37 branch to 5.37.9 Also strip LTO from CGO_CFLAGS per golang/go#43505
gopherbot
added
the
compiler/runtime
It looks like that golang applications currently fails[1] to build with lto enabled on CGO_CFLAGS/CGO_CXXFLAGS. Disabling lto until there is a fix from upstream golang. References ---------- [1] golang/go#43505
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes, although the error is a bit different.
What operating system and processor architecture are you using (
go env)?Ubuntu 20.04 or 20.10 (current devel series).
go envOutputWhat did you do?
I ran a trivial os/user-using program with CFLAGS that caused gcc to create object files ready for link time optimization:
mwhudson@anduril:~/tmp$ cat user.go package main import ( "fmt" "os/user" ) func main() { u, _ := user.Current() fmt.Printf("username: %s\n", u.Name) } mwhudson@anduril:~/tmp$ CGO_CFLAGS='-g -flto -ffat-lto-objects' ~/src/go/bin/go run user.goWhat did you expect to see?
What did you see instead?
Errors of the form "loadelf: /home/mwhudson/.cache/go-build/1c/1cfde03a90c420f612c840a344f1de135898b62010e420c7b14c3e353c13c3d9-d(_x001.o): 67693: sym#26: ignoring symbol in section 5 (type 0)" and then a panic:
The text was updated successfully, but these errors were encountered: