ld.lld: error: section: .exit.data is not contiguous with other relro sections

[nathanchance]

After the series to enable KASan for ARM was applied, ld.lld errors when linking an allyesconfig kernel:

$ make -skj"$(nproc)" ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- KCONFIG_ALLCONFIG=<(echo CONFIG_CPU_BIG_ENDIAN=n) LLVM=1 distclean allyesconfig vmlinux
...
ld.lld: error: section: .exit.data is not contiguous with other relro sections
...

$ git bisect log
# bad: [3f267ec60b922eff2a5c90d532357a39f155b730] Add linux-next specific files for 20201029
# good: [23859ae44402f4d935b9ee548135dd1e65e2cbf4] Merge tag 'trace-v5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace
git bisect start '3f267ec60b922eff2a5c90d532357a39f155b730' '23859ae44402f4d935b9ee548135dd1e65e2cbf4'
# bad: [bfa70a4ea4bfa6f87b58cf8b90b88297389c92b7] Merge remote-tracking branch 'mtd/mtd/next' into master
git bisect bad bfa70a4ea4bfa6f87b58cf8b90b88297389c92b7
# bad: [37a292dcf77532547f335ed5063d9169031c9b08] Merge remote-tracking branch 'sunxi/sunxi/for-next' into master
git bisect bad 37a292dcf77532547f335ed5063d9169031c9b08
# good: [e6d922c77db276a16f0b7933c2a9951dc9c0052c] Merge remote-tracking branch 'drm-misc-fixes/for-linux-next-fixes' into master
git bisect good e6d922c77db276a16f0b7933c2a9951dc9c0052c
# bad: [cbe49fbb8f6c8d29bc1d9a5a9a742ef2c2eb6320] Merge remote-tracking branch 'mvebu/for-next' into master
git bisect bad cbe49fbb8f6c8d29bc1d9a5a9a742ef2c2eb6320
# bad: [d0e12484e7e1ede73c538744cdbe9439f7335d01] Merge remote-tracking branch 'arm-soc/for-next' into master
git bisect bad d0e12484e7e1ede73c538744cdbe9439f7335d01
# good: [24a23387c15f34bad2485a9e1c3b7ac6f0fb35a6] Merge branch 'asm-generic-cleanup' into asm-generic
git bisect good 24a23387c15f34bad2485a9e1c3b7ac6f0fb35a6
# good: [3a8eb4d3421a2ca0f95ac3b1a8f012940d4f0d52] Merge remote-tracking branch 'kbuild/for-next' into master
git bisect good 3a8eb4d3421a2ca0f95ac3b1a8f012940d4f0d52
# bad: [20f96e606509ee5084690179afe1810b95617a92] Merge branches 'fixes' and 'misc' into for-next
git bisect bad 20f96e606509ee5084690179afe1810b95617a92
# good: [d6d51a96c7d63b7450860a3037f2d62388286a52] ARM: 9014/2: Replace string mem* functions for KASan
git bisect good d6d51a96c7d63b7450860a3037f2d62388286a52
# good: [5615f69bc2097452ecc954f5264d784e158d6801] ARM: 9016/2: Initialize the mapping of KASan shadow memory
git bisect good 5615f69bc2097452ecc954f5264d784e158d6801
# bad: [fc2933c133744305236793025b00c2f7d258b687] ARM: 9020/1: mm: use correct section size macro to describe the FDT virtual address
git bisect bad fc2933c133744305236793025b00c2f7d258b687
# bad: [421015713b306e47af95d4d61cdfbd96d462e4cb] ARM: 9017/2: Enable KASan for ARM
git bisect bad 421015713b306e47af95d4d61cdfbd96d462e4cb
# first bad commit: [421015713b306e47af95d4d61cdfbd96d462e4cb] ARM: 9017/2: Enable KASan for ARM

I have reported this upstream: https://lore.kernel.org/linux-arm-kernel/20201030002900.GA2248731@ubuntu-m3-large-x86/

cc @MaskRay

[MaskRay]

ppc has a similar problem http://lore.kernel.org/r/CAFP8O3+Mg6B4WnDzAnT43132JTbtKgkK7JFwm8Sqhc1m0CzY1g@mail.gmail.com "[PATCH] powerpc/boot: move the .got section to after the .dynamic section" @gwelymernans

RELRO sections need to be contiguous. For arm,

// arch/arm/kernel/vmlinux.lds.S
        .exit.text : {
                ARM_EXIT_KEEP(EXIT_TEXT)
        }
// arch/arm/kernel/vmlinux.lds.S
#define EXIT_DATA                                                       \
        *(.exit.data .exit.data.*)                                      \
        *(.fini_array .fini_array.*)                                    \
        *(.dtors .dtors.*)                                              \
        MEM_DISCARD(exit.data*)                                         \
        MEM_DISCARD(exit.rodata*)

If an input section .fini_array exists, in LLD the output section .exit.text will be "upgraded" to SHT_FINI_ARRAY and become a RELRO section. There may be RELRO sections (.dynamic/.data.rel.ro/...) in another place, causing non-contiguous RELRO sections.

[nickdesaulniers]

@ardbiesheuvel pointed out on the list that relro isn't used in the kernel, at least for aarch64; not sure if that applies for all other arches as well?

[ardbiesheuvel]

RELRO is essentially a program header that describes a memory range that may be remapped read-only after the relocations have been applied. We don't use program headers to begin with in Linux, and we don't use PIC code generation or PIE linking for ARM at all, so we prefer text relocations over GOT indirections

[nickdesaulniers]

https://lore.kernel.org/linux-arm-kernel/20201110015632.2509254-1-natechancellor@gmail.com/T/#u

[MaskRay]

@ardbiesheuvel There is more to that. It is probably true that the kernel does not need PT_GNU_RELRO. I thought about:

(1) whether the kernel can pick PT_GNU_RELRO as a hardening feature in the future
(2) the reported problems indicate a section ordering problem, generally it is good to have sections of the same set of SHF_WRITE/SHF_EXECINSTR (whether RELRO) together. The kernel may have problems here and there.

It is probable that the kernel does not care about the problems, though.

[ardbiesheuvel]

As I explained, we don't use ELF program headers at all. The vmlinux ELF file is converted to a binary image using objcopy, and the program headers are not even captured there, and we rely strictly on linker emitted symbols that delineate the various segments, in order to decide which part of the image should be mapped with which attributes. Currently, we even have an ASSERT() that .data.rel.ro be empty, and so we currently cannot build with -fpie, we can only link with -pie (and on the AArch64 small code model, that has worked fine so far, although I suggested multiple times now that we should perhaps introduce a special AArch64 code model for Linux)

Note that .exit.data is dead code in the core kernel: anything that is builtin cannot be unloaded, and so the code is never used. Therefore we map it in the .init segment, which gets unmapped and freed entirely after boot. So moving this section into a place where it could be covered by a PT_GNU_RELRO program header would force us to retain code that we know we will never need.

[nickdesaulniers]

although I suggested multiple times now that we should perhaps introduce a special AArch64 code model for Linux

Let's discuss more in #275. I have a list of links, but it might be worthwhile for us to chat more there. In particular, it looks like @MaskRay has already implemented at least one of the ideas there.

[nathanchance]

v2: https://lore.kernel.org/lkml/20201112183839.1009297-1-natechancellor@gmail.com/

[nickdesaulniers]

bumping @nathanchance to submit v1 to Russel's patch tracker.

[nathanchance]

Yes, sorry, have not had time to reply to your email. I can submit to Russell's tracker tomorrow or Saturday I think, I was waiting for feedback on whether just doing it in arch/arm was acceptable versus the main Makefile then I got caught up in school and work :(

[nathanchance]

Submitted to RMK's patch system: https://www.armlinux.org.uk/developer/patches/viewpatch.php?id=9038/1

[nathanchance]

Accepted: http://git.armlinux.org.uk/cgit/linux-arm.git/commit/?id=1b78828288d09bc3704144ad7779f9df13fc8808

[nickdesaulniers]

0cda9bc