Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add linux-riscv64 nativeaot runtime build #110688

Merged
merged 24 commits into from
Jan 9, 2025

Conversation

am11
Copy link
Member

@am11 am11 commented Dec 13, 2024

This change enables NativeAOT build support on Linux-RISCV64, leveraging the LA64 port https://github.com/dotnet/runtime/pulls?q=sort%3Aupdated-desc+is%3Apr+label%3Aarch-loongarch64+label%3Aarea-NativeAOT-coreclr+is%3Aclosed. While the build setup is functional, it has not yet been tested. This submission primarily aims to share the code changes required to achieve a successful build. Comprehensive testing and validation are contingent on resolving community-supported platform build processes, which are currently blocked on #105004.

Huge props to @sunlijun-610 for helping out with this. 🎉

Contributes to #106223, #84834

@am11 am11 requested review from filipnavara and a team December 13, 2024 11:46
@dotnet-policy-service dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Dec 13, 2024
Copy link
Contributor

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

@am11 am11 added the arch-riscv Related to the RISC-V architecture label Dec 13, 2024
@clamp03
Copy link
Member

clamp03 commented Dec 16, 2024

@tomeksowi @sirntar Please review and test this PR in SRPOL? Thank you.

@am11
Copy link
Member Author

am11 commented Dec 16, 2024

This is bulk of change which just builds. _NativeAotSupportedArch condition:

<_NativeAotSupportedArch Condition="'$(TargetArchitecture)' == 'x64' or '$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'arm' or '$(TargetArchitecture)' == 'loongarch64' or ('$(TargetOS)' == 'windows' and '$(TargetArchitecture)' == 'x86')">true</_NativeAotSupportedArch>

is not changed because that breaks the build in similar way as #108973 (requires LKG apphost). I think after this bulk of change is merged (need to make sure it doesn't break anyone's workflow), we can make smaller fixes once we figure out a way to test nativeaot runtime on community platforms.

@risc-vv
Copy link

risc-vv commented Dec 16, 2024

c01dc7b is being scheduled for building and testing

GIT: c01dc7bbae7789bbf992d827ce9fe662aeb17c79
REPO: dotnet/runtime
BRANCH: main

Release-build FAILED

buildinfo.json

Cloning into '/go-agent/pipelines/Release-build/runtime'...
Updating files:  14% (8701/59681)
Updating files:  15% (8953/59681)
Updating files:  16% (9549/59681)
Updating files:  17% (10146/59681)
Updating files:  18% (10743/59681)
Updating files:  19% (11340/59681)
Updating files:  20% (11937/59681)
Updating files:  21% (12534/59681)
Updating files:  22% (13130/59681)
Updating files:  23% (13727/59681)
Updating files:  24% (14324/59681)
Updating files:  25% (14921/59681)
Updating files:  26% (15518/59681)
Updating files:  27% (16114/59681)
Updating files:  28% (16711/59681)
Updating files:  29% (17308/59681)
Updating files:  30% (17905/59681)
Updating files:  31% (18502/59681)
Updating files:  32% (19098/59681)
Updating files:  32% (19410/59681)
Updating files:  33% (19695/59681)
Updating files:  34% (20292/59681)
Updating files:  35% (20889/59681)
Updating files:  36% (21486/59681)
Updating files:  37% (22082/59681)
Updating files:  38% (22679/59681)
Updating files:  39% (23276/59681)
Updating files:  40% (23873/59681)
Updating files:  41% (24470/59681)
Updating files:  42% (25067/59681)
Updating files:  43% (25663/59681)
Updating files:  44% (26260/59681)
Updating files:  45% (26857/59681)
Updating files:  46% (27454/59681)
Updating files:  47% (28051/59681)
Updating files:  48% (28647/59681)
Updating files:  49% (29244/59681)
Updating files:  50% (29841/59681)
Updating files:  51% (30438/59681)
Updating files:  51% (30671/59681)
Updating files:  52% (31035/59681)
Updating files:  53% (31631/59681)
Updating files:  54% (32228/59681)
Updating files:  55% (32825/59681)
Updating files:  56% (33422/59681)
Updating files:  57% (34019/59681)
Updating files:  58% (34615/59681)
Updating files:  59% (35212/59681)
Updating files:  60% (35809/59681)
Updating files:  61% (36406/59681)
Updating files:  62% (37003/59681)
Updating files:  63% (37600/59681)
Updating files:  64% (38196/59681)
Updating files:  65% (38793/59681)
Updating files:  66% (39390/59681)
Updating files:  67% (39987/59681)
Updating files:  68% (40584/59681)
Updating files:  69% (41180/59681)
Updating files:  70% (41777/59681)
Updating files:  71% (42374/59681)
Updating files:  72% (42971/59681)
Updating files:  73% (43568/59681)
Updating files:  74% (44164/59681)
Updating files:  75% (44761/59681)
Updating files:  76% (45358/59681)
Updating files:  77% (45955/59681)
Updating files:  77% (46219/59681)
Updating files:  78% (46552/59681)
Updating files:  79% (47148/59681)
Updating files:  80% (47745/59681)
Updating files:  81% (48342/59681)
Updating files:  82% (48939/59681)
Updating files:  83% (49536/59681)
Updating files:  84% (50133/59681)
Updating files:  85% (50729/59681)
Updating files:  86% (51326/59681)
Updating files:  87% (51923/59681)
Updating files:  88% (52520/59681)
Updating files:  89% (53117/59681)
Updating files:  90% (53713/59681)
Updating files:  91% (54310/59681)
Updating files:  91% (54552/59681)
Updating files:  92% (54907/59681)
Updating files:  93% (55504/59681)
Updating files:  94% (56101/59681)
Updating files:  95% (56697/59681)
Updating files:  96% (57294/59681)
Updating files:  97% (57891/59681)
Updating files:  98% (58488/59681)
Updating files:  99% (59085/59681)
Updating files: 100% (59681/59681)
Updating files: 100% (59681/59681), done.

@risc-vv
Copy link

risc-vv commented Dec 16, 2024

RISC-V Release-CLR-QEMU: 9440 / 9461 (99.78%)
=======================
      passed: 9440
      failed: 4
     skipped: 107
      killed: 17
------------------------
  TOTAL libs: 9568
 TOTAL tests: 9568
   REAL time: 57min 21s 277ms
=======================

Release-CLR-QEMU.md, Release-CLR-QEMU.xml, testclr_output.tar.gz

RISC-V Release-FX-QEMU: 603280 / 643384 (93.77%)
=======================
      passed: 603280
      failed: 321
     skipped: 1443
      killed: 39783
------------------------
  TOTAL libs: 256
 TOTAL tests: 644827
   REAL time: 2h 22min 41s 187ms
=======================

Release-FX-QEMU.md, Release-FX-QEMU.xml, testfx_output.tar.gz

Build information and commands

GIT: c01dc7bbae7789bbf992d827ce9fe662aeb17c79
CI: 64a5c905f307e2e98dbf9ad1d31e22d0eea08eed
REPO: dotnet/runtime
BRANCH: main
CONFIG: Release
LIB_CONFIG: Release

# CORE_LIBS_BUILD_CMD
runtime/build.sh --arch riscv64 --cross -c Release -s libs /p:EnableSourceLink=false
# CORE_BUILD_CMD
runtime/build.sh --arch riscv64 --cross -c Release -s clr+libs+host /p:EnableSourceLink=false

# TESTCLR_BUILD_CMD
runtime/src/tests/build.sh -riscv64 -cross -Release -priority1 -p:UsePublishedCrossgen2=false -p:UseLocalAppHostPack=true
# TESTCLR_CMD
python3 riscv-CI/goci/agent/TestRunner/run.py --core_root ./coreclr.Release/Tests/Core_Root --testhost ./testhost.Release --atest ./coreclr.Release --test ./ --log_dir ./logs  --timeout 2700 --log_level DEBUG --xunit xunit.Release
# TESTCLR_RUN
/godata/pipelines/Release-CLR-QEMU/logs/run_tests.log
cp -R /godata/pipelines/Release-CLR-QEMU/xunit.Release "/_PATH_/_WITH_/_TEST_"/ ; cd "/_PATH_/_WITH_/_TEST_" && ROOTFS_DIR=/crossrootfs/riscv64 QEMU_LD_PREFIX=/crossrootfs/riscv64 __TestDotNetCmd=/godata/pipelines/Release-CLR-QEMU/testhost.Release/dotnet CORE_ROOT=/godata/pipelines/Release-CLR-QEMU/coreclr.Release/Tests/Core_Root  /usr/bin/time -f "exec_time: %e" ./_TEST_BINARY_

# TESTFX_BUILD_CMD
runtime/build.sh --arch riscv64 --cross -c Release -rc Release -hc Release -lc Release -s libs.tests --testscope innerloop /p:EnableSourceLink=false /p:UseLocalAppHostPack=true
# TESTFX_CMD
unknown command
# TESTFX_RUN
unknown command

# TEST_ENV
DOTNET_JitStress=;DOTNET_JitStressRegs=;DOTNET_GCStress=;DOTNET_JITMinOpts=;DOTNET_TailcallStress=;DOTNET_TieredCompilation=
RISC-V Release-CLR-VF2: 9440 / 9461 (99.78%)
=======================
      passed: 9440
      failed: 4
     skipped: 107
      killed: 17
------------------------
  TOTAL libs: 9568
 TOTAL tests: 9568
   REAL time: 2h 58min 45s 591ms
=======================

Release-CLR-VF2.md, Release-CLR-VF2.xml, testclr_output.tar.gz

RISC-V Release-FX-VF2: 572716 / 609386 (93.98%)
=======================
      passed: 572716
      failed: 84
     skipped: 1452
      killed: 36586
------------------------
  TOTAL libs: 256
 TOTAL tests: 610838
   REAL time: 2h 40min 43s 79ms
=======================

Release-FX-VF2.md, Release-FX-VF2.xml, testfx_output.tar.gz

Build information and commands

GIT: c01dc7bbae7789bbf992d827ce9fe662aeb17c79
CI: 64a5c905f307e2e98dbf9ad1d31e22d0eea08eed
REPO: dotnet/runtime
BRANCH: main
CONFIG: Release
LIB_CONFIG: Release

# CORE_LIBS_BUILD_CMD
runtime/build.sh --arch riscv64 --cross -c Release -s libs /p:EnableSourceLink=false
# CORE_BUILD_CMD
runtime/build.sh --arch riscv64 --cross -c Release -s clr+libs+host /p:EnableSourceLink=false

# TESTCLR_BUILD_CMD
runtime/src/tests/build.sh -riscv64 -cross -Release -priority1 -p:UsePublishedCrossgen2=false -p:UseLocalAppHostPack=true
# TESTCLR_CMD
python3 riscv-CI/goci/agent/TestRunner/run.py --core_root ./coreclr.Release/Tests/Core_Root --testhost ./testhost.Release --atest ./coreclr.Release --test ./ --log_dir ./logs  --timeout 2700 --log_level DEBUG --xunit xunit.Release
# TESTCLR_RUN
/var/lib/go-agent/pipelines/Release-CLR-VF2/logs/run_tests.log
cp -R /var/lib/go-agent/pipelines/Release-CLR-VF2/xunit.Release "/_PATH_/_WITH_/_TEST_"/ ; cd "/_PATH_/_WITH_/_TEST_" && __TestDotNetCmd=/var/lib/go-agent/pipelines/Release-CLR-VF2/testhost.Release/dotnet CORE_ROOT=/var/lib/go-agent/pipelines/Release-CLR-VF2/coreclr.Release/Tests/Core_Root  /usr/bin/time -f "exec_time: %e" ./_TEST_BINARY_

# TESTFX_BUILD_CMD
runtime/build.sh --arch riscv64 --cross -c Release -rc Release -hc Release -lc Release -s libs.tests --testscope innerloop /p:EnableSourceLink=false /p:UseLocalAppHostPack=true
# TESTFX_CMD
unknown command
# TESTFX_RUN
unknown command

# TEST_ENV
DOTNET_JitStress=;DOTNET_JitStressRegs=;DOTNET_GCStress=;DOTNET_JITMinOpts=;DOTNET_TailcallStress=;DOTNET_TieredCompilation=

Copy link
Contributor

@tomeksowi tomeksowi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments. Thanks for this bring-up!

Co-authored-by: Tomek Sowiński <tomeksowi@gmail.com>
Copy link
Contributor

@tomeksowi tomeksowi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a build unblocking PR it looks ok, we can fix more if/when we actually start using NativeAOT.

{
uint32_t instr = *pInstr;

if (((instr & STW_PAIR_MASK) == STW_PAIR_BITS) &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be something like TrailingEpilogueInstructionsCount but detecting stores instead of loads.

Co-authored-by: Tomasz Sowiński <tomeksowi@gmail.com>
@am11
Copy link
Member Author

am11 commented Dec 20, 2024

@jkotas please review this. Trying to avoid more merge conflicts emerging from new PRs.

@am11 am11 force-pushed the feature/nativeaot/riscv64-port branch 2 times, most recently from b6d6635 to 529e2a8 Compare January 9, 2025 01:36
#define PLAT_ASM_OFFSET(offset, cls, member) \
static_assert((offsetof(cls, member) == 0x##offset) || (offsetof(cls, member) > 0x##offset), "Bad asm offset for '" #cls "." #member "', the actual offset is smaller than 0x" #offset "."); \
static_assert((offsetof(cls, member) == 0x##offset) || (offsetof(cls, member) < 0x##offset), "Bad asm offset for '" #cls "." #member "', the actual offset is larger than 0x" #offset ".");
static_assert(offsetof(cls, member) == 0x##offset, "Bad asm offset for '" #cls "." #member "'. Actual offset: " OFFSET_STRING(cls, member));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me, this is producing message like:

static_assert failed: 'Bad asm offset for 'PAL_LIMITED_CONTEXT.Rbp'. Actual offset: offsetof(PAL_LIMITED_CONTEXT, Rbp)'

This is less informative message than the existing one that tries to tell you whether the actual offset is less or more. Why is it an improvement?

Also, if it is an improvement, should the other macros in this file get the same treatment?

Copy link
Member Author

@am11 am11 Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. I just changed the value of ExInfo and ran build.sh -a loongarch64 in the container I was already running, it shows the actual computed value:

  [ 33%] Built target unwinder_wks
  In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:46:
  In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsets.h:84:
  /runtime/src/coreclr/nativeaot/Runtime/loongarch64/AsmOffsetsCpu.h:11:1: error: static assertion failed due to requirement '__builtin_offsetof(ExInfo, m_pPrevExInfo) == 1': Bad asm offset for 'ExInfo.m_pPrevExInfo'. Actual offset: offsetof(ExInfo, m_pPrevExInfo)
     11 | PLAT_ASM_OFFSET(1, ExInfo, m_pPrevExInfo)
        | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:36:19: note: expanded from macro 'PLAT_ASM_OFFSET'
     36 |     static_assert(offsetof(cls, member) == 0x##offset, "Bad asm offset for '" #cls "." #member "'. Actual offset: " OFFSET_STRING(cls, member));
        |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/lib/llvm-19/lib/clang/19/include/__stddef_offsetof.h:16:24: note: expanded from macro 'offsetof'
     16 | #define offsetof(t, d) __builtin_offsetof(t, d)
        |                        ^
  /runtime/src/coreclr/nativeaot/Runtime/loongarch64/AsmOffsetsCpu.h:11:1: note: expression evaluates to '0 == 1'
     11 | PLAT_ASM_OFFSET(1, ExInfo, m_pPrevExInfo)
        | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:36:41: note: expanded from macro 'PLAT_ASM_OFFSET'
     36 |     static_assert(offsetof(cls, member) == 0x##offset, "Bad asm offset for '" #cls "." #member "'. Actual offset: " OFFSET_STRING(cls, member));
        |                   ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~

It helped me figuring out a few issues during the port. I can update other macros as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not show the computed value with Windows compilers:

  C:\runtime\src\coreclr\nativeaot\Runtime\amd64\AsmOffsetsCpu.h(19): error C2338: static_assert failed: 'Bad asmoffset for 'ExInfo.m_notifyDebuggerSP'. Actual offset: offsetof(ExInfo, m_notifyDebuggerSP)'

I am fine with changing it. All macros in the file should be changed the same way.

am11 and others added 2 commits January 9, 2025 06:27
…r/ElfObjectWriter.cs

Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>
@jkotas
Copy link
Member

jkotas commented Jan 9, 2025

/ba-g known infrastructure issue

Copy link
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@jkotas jkotas merged commit 289aa17 into dotnet:main Jan 9, 2025
137 of 149 checks passed
@am11 am11 deleted the feature/nativeaot/riscv64-port branch January 9, 2025 23:05
@@ -151,7 +151,7 @@
# else
# define RISCV_FLEN 0
# endif
# define _LIBUNWIND_CONTEXT_SIZE (32 * (__riscv_xlen + RISCV_FLEN) / 64)
# define _LIBUNWIND_CONTEXT_SIZE (32 * (__riscv_xlen + RISCV_FLEN) / 64) + 32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, llvm-libunwind is basically not upstreamed. That one commit listed there is a collection of all the patches we applied since 2015. Tracking issue #72655. I will update llvm-libunwind soon to v19 or 20, so I will recreate that composite patch.

@github-actions github-actions bot locked and limited conversation to collaborators Feb 10, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-riscv Related to the RISC-V architecture area-NativeAOT-coreclr community-contribution Indicates that the PR has been added by a community member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants