Upgrades to spack v0.20.1 #792

dominichofer · 2023-07-18T13:44:30Z

Modifications for v0.20.1

Packages

All packages

change ssh links from ssh://git@github.com to git@github.com:

ICON

adapt custom build extensions to new version of spack
Pop PYTHONHOME only if set while testing
Change delimiter of variant fcgroup from ; to .
Limit out-of-source builds to spec with dev_path

COSMO

use environments instead of devbuildcosmo and installcosmo
remove all srun-related variants because of Spack spec fails with "complex" flags string even if quoted spack/spack#31214
set manual_download = True to prevent wrong usage of recipe

COSMO-Dycore

add missing include through patch
change test_dycore.py through patch

py-gt4py

limit version of py-pybind to range, see [gt4py,icon4py] Compiled Python modules not compatible with actual Python in spec #803

py-icon4py

adapt custome build extensions to new version of spack

Infrastructure

rearange software in upstream for Daint
remove devbuildcosmo and installcosmo
spack clean -a before any test run

Todo before merge

Adapt upstream paths for Daint

github-actions · 2023-07-18T13:45:15Z

PR Preview Action v1.4.4
Preview removed because the pull request was closed.
2023-09-14 14:56 UTC

AnnikaLau · 2023-07-19T07:17:33Z

@AnnikaLau, I gave this an other try. Maybe if we tackle this together we will succeed.
On Balfrin cosmo-dycore ran into this: spack/spack#31214

Ah great! I also did another fix yesterday (before I saw your new pull request) to fix the configure phase for icon on Daint. If you agree with my change (60962be), we can add it here and I can close my pull request?

dominichofer · 2023-07-19T07:42:36Z

@AnnikaLau Yes, sounds good to me.

AnnikaLau · 2023-07-20T09:29:15Z

When running the system test from Jenkins to build ICON, the following error appears for daint_cpu_cce, daint_cpu_gcc, daint_cpu_nvhpc, daint_cpu_nvhpc_out_of_source, daint_gpu_nvhpc at the step spack install -j 4 --until build -n -v (https://jenkins-mch.cscs.ch/job/Spack/job/spack_PR/1150/):

==> Error: develop does not satisfy 2. You could consider setting concretizer:unify to when_possible or false to allow multiple versions of some packages.

However, when running the test on my login nodes, I don't get this error.

jenkins-apn · 2023-09-14T07:12:28Z

balfrin

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

🟢 system test

	Test
🟢	Claw-install_default

jenkins-apn · 2023-09-14T07:17:45Z

daint

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

🔴 system test

	Test
🔴	Claw-install_default

WARNING: Serial tests did not run for system tests

jenkins-apn · 2023-09-14T07:20:30Z

tsa

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

🟢 system test

	Test
🟢	Claw-install_default

jonasjucker · 2023-09-14T10:59:07Z

launch jenkins claw

jenkins-apn · 2023-09-14T11:03:13Z

balfrin

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

jenkins-apn · 2023-09-14T11:03:55Z

tsa

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

jenkins-apn · 2023-09-14T12:08:33Z

daint

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

🟢 system test

	Test
🟢	Claw-install_default

jonasjucker · 2023-09-14T12:55:34Z

launch jenkins claw

jenkins-apn · 2023-09-14T13:04:13Z

balfrin

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

🟢 system test

	Test
🟢	Claw-install_default

jenkins-apn · 2023-09-14T13:05:40Z

tsa

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

🟢 system test

	Test
🟢	Claw-install_default

jenkins-apn · 2023-09-14T13:41:48Z

daint

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	claw-spack_info
🟢	claw-spack_spec

🟢 system test

	Test
🟢	Claw-install_default_build_only

dominichofer · 2023-09-14T13:53:17Z

launch jenkins icon

jonasjucker · 2023-09-14T13:54:36Z

launch jenkins icon balfrin

jenkins-apn · 2023-09-14T13:58:25Z

tsa

🟢 unit test

	Test
🟢	summary

jenkins-apn · 2023-09-14T14:02:10Z

daint

🟢 unit test

	Test
🟢	summary

jenkins-apn · 2023-09-14T14:17:03Z

balfrin

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	icon-spack_info
🟢	icon-spack_spec
🟢	dace_icon.-O1-spack_spec
🟢	icon_serialization=create_claw=std-spack_spec

🟢 system test

	Test
🟢	Icon-install_2_6_6_gcc
🟢	Icon-install_2_6_6_nvhpc
🟢	Icon-install_nwp_gpu

Co-authored-by: Jonas Jucker <jonas.jucker@c2sm.ethz.ch>

jonasjucker · 2023-09-14T14:41:21Z

launch jenkins onnx-runtime

jenkins-apn · 2023-09-14T14:47:00Z

tsa

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	onnx-runtime-spack_info
🟢	onnx-runtime-spack_spec

🟢 system test

	Test
🟢	OnnxRuntime-install_default

jenkins-apn · 2023-09-14T14:47:30Z

balfrin

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	onnx-runtime-spack_info
🟢	onnx-runtime-spack_spec

🟢 system test

	Test
🟢	OnnxRuntime-install_default

jenkins-apn · 2023-09-14T14:49:49Z

daint

🟢 unit test

	Test
🟢	summary

🟢 integration test

	Test
🟢	onnx-runtime-spack_info
🟢	onnx-runtime-spack_spec

🟢 system test

	Test
🟢	OnnxRuntime-install_default

AnnikaLau and others added 6 commits June 19, 2023 12:06

Remove cosmo-dycore duplicate

66230c7

Remove variant ensurepip

538aa18

Remove duplication of cosmo-dycore

e232633

Updates submodule spack to v0.20.1

e8d2908

[balfrin][tasna]Removes variant ensurepip from python.

6a6bb5a

Adds ruamel.yaml dependency

d426445

Dominic Hofer added 3 commits July 18, 2023 15:53

Merge remote-tracking branch 'origin/main' into dev0.20.1

b5ad454

Bootstrap with gnuconfig.

7989775

Removes ssh workaround.

39d1abd

C2SM deleted a comment from jenkins-apn Jul 18, 2023

Dominic Hofer added 6 commits July 18, 2023 17:39

Addendum

91106a8

Removes broken tests.

1677354

Removes test=root from eccodes.

1c7dd04

Removes ssh workaround.

569c6bc

Removes libfyaml and libxml2

7c7eab2

Replaces ruamel with pyyaml.

77fea73

[icon] Call AutotoolsPackage function with super.

d231798

dominichofer mentioned this pull request Jul 20, 2023

Update FDB Flexpart package.py #790

Closed

AnnikaLau and others added 6 commits July 21, 2023 11:58

Run spack find - TO BE REVERTED

2937fe5

GitHub Action: Apply Pep8-formatting

c349b10

remove upstream to check if ghost-variant still appear

a946047

make cython mandatory

8ecd9de

make infero tests work again

96ef8d1

GitHub Action: Apply Pep8-formatting

8e2e1c4

Update system_test.py

5f27e96

GitHub Action: Apply Pep8-formatting

2a3cf7c

Update system_test.py

35d5abf

fix typo in cmake version

40d253f

jonasjucker and others added 2 commits September 14, 2023 16:37

set new upstream (#830)

a184e08

Co-authored-by: Jonas Jucker <jonas.jucker@c2sm.ethz.ch>

Update unit_test.py

f08a009

jonasjucker merged commit 8fc904d into main Sep 14, 2023

jonasjucker deleted the dev0.20.1 branch September 14, 2023 14:55

jonasjucker restored the dev0.20.1 branch September 14, 2023 14:55

dominichofer deleted the dev0.20.1 branch September 15, 2023 14:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrades to spack v0.20.1 #792

Upgrades to spack v0.20.1 #792

dominichofer commented Jul 18, 2023 •

edited by jonasjucker

Loading

github-actions bot commented Jul 18, 2023 •

edited

Loading

AnnikaLau commented Jul 19, 2023

dominichofer commented Jul 19, 2023

AnnikaLau commented Jul 20, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

dominichofer commented Sep 14, 2023

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

Upgrades to spack v0.20.1 #792

Upgrades to spack v0.20.1 #792

Conversation

dominichofer commented Jul 18, 2023 • edited by jonasjucker Loading

Modifications for v0.20.1

Packages

All packages

ICON

COSMO

COSMO-Dycore

py-gt4py

py-icon4py

Infrastructure

Todo before merge

github-actions bot commented Jul 18, 2023 • edited Loading

AnnikaLau commented Jul 19, 2023

dominichofer commented Jul 19, 2023

AnnikaLau commented Jul 20, 2023

jenkins-apn commented Sep 14, 2023

balfrin

jenkins-apn commented Sep 14, 2023

daint

jenkins-apn commented Sep 14, 2023

tsa

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

balfrin

jenkins-apn commented Sep 14, 2023

tsa

jenkins-apn commented Sep 14, 2023

daint

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

balfrin

jenkins-apn commented Sep 14, 2023

tsa

jenkins-apn commented Sep 14, 2023

daint

dominichofer commented Sep 14, 2023

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

tsa

jenkins-apn commented Sep 14, 2023

daint

jenkins-apn commented Sep 14, 2023

balfrin

jonasjucker commented Sep 14, 2023

jenkins-apn commented Sep 14, 2023

tsa

jenkins-apn commented Sep 14, 2023

balfrin

jenkins-apn commented Sep 14, 2023

daint

dominichofer commented Jul 18, 2023 •

edited by jonasjucker

Loading

github-actions bot commented Jul 18, 2023 •

edited

Loading