Skip to content

Commit 6c5b9bb

Browse files
DavidHuber-NOAADavidHuber-NOAA
authored
Update HPC Tier Definitions (NOAA-EMC#3138)
Updates the tier definitions for supported platforms. This also updates and reformats the README. ~Note that the README badges will display errors until this PR is merged.~ @TerrenceMcGuinness-NOAA updated the gist, so the badges are now showing up correctly in the README. Resolves NOAA-EMC#3135
1 parent 6585798 commit 6c5b9bb

File tree

6 files changed

+296
-25
lines changed

6 files changed

+296
-25
lines changed

‎.github/workflows/orion.yaml ‎.github/workflows/hercules.yaml

+10-10
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Orion
1+
name: Hercules
22

33
on:
44
pull_request_target:
@@ -26,7 +26,7 @@ jobs:
2626
echo "labels=$LABELS" >> $GITHUB_OUTPUT
2727
2828
passed:
29-
if: contains( needs.getlabels.outputs.labels, 'CI-Orion-Passed') && github.event.pull_request.merged
29+
if: contains( needs.getlabels.outputs.labels, 'CI-Hercules-Passed') && github.event.pull_request.merged
3030
runs-on: ubuntu-22.04
3131
needs:
3232
- getlabels
@@ -38,13 +38,13 @@ jobs:
3838
forceUpdate: true
3939
auth: ${{ secrets.CLI_DYNAMIC_BADGES }}
4040
gistID: e35aa2904a54deae6bbb1fdc2d960c71
41-
filename: orion.json
42-
label: orion
41+
filename: hercules.json
42+
label: hercules
4343
message: passing
4444
color: green
4545

4646
failed:
47-
if: contains( needs.getlabels.outputs.labels, 'CI-Orion-Failed') && github.event.pull_request.merged
47+
if: contains( needs.getlabels.outputs.labels, 'CI-Hercules-Failed') && github.event.pull_request.merged
4848
runs-on: ubuntu-latest
4949
needs:
5050
- getlabels
@@ -56,14 +56,14 @@ jobs:
5656
forceUpdate: true
5757
auth: ${{ secrets.CLI_DYNAMIC_BADGES }}
5858
gistID: e35aa2904a54deae6bbb1fdc2d960c71
59-
filename: orion.json
60-
label: orion
59+
filename: hercules.json
60+
label: hercules
6161
message: failing
6262
color: red
6363

6464

6565
pending:
66-
if: "!contains( needs.getlabels.outputs.labels, 'CI-Orion-Passed') && !contains( needs.getlabels.outputs.labels, 'CI-Orion-Failed')"
66+
if: "!contains( needs.getlabels.outputs.labels, 'CI-Hercules-Passed') && !contains( needs.getlabels.outputs.labels, 'CI-Hercules-Failed')"
6767
runs-on: ubuntu-latest
6868
needs:
6969
- getlabels
@@ -75,7 +75,7 @@ jobs:
7575
forceUpdate: true
7676
auth: ${{ secrets.CLI_DYNAMIC_BADGES }}
7777
gistID: e35aa2904a54deae6bbb1fdc2d960c71
78-
filename: orion.json
79-
label: orion
78+
filename: hercules.json
79+
label: hercules
8080
message: pending
8181
color: orange

‎.github/workflows/wcoss2.yaml

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: WCOSS2
2+
3+
on:
4+
pull_request_target:
5+
branches:
6+
- develop
7+
types: [closed]
8+
9+
jobs:
10+
11+
getlabels:
12+
runs-on: ubuntu-22.04
13+
outputs:
14+
labels: ${{ steps.id.outputs.labels }}
15+
steps:
16+
- name: Get Label Steps
17+
id: id
18+
env:
19+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
20+
OWNER: ${{ github.repository_owner }}
21+
REPO_NAME: ${{ github.event.repository.name }}
22+
PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }}
23+
run: |
24+
LABELS1="$(gh api repos/$OWNER/$REPO_NAME/pulls/$PULL_REQUEST_NUMBER --jq '.labels.[].name')"
25+
LABELS=$(echo "$LABELS1" | tr '\n' ' ')
26+
echo "labels=$LABELS" >> $GITHUB_OUTPUT
27+
28+
passed:
29+
if: contains( needs.getlabels.outputs.labels, 'CI-Wcoss2-Passed') && github.event.pull_request.merged
30+
runs-on: ubuntu-latest
31+
needs:
32+
- getlabels
33+
34+
steps:
35+
- name: Passed
36+
uses: schneegans/dynamic-badges-action@v1.6.0
37+
with:
38+
forceUpdate: true
39+
auth: ${{ secrets.CLI_DYNAMIC_BADGES }}
40+
gistID: e35aa2904a54deae6bbb1fdc2d960c71
41+
filename: wcoss2.json
42+
label: wcoss2
43+
message: passing
44+
color: green
45+
46+
failed:
47+
if: contains( needs.getlabels.outputs.labels, 'CI-Wcoss2-Failed') && github.event.pull_request.merged
48+
runs-on: ubuntu-latest
49+
needs:
50+
- getlabels
51+
52+
steps:
53+
- name: Failed
54+
uses: schneegans/dynamic-badges-action@v1.6.0
55+
with:
56+
forceUpdate: true
57+
auth: ${{ secrets.CLI_DYNAMIC_BADGES }}
58+
gistID: e35aa2904a54deae6bbb1fdc2d960c71
59+
filename: wcoss2.json
60+
label: wcoss2
61+
message: failing
62+
color: red
63+
64+
65+
pending:
66+
if: "!contains( needs.getlabels.outputs.labels, 'CI-Wcoss2-Passed') && !contains( needs.getlabels.outputs.labels, 'CI-Wcoss2-Failed')"
67+
runs-on: ubuntu-latest
68+
needs:
69+
- getlabels
70+
71+
steps:
72+
- name: Pending
73+
uses: schneegans/dynamic-badges-action@v1.6.0
74+
with:
75+
forceUpdate: true
76+
auth: ${{ secrets.CLI_DYNAMIC_BADGES }}
77+
gistID: e35aa2904a54deae6bbb1fdc2d960c71
78+
filename: wcoss2.json
79+
label: wcoss2
80+
message: pending
81+
color: orange

‎README.md

+29-15
Original file line numberDiff line numberDiff line change
@@ -3,28 +3,42 @@
33
[![pynorms](https://github.com/NOAA-EMC/global-workflow/actions/workflows/pynorms.yaml/badge.svg)](https://github.com/NOAA-EMC/global-workflow/actions/workflows/pynorms.yaml)
44

55
![Custom badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/emcbot/e35aa2904a54deae6bbb1fdc2d960c71/raw/hera.json)
6-
![Custom badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/emcbot/e35aa2904a54deae6bbb1fdc2d960c71/raw/orion.json)
6+
![Custom badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/emcbot/e35aa2904a54deae6bbb1fdc2d960c71/raw/hercules.json)
7+
![Custom badge](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/emcbot/e35aa2904a54deae6bbb1fdc2d960c71/raw/wcoss2.json)
78

8-
# global-workflow
9-
Global Workflow currently supporting the Global Forecast System (GFS) with the [UFS-weather-model](https://github.com/ufs-community/ufs-weather-model) and [GSI](https://github.com/NOAA-EMC/GSI)-based Data Assimilation System.
9+
The Global Workflow supporting the Global Forecast System (GFS), the Global Ensemble Forecasting System (GEFS), and the Seasonal Forecast System (SFS) with the [UFS-weather-model](https://github.com/ufs-community/ufs-weather-model). Data assimilation, currently only available for the GFS, is provides by both the [GSI](https://github.com/NOAA-EMC/GSI)- and [GDASApp (JEDI)](https://github.com/NOAA-EMC/GDASApp)-based Data Assimilation systems.
1010

11-
The `global-workflow` depends on the following prerequisities to be available on the system:
11+
In progress [documentation](https://global-workflow.readthedocs.io/en/latest/) is available.
1212

13-
* Workflow Engine - [Rocoto](https://github.com/christopherwharrop/rocoto) and [ecFlow](https://github.com/ecmwf/ecflow) (for NWS Operations)
14-
* Compiler - Intel Compiler Suite
15-
* Software - NCEPLIBS (various), ESMF, HDF5, NetCDF, and a host of other softwares (see module files under /modulefiles for additional details)
13+
# Prerequisites
1614

17-
The `global-workflow` current supports the following tier-1 machines:
15+
The Global Workflow depends on the following prerequisities to be available on the system:
1816

19-
* NOAA RDHPCS - Hera
20-
* MSU HPC - Orion
21-
* MSU HPC - Hercules
22-
* NOAA's operational HPC - WCOSS2
17+
* Workflow Engines - [Rocoto](https://github.com/christopherwharrop/rocoto) and [ecFlow](https://github.com/ecmwf/ecflow) (for NWS Operations)
18+
* Compiler - Intel classic compiler suite version 2021.x
19+
* Software - NCEPLIBS (various), ESMF, HDF5, NetCDF, and a host of other softwares (see module files under /modulefiles for additional details).
20+
- [Spack-stack](https://github.com/JCSDA/spack-stack) is recommended for the installation of dependent libraries.
2321

24-
Additionally, the following tier-2 machine is supported:
25-
* SSEC at Univ. of Wisconsin - S4 (Note that S2S+ experiments are not fully supported)
22+
## Supported platforms
2623

27-
Documentation (in progress) is available [here](https://global-workflow.readthedocs.io/en/latest/).
24+
The Global Workflow currently supports the following machines at the indicated tier.
25+
26+
| HPC | Tier | Notes |
27+
| --------------------------------------- |:----:|:--------------------------------------------------------------------------:|
28+
| WCOSS2<br>NCO | 1 | GEFS testing is not regularly performed. |
29+
| Hera<br>NOAA RDHPCS | 1 | |
30+
| Hercules<br>MSU | 1 | Currently does not support the TC Tracker. |
31+
| Orion<br>MSU | 2 | The GSI runs very slowly on Orion and the TC tracker is not supported. |
32+
| Gaea C5/C6<br>RDHPCS | 3 | Currently non-operational following an OS upgrade.<br>Supported by EPIC. |
33+
| AWS, GCP, Azure <br>NOAA Parallel Works | 3 | Supported by EPIC. |
34+
| Jet<br>RDHPCS | 3 | Supported by NESDIS. |
35+
| S4<br>SSEC | 3 | Currently non-operational following an OS upgrade.<br>Supported by NESDIS. |
36+
37+
<ins>**Tier Definitions**</ins>
38+
39+
1. Fully supported by the EMC global workflow team. CI testing is regularly performed on these systems, the majority of the global workflow features are supported, and the team will address any platform-specific features, bugs, upgrades, and requests for data.
40+
2. Supported by the global workflow team on an ad-hoc basis. CI tests are supported on these systems, but not regularly performed.
41+
3. No official support by the global workflow team, but may be supported by other entities (e.g. EPIC).
2842

2943
# Disclaimer
3044

‎docs/source/components.rst

+1
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ The major components of the system are:
1212
* Forecast
1313
* Post-processing
1414
* Verification
15+
* Archiving
1516

1617
The Global Workflow repository contains the workflow and script layers. External components will be checked out as git submodules. All of the submodules of the system reside in their respective repositories on GitHub.
1718

‎docs/source/hpc.rst

+174
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,180 @@ The following system software requirements are the minimum for any new or existi
4444
| | | WCOSS2 |
4545
+--------------+-------------+---------------------------------------+
4646

47+
===========================
48+
Feature availability by HPC
49+
===========================
50+
51+
The Global Workflow provides capabilities for deterministic and ensemble forecasts along with data assimilation on multiple platforms. However, not all features are currently supported on all platforms. The following table lists the features by platform and states their level of support.
52+
53+
.. list-table:: Capabilities matrix by HPC
54+
:header-rows: 1
55+
:align: center
56+
57+
* - HPC
58+
- Tier
59+
- Coupled
60+
GFS
61+
- Coupled
62+
GEFS
63+
- GSI
64+
DA
65+
- GDASApp
66+
DA
67+
- Coupled
68+
DA
69+
- TC Tracker
70+
- AWIPS
71+
- MOS
72+
- Fit2Obs
73+
- METplus
74+
Verification
75+
- HPSS
76+
Archiving
77+
* - WCOSS2
78+
- 1
79+
- X
80+
- X
81+
- X
82+
- X
83+
-
84+
- X
85+
- X
86+
- X
87+
- X
88+
- X
89+
- X
90+
* - Hera
91+
- 1
92+
- X
93+
- X
94+
- X
95+
- X
96+
- X
97+
- X
98+
- X
99+
-
100+
- X
101+
- X
102+
- X
103+
* - Hercules
104+
- 1
105+
- X
106+
- X
107+
- X
108+
- X
109+
- X
110+
-
111+
-
112+
-
113+
- X
114+
- X
115+
-
116+
* - Orion
117+
- 2
118+
- X
119+
- X
120+
- X
121+
- X
122+
-
123+
- X
124+
-
125+
-
126+
- X
127+
- X
128+
-
129+
* - Gaea C5
130+
- 3
131+
- X
132+
- X
133+
- X
134+
- X
135+
-
136+
-
137+
-
138+
-
139+
-
140+
-
141+
- X
142+
* - Gaea C6
143+
- 3
144+
- X
145+
- X
146+
- X
147+
- X
148+
-
149+
-
150+
-
151+
-
152+
-
153+
-
154+
- X
155+
* - AWS (PW)
156+
- 3
157+
- X
158+
- X
159+
- X
160+
-
161+
-
162+
-
163+
-
164+
-
165+
-
166+
-
167+
-
168+
* - GCP (PW)
169+
- 3
170+
- X
171+
- X
172+
-
173+
-
174+
-
175+
-
176+
-
177+
-
178+
-
179+
-
180+
-
181+
* - Azure
182+
- 3
183+
- X
184+
- X
185+
-
186+
-
187+
-
188+
-
189+
-
190+
-
191+
-
192+
-
193+
-
194+
* - Jet
195+
- 3
196+
- X
197+
-
198+
- X
199+
-
200+
-
201+
- X
202+
-
203+
-
204+
- X
205+
- X
206+
- X
207+
* - S4
208+
- 3
209+
-
210+
-
211+
- X
212+
-
213+
-
214+
- X
215+
-
216+
-
217+
- X
218+
- X
219+
-
220+
47221
================================
48222
Experiment troubleshooting help
49223
================================

‎docs/source/index.rst

+1
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ Code managers
1818

1919
* Kate Friedman - @KateFriedman-NOAA / kate.friedman@noaa.gov
2020
* Walter Kolczynski - @WalterKolczynski-NOAA / walter.kolczynski@noaa.gov
21+
* David Huber - @DavidHuber-NOAA / david.huber@noaa.gov
2122

2223
=============
2324
Announcements

0 commit comments

Comments
 (0)