Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port global-workflow to S4 #138

Closed
87 of 91 tasks
DavidHuber-NOAA opened this issue Sep 23, 2020 · 11 comments · Fixed by #1023
Closed
87 of 91 tasks

Port global-workflow to S4 #138

DavidHuber-NOAA opened this issue Sep 23, 2020 · 11 comments · Fixed by #1023
Assignees
@DavidHuber-NOAA
Labels
port Porting the system to a new platform

Comments

@DavidHuber-NOAA
Copy link
Contributor

DavidHuber-NOAA commented Sep 23, 2020 •
edited
Loading

The global-workflow should be ported to S4.

Branch: DavidHuber-NOAA:port2s4

Below is a checklist taken from #5 for porting to Orion which I will use as a template:

  • Test: connection to GitHub
  • Build: hpc-stack (personal area)
  • Build: hpc-stack (official area)
  • Install: manage_externals manic-v1.1.8 tag (personal area)
  • Build: FV3GFS (w/WW3)
  • Build: ProdGSI
  • Build: GLDAS
  • Build: UFS_UTILS
  • Build: gfs_wafs
  • Build: obsproc
  • Build: chgres_recenter
  • Build: chgres_recenter_ncio
  • Build: fbwndgfs
  • Build: fv3nc2nemsio
  • Build: gaussian_sfcanl
  • Build: gfs_bufr
  • Build: regrid_nemsio
  • Build: supvit
  • Build: syndat_getjtbul
  • Build: syndat_maksynrc
  • Build: syndat_qctropcy
  • Build: tave
  • Build: tocsbufr
  • Build: vint
  • Test: fv3gfs_chgres.sh
  • Establish: "glopara" space
  • Establish: FIX_DIR under "glopara"
  • Establish: BASE_GIT under "glopara"
  • Establish: GDA under "glopara"
  • Establish: obsproc_prep and obsproc_global under "glopara"
  • Establish: Fit2Obs under "glopara"
  • Establish: METplus
  • Establish: ens_tracker under "glopara"
  • Create: standard S4 environment
  • Add: system paths to config.base.emc.dyn
  • Add: system to machine-setup.sh
  • Add: system to modulefiles/module_setup.sh[csh].inc
  • Add: system to setup_expt.py
  • Add: system to setup_expt_fcstonly.py
  • Add: system to setup_workflow.py
  • Add: system to setup_workflow_fcstonly.py
  • Add: system to workflow_utils.py
  • Review: S4 module files for final modules and hpc-stack
  • Test: run setup_expt.py
  • Test: run setup_expt_fcstonly.py
  • Test: run setup_workflow.py
  • Test: run setup_workflow_fcstonly.py
  • Test: crons
  • Test: ROCOTO
  • Test: gdas[gfs]fcst job
  • Test: gdas[gfs]post job
  • Test: gdasvrfy job
  • Test: gfsvrfy job
  • Test: METplus jobs
  • Test: gdas[gfs]arch job
  • Test: complete free-forecast
  • Test: gdas[gfs]prep job
  • Test: gdaseobs job
  • Test: gdas[gfs]anal job
  • Test: gdas[gfs]analcalc job
  • Test: gdasanaldiag job
  • Test: gdasgldas job
  • Test: gdasediag job
  • Test: gdaseupd job
  • Test: gdasecen job
  • Test: gdasesfc job
  • Test: gdasefcs job
  • Test: gdasepos job
  • Test: gdas[gfs]waveinit job
  • Test: gdas[gfs]waveprep job
  • Test: gdas[gfs]wavepostsbs job
  • Test: GSI monitoring jobs
  • Test: Fit2Obs
  • Test: ens_tracker
  • Test: C96C48 cycled
  • Test: C192C96 cycled
  • Test: C384C192 cycled
  • Test: C96 ff
  • Test: C192 ff
  • Test: C384 ff
  • Test: beta-testers
  • Pre-commit testing: cycled and ff on Hera
  • Pre-commit testing: cycled and ff on WCOSS-Cray
  • Pre-commit testing: cycled and ff on WCOSS-Dell
  • Perform review
  • Supported in checkout: ufs-weather-model/fv3atm
  • Supported in checkout: ProdGSI
  • Supported in checkout: GLDAS
  • Supported in checkout: EMC_post
  • Supported in checkout: UFS_UTILS
  • Supported in checkout: EMC_verif-global
@DavidHuber-NOAA DavidHuber-NOAA mentioned this issue Sep 23, 2020
Closed
@KateFriedman-NOAA KateFriedman-NOAA added the port Porting the system to a new platform label Oct 15, 2020
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Oct 26, 2020
@DavidHuber-NOAA
NOAA-EMC#138 Modifications to build on S4.
389c468
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Feb 1, 2021
@DavidHuber-NOAA
Created an env file for S4.
8bfdb21 
References: NOAA-EMC#138
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Feb 1, 2021
@DavidHuber-NOAA
Updated S4 module files.
0362365 
References: NOAA-EMC#138
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Feb 1, 2021
@DavidHuber-NOAA
Added S4-specific variables to three config files.
3d324de 
References: NOAA-EMC#138
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Feb 1, 2021
@DavidHuber-NOAA
Added S4 to the postsnd script; created run script for S4.
2312e31 
Also disabled the Gempak commands on S4.

References: NOAA-EMC#138
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Feb 1, 2021
@DavidHuber-NOAA
Modified default directories for S4.
9c7c49f 
References: NOAA-EMC#138
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Feb 1, 2021
@DavidHuber-NOAA
Add a ulimit command for S4's default crontab template.
eeaf242 
References: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 2, 2021
@DavidHuber-NOAA
Added S4 to module-setup.csh.inc.
2f7580b 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 2, 2021
@DavidHuber-NOAA
Established an environment file for S4.
05e6dfc 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 2, 2021
@DavidHuber-NOAA
Updated S4 module files.
7d78341 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 2, 2021
@DavidHuber-NOAA
Added S4-specific variables to three config files.
7b3748a 
The configuration files are for vrfy, prepbufr, and fcst jobs.

Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 2, 2021
@DavidHuber-NOAA
Added a postsnd script and driver for S4.
92a0e14 
Note that Gempak commands are disabled on S4.

Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 2, 2021
@DavidHuber-NOAA
Modified default directories for S4 in the setup_expt*.py scripts.
29b0c99 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 2, 2021
@DavidHuber-NOAA
Add a ulimit command for S4's default crontab template.
56bf340 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 9, 2021
@DavidHuber-NOAA
Updated fv3gfs to v16.0.15.
f4073b0 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 9, 2021
@DavidHuber-NOAA
Updated branch checkouts for the S4 port.
7d79d6a 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 9, 2021
@DavidHuber-NOAA
Removed an unused variable from config.vrfy for S4
110097d 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Feb 9, 2021
@DavidHuber-NOAA
Added initial support for the ivy partition on S4.
d5e6644 
Refs NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Apr 14, 2021
@DavidHuber-NOAA
Additional fixes to workflow scripts for S4
9472683 
This sets the default partition (s4) and specifies that the firstcyc job
should be run on the serial queue (QUEUE_SERVICE).

Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Apr 14, 2021
@DavidHuber-NOAA
Added a missing ":"
74a4409 
Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Jun 22, 2021
@DavidHuber-NOAA
Amended the module-setup.csh.inc script.
4a7d99d 
Recent changes on S4 now require /etc/profile to be sourced.  Also found
that the script was sourcing the wrong lmod location.

Refs: NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Jun 22, 2021
@DavidHuber-NOAA
Changed the default C384 GFS tiling to match GDAS.
ebf922b 
This seems to only matter if running with DO_WAVE="NO".

Refs:NOAA-EMC#138
DavidHuber-NOAA referenced this issue in DavidHuber-NOAA/global-workflow Jun 22, 2021
@DavidHuber-NOAA
Adjusted S4 resources to optimize cycle timing
78b8fd5 
Refs: NOAA-EMC#138
@DavidHuber-NOAA DavidHuber-NOAA mentioned this issue Jul 29, 2021
Closed
3 tasks
@DavidHuber-NOAA
Copy link
Contributor Author

DavidHuber-NOAA commented Aug 4, 2021 •
edited
Loading

The verif-global port has been completed. The tag including this port is verif_global_v2.2.1.

@DavidHuber-NOAA DavidHuber-NOAA mentioned this issue Aug 6, 2021
Closed
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Remove S4 reference to S2S+. NOAA-EMC#138
5c62602
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Added S2S note for S4 in the README. NOAA-EMC#138
9755264
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Apply lint fixes. NOAA-EMC#138
8898a65
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Added nsstbufr, acftbufr fix to ENKF scripts. NOAA-EMC#138
84abbed
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Apply lint fixes. NOAA-EMC#138
6da1f7d
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Address linter warnings. NOAA-EMC#138
fbc7171
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Fixed shellcheck directive. Address linter errors. NOAA-EMC#138
e49ecf1
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Fixed linter warnings. NOAA-EMC#138
a23b38c
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 19, 2022
@DavidHuber-NOAA
Fixed shellcheck error. NOAA-EMC#138
e991e25
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 21, 2022
@DavidHuber-NOAA
Removed S4 resolution restrictions. NOAA-EMC#138
5118de8
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 21, 2022
@DavidHuber-NOAA
Addressed linter warnings in S4.env. NOAA-EMC#138
53ae609
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 21, 2022
@DavidHuber-NOAA
Addressed linter warnings in S4.env. NOAA-EMC#138
bb3f90d
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 23, 2022
@DavidHuber-NOAA
Shortened DO_MAKE to MAKE for prepbufr files. NOAA-EMC#138
cee1d31
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 23, 2022
@DavidHuber-NOAA
Cleaned up S4.env and converted it to bash. NOAA-EMC#138
95622a5
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 23, 2022
@DavidHuber-NOAA
Renamed DO_MAKEPREPBUFR to MAKE_PREPBUFR. NOAA-EMC#138
7abf40d
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 23, 2022
@DavidHuber-NOAA
Address shellcheck warning in prep.sh. NOAA-EMC#138
c5bd8c2
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 27, 2022
@DavidHuber-NOAA
Remove S4-specific optimizations. NOAA-EMC#138
380cf39
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 27, 2022
@DavidHuber-NOAA
Removed S4-specific wtime request for gdaseobs. NOAA-EMC#138
5dfddef
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Sep 27, 2022
@DavidHuber-NOAA
Updated S4 host file with supported resolutions. NOAA-EMC#138
c75c161
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Oct 3, 2022
@DavidHuber-NOAA
Updates S4 env and host files. NOAA-EMC#138
5f018ad
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Oct 3, 2022
@DavidHuber-NOAA
Updates S4 hosts file. NOAA-EMC#138
a753d4a
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Oct 3, 2022
@DavidHuber-NOAA
Address lint warnings. NOAA-EMC#138
10513fd
@DavidHuber-NOAA DavidHuber-NOAA mentioned this issue Oct 4, 2022
Closed
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Oct 4, 2022
@DavidHuber-NOAA
Added S4 to link_workflow.sh. NOAA-EMC#138
b175caa
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Oct 5, 2022
@DavidHuber-NOAA
Updated gfs_utils hash to include S4 port. NOAA-EMC#138
081e172
@DavidHuber-NOAA DavidHuber-NOAA mentioned this issue Oct 6, 2022
Merged
9 tasks
DavidHuber-NOAA added a commit to DavidHuber-NOAA/global-workflow that referenced this issue Oct 7, 2022
@DavidHuber-NOAA
Updated to include the new prepobs package. NOAA-EMC#138
bd9fc79
WalterKolczynski-NOAA pushed a commit that referenced this issue Oct 8, 2022
@DavidHuber-NOAA
Port to S4 (#1023)
e09989b 
Ports the global workflow to the S4 cluster.  Note that this does not include support for S2S experiments.  Additionally, S4 does not support C768 experiments.

A couple of special notes:

- S4 does not have access to rstprod data.  Among other things, this means that the nsstbufr and prepbufr.acft_profiles files must be created on the fly.  The way I accomplished this was by moving the `MAKE_NSSTBUFR` to and creating `MAKE_ACFTBUFR` in config.base.emc.dyn and setting them via setup_workflow.xml.  This seems like a bit of a kludge and I am open to suggestions on how else to address this.  Both options need to be set for the prep and analysis jobs.
- S4 _can_ run S2S+ experiments, but this requires significant, and convoluted, modifications to the configuration files.  Support for these are thus not enabled by default.  Instead, I have placed a set of configuration files in S4:/data/users/dhuber/save/s2s_configs. Users interested in performing these experiments should contact me to set it up.

Closes #138
kayeekayee pushed a commit to kayeekayee/global-workflow that referenced this issue May 30, 2024
@climbfuji
Cleanup work: CCPP metadata units and more (NOAA-EMC#140)
f053db2 
This PR contains cleanup work to address wrong units and some issues raised in recently merged PRs:
- correct units for latitude, longitude, and pi
- correct standard name air_temperature_save_from_cumulus_paramterization to air_temperature_save_from_convective_parameterization
- remove unused variables in two routines in `GFS_typedefs.F90`
- remove recently introduced variable `cycling` (part of GFS_control DDT), now a local variable in `physics/module_MYNNPBL_wrapper.F90`
- change order of interstitial schemes in `ccpp/suites/suite_FV3_GFS_2017_fv3wam.xml` to match other SDFs

Fixes NOAA-EMC#138 and parts of NOAA-EMC#137.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DavidHuber-NOAA DavidHuber-NOAA

Labels
port Porting the system to a new platform
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Port to S4 DavidHuber-NOAA/global-workflow
5 participants
@JessicaMeixner-NOAA @WalterKolczynski-NOAA @KateFriedman-NOAA @arunchawla-NOAA @DavidHuber-NOAA