Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring GFS.v16 features back to develop branch #274

Merged
merged 25 commits into from
Nov 17, 2020
Merged

Bring GFS.v16 features back to develop branch #274

Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8b309e2
bring GFS.v16 code changes back to develop branch
junwang-noaa Oct 15, 2020
36e928c
update upp lib in hera module
junwang-noaa Oct 27, 2020
5b4a845
use test hpc stack lib
junwang-noaa Oct 29, 2020
796bec2
update upp lib in fv3 for cmake
junwang-noaa Oct 29, 2020
32a4352
udpate fv3 to the latest develop branch
junwang-noaa Oct 29, 2020
ef6b823
Merge remote-tracking branch 'upstream/develop' into gfsv16_20201015
junwang-noaa Oct 29, 2020
eb1460b
update post control files
junwang-noaa Oct 30, 2020
22b4299
update upp lib in hpcstack on hera, orion and dell and in nceplibs on…
junwang-noaa Nov 12, 2020
8a586fc
add change in fv3 dycore #35
junwang-noaa Nov 12, 2020
5113ed8
update FV3 to the develop branch
junwang-noaa Nov 12, 2020
09d25b6
Merge remote-tracking branch 'upstream/develop' into gfsv16_20201015
junwang-noaa Nov 12, 2020
0c1f2b5
remove duplicate read_data in fms_mod in external_ic.F90
junwang-noaa Nov 13, 2020
401a357
udpate library for hera.gnu
junwang-noaa Nov 13, 2020
c9e18df
update FV3 to the top of develop branch; skip-ci
junwang-noaa Nov 13, 2020
f8fd3b9
Merge remote-tracking branch 'upstream/develop' into gfsv16_20201015
junwang-noaa Nov 13, 2020
c2cede8
update hera.gnu hpc stack; skip-ci
junwang-noaa Nov 13, 2020
e81b7da
Update modulefiles for cheyenne.intel and cheyenne.gnu to use hpc-stack
climbfuji Oct 23, 2020
cf8e648
update dell/cray log files
junwang-noaa Nov 16, 2020
b7a1f1e
Merge branch 'gfsv16_20201015' of https://github.com/junwang-noaa/ufs…
climbfuji Nov 16, 2020
6903894
Update modulefiles for Cheyenne to work with feature/hpc-stack-nco br…
climbfuji Nov 16, 2020
def0296
Regression test logs for cheyenne.intel and cheyenne.gnu
climbfuji Nov 16, 2020
8cc59ca
Regression test logs for hera.intel and hera.gnu
climbfuji Nov 16, 2020
f3544dc
Merge pull request #5 from climbfuji/gfsv16_20201015
junwang-noaa Nov 17, 2020
8118efe
update orion log files
junwang-noaa Nov 17, 2020
1553c0c
point fv3 to EMC develop branch
junwang-noaa Nov 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[submodule "FV3"]
path = FV3
url = https://github.com/NOAA-EMC/fv3atm
branch = develop
url = https://github.com/junwang-noaa/fv3atm
branch = gfsv16
[submodule "NEMS"]
path = NEMS
url = https://github.com/NOAA-EMC/NEMS
Expand Down
2 changes: 1 addition & 1 deletion CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ find_package(sp REQUIRED)
find_package(w3emc REQUIRED)
find_package(w3nco REQUIRED)
if(INLINE_POST)
find_package(nceppost REQUIRED)
find_package(upp REQUIRED)
endif()

# See https://github.com/NOAA-EMC/NCEPLIBS-nemsio/pull/22
Expand Down
2 changes: 1 addition & 1 deletion FV3
Open in desktop
Submodule FV3 updated 5 files
+2 −2 .gitmodules
+1 −1 CMakeLists.txt
+1 −1 atmos_cubed_sphere
+1 −1 io/CMakeLists.txt
+2 −2 io/makefile
77 changes: 31 additions & 46 deletions modulefiles/cheyenne.gnu/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,60 +1,45 @@
#%Module######################################################################
##
## NEMS FV3 Prerequisites: Cheyenne/GNU
#%Module

proc ModulesHelp {} {
puts stderr "\tcit - loads modules required for building and running FV3 under NEMS on Cheyenne/GNU"
puts stderr "\tcit - loads modules required for building and running UFS Model on Cheyenne/GNU"
}

module-whatis "loads NEMS FV3 prerequisites for Cheyenne/GNU"
module-whatis "loads UFS Model prerequisites for Cheyenne/GNU"

# NOTE: the "module purge" and loading of the module command are
# handled by the module-setup.sh (or .csh) script.
module load cmake/3.16.4
setenv CMAKE_C_COMPILER mpicc
setenv CMAKE_CXX_COMPILER mpicxx
setenv CMAKE_Fortran_COMPILER mpif90
setenv CMAKE_Platform cheyenne.gnu

##
## load programming environment
## this typically includes compiler, MPI and job scheduler
##
# load programming environment
module load ncarenv/1.3
module load gnu/9.1.0
module load mpt/2.19
module load mpt/2.22
module load ncarcompilers/0.5.0
module unload netcdf

module use /glade/p/ral/jntp/GMTB/tools/hpc-stack-nco-20201113/modulefiles/stack
module load hpc/1.0.0-beta1
module load hpc-gnu/9.1.0
module load hpc-mpt/2.22

##
## use pre-compiled PNG, NetCDF, ESMF and NCEP libraries for above compiler / MPI combination
##
module use -a /glade/p/ral/jntp/GMTB/tools/ufs-stack-20200909/gnu-9.1.0/mpt-2.19/modules
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

module load libpng/1.6.35
module load hdf5/1.10.6
module load netcdf/4.7.4
module load esmf/8.1.0bs27
module load pio/2.5.1
module load esmf/8_1_0_beta_snapshot_27

module load bacio/2.4.0
module load bacio/2.4.1
module load crtm/2.3.0
module load g2/3.4.0
module load g2tmpl/1.9.0
module load ip/3.3.0
module load nceppost/dceca26
module load nemsio/2.5.1
module load sp/2.3.0
module load w3emc/2.7.0
module load w3nco/2.4.0

module load gfsio/1.4.0
module load sfcio/1.4.0
module load sigio/2.3.0

##
## SIONlib library
##
module use -a /glade/p/ral/jntp/GMTB/tools/modulefiles/gnu-9.1.0/mpt-2.19
module load SIONlib/1.7.4

##
## load cmake
##
module load cmake/3.16.4
setenv CMAKE_C_COMPILER mpicc
setenv CMAKE_CXX_COMPILER mpicxx
setenv CMAKE_Fortran_COMPILER mpif90
setenv CMAKE_Platform cheyenne.gnu
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
77 changes: 31 additions & 46 deletions modulefiles/cheyenne.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,60 +1,45 @@
#%Module######################################################################
##
## NEMS FV3 Prerequisites: Cheyenne/Intel
#%Module

proc ModulesHelp {} {
puts stderr "\tcit - loads modules required for building and running FV3 under NEMS on Cheyenne/Intel"
puts stderr "\tcit - loads modules required for building and running UFS Model on Cheyenne/Intel"
}

module-whatis "loads NEMS FV3 prerequisites for Cheyenne/Intel"
module-whatis "loads UFS Model prerequisites for Cheyenne/Intel"

# NOTE: the "module purge" and loading of the module command are
# handled by the module-setup.sh (or .csh) script.
module load cmake/3.16.4
setenv CMAKE_C_COMPILER mpicc
setenv CMAKE_CXX_COMPILER mpicxx
setenv CMAKE_Fortran_COMPILER mpif90
setenv CMAKE_Platform cheyenne.intel

##
## load programming environment
## this typically includes compiler, MPI and job scheduler
##
# load programming environment
module load ncarenv/1.3
module load intel/19.1.1
module load mpt/2.19
module load mpt/2.22
module load ncarcompilers/0.5.0
module unload netcdf

module use /glade/p/ral/jntp/GMTB/tools/hpc-stack-nco-20201113/modulefiles/stack
module load hpc/1.0.0-beta1
module load hpc-intel/19.1.1
module load hpc-mpt/2.22

##
## use pre-compiled PNG, NetCDF, ESMF and NCEP libraries for above compiler / MPI combination
##
module use -a /glade/p/ral/jntp/GMTB/tools/ufs-stack-20200909/intel-19.1.1/mpt-2.19/modules
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

module load libpng/1.6.35
module load hdf5/1.10.6
module load netcdf/4.7.4
module load esmf/8.1.0bs27
module load pio/2.5.1
module load esmf/8_1_0_beta_snapshot_27

module load bacio/2.4.0
module load bacio/2.4.1
module load crtm/2.3.0
module load g2/3.4.0
module load g2tmpl/1.9.0
module load ip/3.3.0
module load nceppost/dceca26
module load nemsio/2.5.1
module load sp/2.3.0
module load w3emc/2.7.0
module load w3nco/2.4.0

module load gfsio/1.4.0
module load sfcio/1.4.0
module load sigio/2.3.0

##
## SIONlib library
##
module use -a /glade/p/ral/jntp/GMTB/tools/modulefiles/intel-19.1.1/mpt-2.19
module load SIONlib/1.7.4

##
## load cmake
##
module load cmake/3.16.4
setenv CMAKE_C_COMPILER mpicc
setenv CMAKE_CXX_COMPILER mpicxx
setenv CMAKE_Fortran_COMPILER mpif90
setenv CMAKE_Platform cheyenne.intel
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
10 changes: 5 additions & 5 deletions modulefiles/hera.gnu/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ setenv CMAKE_CXX_COMPILER mpicxx
setenv CMAKE_Fortran_COMPILER mpif90
setenv CMAKE_Platform hera.gnu

module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack

module use /scratch2/NCEPDEV/nwprod/hpc-stack/test/modulefiles/stack
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and elsewhere, please use a tagged version from hpc-stack before merging this in develop

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks to me that this is an issue with NCEPLIBS group, not the model. If there is an requirement that hpc-stack needs to have version number, then nceplibs group needs to know it and follow that. We need to get the GFS.v16 merged to develop branch so people can start further development. This is the only hpc stack lib provided to us. We can have a meeting with NCEPLIBS group asking them to accept the rule about version requirement, when they come out a new hpc-stack version, we will update it in the model. But so far we have tested this hpc stack library in the model and it is working and we can not afford further delay.

module load hpc/1.0.0-beta1
module load hpc-gnu/9.2.0
module load hpc-mpich/3.3.2
module load hpc-mpich/3.3.2
module load mpich/3.3.2

module load jasper/2.0.15
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

Expand All @@ -35,8 +35,8 @@ module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
6 changes: 3 additions & 3 deletions modulefiles/hera.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform hera.intel

module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack
module use /scratch2/NCEPDEV/nwprod/hpc-stack/test/modulefiles/stack
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the same naming convention for hpc-stack installation (lib/hpc-stack/vX.Y.Z). hpc-stack repo must have a tag pointing to that exact version number.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a tag.


module load hpc/1.0.0-beta1
module load hpc-intel/18.0.5.274
module load hpc-impi/2018.0.4

module load jasper/2.0.15
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

Expand All @@ -35,8 +35,8 @@ module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
6 changes: 3 additions & 3 deletions modulefiles/hera.intel/fv3_debug
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform hera.intel

module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack
module use /scratch2/NCEPDEV/nwprod/hpc-stack/test/modulefiles/stack
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a tag.


module load hpc/1.0.0-beta1
module load hpc-intel/18.0.5.274
module load hpc-impi/2018.0.4

module load jasper/2.0.15
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

Expand All @@ -35,8 +35,8 @@ module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
6 changes: 3 additions & 3 deletions modulefiles/orion.intel/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform orion.intel

module use /apps/contrib/NCEP/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack
module use /apps/contrib/NCEP/test/hpc-stack-nco/modulefiles/stack
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a tag from develop


module load hpc/1.0.0-beta1
module load hpc-intel/2018.4
module load hpc-impi/2018.4

module load jasper/2.0.15
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

Expand All @@ -34,8 +34,8 @@ module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
6 changes: 3 additions & 3 deletions modulefiles/orion.intel/fv3_debug
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform orion.intel

module use /apps/contrib/NCEP/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack
module use /apps/contrib/NCEP/test/hpc-stack-nco/modulefiles/stack

module load hpc/1.0.0-beta1
module load hpc-intel/2018.4
module load hpc-impi/2018.4

module load jasper/2.0.15
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

Expand All @@ -34,8 +34,8 @@ module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
3 changes: 2 additions & 1 deletion modulefiles/wcoss_cray/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ module load crtm/2.3.0
module load g2/3.4.0
module load g2tmpl/1.9.0
module load ip/3.3.0
module load nceppost/dceca26
module load nemsio/2.5.1
module load sp/2.3.0
module load w3emc/2.7.0
Expand All @@ -54,6 +53,8 @@ module load gfsio/1.4.0
module load sfcio/1.4.0
module load sigio/2.3.0

module use /usrx/local/nceplibs/NCEPLIBS/cmake/install/NCEPLIBS_UFS/modules
module load upp/10.0.0

## WCOSS Cray execution prereqs:
module load rca
Expand Down
6 changes: 3 additions & 3 deletions modulefiles/wcoss_dell_p3/fv3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ setenv CMAKE_CXX_COMPILER mpiicpc
setenv CMAKE_Fortran_COMPILER mpiifort
setenv CMAKE_Platform wcoss_dell_p3

module use /usrx/local/nceplibs/dev/hpc-stack/libs/hpc-stack/v1.0.0-beta1/modulefiles/stack
module use /usrx/local/nceplibs/dev/hpc-stack/test/hpc-stack/modulefiles/stack
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NCEPLIBS team decided on using tags and versions with x.y.z and not using test or branches.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul would you please open an issue in NCEPLIBS about this requirement?


module load hpc/1.0.0-beta1
module load hpc-ips/18.0.1.163
module load hpc-impi/18.0.1

module load jasper/2.0.15
module load jasper/2.0.22
module load zlib/1.2.11
module load png/1.6.35

Expand All @@ -38,8 +38,8 @@ module load crtm/2.3.0
module load g2/3.4.1
module load g2tmpl/1.9.1
module load ip/3.3.3
module load nceppost/dceca26
module load nemsio/2.5.2
module load sp/2.3.3
module load w3emc/2.7.3
module load w3nco/2.4.1
module load upp/10.0.0
Loading