Update to use project configs from NCEPLIBS #118

aerorahul · 2020-06-17T02:58:37Z

NCEPLIBS is updated to provide a project-config file with its cmake build.
This PR uses imported targets from these NCEPLIBS builds across all of the UFS_UTILS applications.

The branch feature/cmakepkgs was built with a fresh clone and build of all develop branches of NCEPLIBS-* using the umbrella build in NCEPLIBS PR #76

Change-Id: I8e7526ef1b796c3653f06d83d65ccc1d4df2483e

Change-Id: I20a151f91202f78f4b125b6d3b99c8950c265ce7

Change-Id: I332a52f9a1ae79bfa5f05e8565b487f2672f5c7d

GeorgeGayno-NOAA

I will test your branch on all officially supported machines.

DusanJovic-NOAA · 2020-06-17T12:20:38Z

Compiled it successfully on my Ubuntu 20.04 (gcc 9.3.0) laptop.

CMakeLists.txt

sorc/emcsfc_ice_blend.fd/CMakeLists.txt

sorc/emcsfc_snow2mdl.fd/CMakeLists.txt

sorc/nst_tf_chg.fd/CMakeLists.txt

GeorgeGayno-NOAA · 2020-06-17T14:30:24Z

I will test your branch on all officially supported machines.

I can't compile it on Hera. It can find "Findgfsio.cmake". There are other package files missing as well.

kgerheiser · 2020-06-17T14:34:23Z

It can't find Findgfsio.cmake? It shouldn't be looking for those, it should be looking for gfsio-config.cmake

GeorgeGayno-NOAA · 2020-06-17T14:36:59Z

It can't find Findgfsio.cmake? It shouldn't be looking for those, it should be looking for -config.cmake

Here is the entire message:

By not providing "Findgfsio.cmake" in CMAKE_MODULE_PATH this project has
 asked CMake to find a package configuration file provided by "gfsio", but
 CMake did not find one.

 Could not find a package configuration file provided by "gfsio" with any of
 the following names:

   gfsioConfig.cmake
   gfsio-config.cmake

kgerheiser · 2020-06-17T14:39:06Z

Oh, well, that's better. Does gfsio-config.cmake exist on your CMAKE_PREFIX_PATH or <package>_ROOT? It would be under lib/cmake/<package>/<package>-config.cmake

aerorahul · 2020-06-17T14:40:28Z

It can't find Findgfsio.cmake? It shouldn't be looking for those, it should be looking for -config.cmake

Here is the entire message:

By not providing "Findgfsio.cmake" in CMAKE_MODULE_PATH this project has
 asked CMake to find a package configuration file provided by "gfsio", but
 CMake did not find one.

 Could not find a package configuration file provided by "gfsio" with any of
 the following names:

   gfsioConfig.cmake
   gfsio-config.cmake

@GeorgeGayno-NOAA
Did you first build and install the NCEPLIBS-*?
If you did it in an single installation directory, we should be passing -DCMAKE_PREFIX_PATH=/path/to/nceplibs/install.

@kgerheiser We need a better way to test this combination of NCEPLIBS-* and UFS_UTILS, EMC_post.

kgerheiser · 2020-06-17T14:41:07Z

Update the NCEPLIBS repo and have the entire thing install?

aerorahul · 2020-06-17T14:42:30Z

Update the NCEPLIBS repo and have the entire thing install?

GeorgeGayno-NOAA · 2020-06-17T14:52:35Z

It can't find Findgfsio.cmake? It shouldn't be looking for those, it should be looking for -config.cmake

Here is the entire message:
By not providing "Findgfsio.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "gfsio", but
CMake did not find one.

Could not find a package configuration file provided by "gfsio" with any of
the following names:

gfsioConfig.cmake
gfsio-config.cmake

@GeorgeGayno-NOAA
Did you first build and install the NCEPLIBS-*?
If you did it in an single installation directory, we should be passing -DCMAKE_PREFIX_PATH=/path/to/nceplibs/install.

I did not build NCEPLIBS. I am trying to use the "official" NCEPLIBS on Hera, just as I always do.

@kgerheiser We need a better way to test this combination of NCEPLIBS-* and UFS_UTILS, EMC_post.

kgerheiser · 2020-06-17T14:54:23Z

Building with the new CMake won't work with existing modules or libraries not also built with these changes.

aerorahul · 2020-06-17T14:57:24Z

It can't find Findgfsio.cmake? It shouldn't be looking for those, it should be looking for -config.cmake
Here is the entire message:
By not providing "Findgfsio.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "gfsio", but
CMake did not find one.
Could not find a package configuration file provided by "gfsio" with any of
the following names:
gfsioConfig.cmake
gfsio-config.cmake
@GeorgeGayno-NOAA
Did you first build and install the NCEPLIBS-*?
If you did it in an single installation directory, we should be passing -DCMAKE_PREFIX_PATH=/path/to/nceplibs/install.

I did not build NCEPLIBS. I am trying to use the "official" NCEPLIBS on Hera, just as I always do.

@kgerheiser We need a better way to test this combination of NCEPLIBS-* and UFS_UTILS, EMC_post.

@GeorgeGayno-NOAA That makes sense. This PR uses the imported targets from the NCEPLIBS project-config files. The "official" NCEPLIBS don't provide that information.

We may wait on the NCEPLIBS team to install the NCEPLIBS. They should be identical in terms of the library. The only addition is a name-spaced imported target that CMake creates for us and transitive dependency.

…ved in nceplibs. cleaned up nemsio utilities dependency list

Hang-Lei-NOAA · 2020-09-10T15:52:31Z

Yes, as Kyle said. openmp is also required by the operational use. Hang

…

On Thu, Sep 10, 2020 at 11:48 AM Kyle Gerheiser ***@***.***> wrote: Not so much a bug, but compiled without that option. The default is off. I (and Hang) compiled with -DOPENMP=ON on the non-Jet platforms — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKWSMFCNMFYIWPJ3PVBSS5LSFDYLNANCNFSM4OAFR52A> .

climbfuji · 2020-09-10T16:30:55Z

Yes, as Kyle said. openmp is also required by the operational use. Hang
…

Ok, I'll do the jet install again, this time for NCEPLIBS-external release/public-v2 + NCEPLIBS develop with OpenMP on. Will let you know when it is done.

Hang-Lei-NOAA · 2020-09-10T16:35:07Z

Thanks Dom! Hang

…

On Thu, Sep 10, 2020 at 12:31 PM Dom Heinzeller ***@***.***> wrote: Yes, as Kyle said. openmp is also required by the operational use. Hang … <#m_7138251986199949771_> Ok, I'll do the jet install again, this time for NCEPLIBS-external release/public-v2 + NCEPLIBS develop with OpenMP on. Will let you know when it is done. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKWSMFAHJ7ROGRSWSD3XMEDSFD5NDANCNFSM4OAFR52A> .

climbfuji · 2020-09-10T17:40:32Z

Yes, as Kyle said. openmp is also required by the operational use. Hang
…

Ok, I'll do the jet install again, this time for NCEPLIBS-external release/public-v2 + NCEPLIBS develop with OpenMP on. Will let you know when it is done.

I have this combination ready, but it lacks the modulefiles for the thirdparty (external libraries). I will add this functionality later for a beta02 version.

module purge
module load intel/18.0.5.274
module load impi/2018.4.274
module load netcdf/4.7.0
module load cmake/3.16.1

export CC=icc
export FC=ifort
export CXX=icpc
export ESMFMKFILE=/lfs4/HFIP/hfv3gfs/software/NCEPLIBS-ufs-v2.0.0beta01/intel-18.0.5.274/impi-2018.4.274/lib64/esmf.mk

module use /lfs4/HFIP/hfv3gfs/software/NCEPLIBS-ufs-v2.0.0beta01/intel-18.0.5.274/impi-2018.4.274/modules
module av

Note that the modules (in particular) netCDF for release/public-v2 are different from the "develop" versions, because we don't need to build netCDF ourselves (no parallel netCDF in SRW App 1.0).

If this works for UFS_UTILS, then I will add the missing modulefile creating for NCEPLIBS-external, do a few additional updates not yet included in the release/public-v2 branch on GitHub, and forward NCEPLIBS release/public-v2 to the tag v1.2.0.

kgerheiser · 2020-09-10T20:02:41Z

I updated build.jet to point to Dom's new build

kgerheiser · 2020-09-10T21:30:34Z

I believe my regression tests passed on Jet with the current build

GeorgeGayno-NOAA · 2020-09-11T12:14:44Z

I believe my regression tests passed on Jet with the current build

I just tried cbd3a43 on Jet. My tests passed as well.

Two more machines to go: WCOSS-Cray and Dell. They should be back in service today.

GeorgeGayno-NOAA · 2020-09-11T15:14:52Z

Trying to compile cbd3a43 on Surge. It can't find the gfsio library. I think the base path in the gfsio module is wrong. The gfsio module on Luna looks correct. @Hang-Lei-NOAA can you please check.

Hang-Lei-NOAA · 2020-09-11T15:16:45Z

I will check it. Thanks, Hang

…

On Fri, Sep 11, 2020 at 11:15 AM GeorgeGayno-NOAA ***@***.***> wrote: Trying to compile cbd3a43 <cbd3a43> on Surge. It can't find the gfsio library. I think the base path in the gfsio module is wrong. The gfsio module on Luna looks correct. @Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> can you please check. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKWSMFEIVISVYG4JZY53ZG3SFI5H5ANCNFSM4OAFR52A> .

Hang-Lei-NOAA · 2020-09-11T15:22:26Z

Hi, George Please module use /usrx/local/nceplibs/NCEPLIBS/cmake/install/NCEPLIBS-v1.2.0/modules module load gfsio/1.4.1 This is rsyn from Luna to ensure identical. Please test. Thanks, Hang On Fri, Sep 11, 2020 at 11:16 AM Hang Lei - NOAA Affiliate < hang.lei@noaa.gov> wrote:

…

I will check it. Thanks, Hang On Fri, Sep 11, 2020 at 11:15 AM GeorgeGayno-NOAA < ***@***.***> wrote: > Trying to compile cbd3a43 > <cbd3a43> > on Surge. It can't find the gfsio library. I think the base path in the > gfsio module is wrong. The gfsio module on Luna looks correct. > @Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> can you please check. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#118 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AKWSMFEIVISVYG4JZY53ZG3SFI5H5ANCNFSM4OAFR52A> > . >

GeorgeGayno-NOAA · 2020-09-11T15:35:37Z

Hi, George Please module use /usrx/local/nceplibs/NCEPLIBS/cmake/install/NCEPLIBS-v1.2.0/modules module load gfsio/1.4.1 This is rsyn from Luna to ensure identical. Please test. Thanks, Hang On Fri, Sep 11, 2020 at 11:16 AM Hang Lei - NOAA Affiliate < hang.lei@noaa.gov> wrote:
…
I will check it. Thanks, Hang On Fri, Sep 11, 2020 at 11:15 AM GeorgeGayno-NOAA < @.***> wrote: > Trying to compile cbd3a43 > <cbd3a43> > on Surge. It can't find the gfsio library. I think the base path in the > gfsio module is wrong. The gfsio module on Luna looks correct. > @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA can you please check. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#118 (comment)>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/AKWSMFEIVISVYG4JZY53ZG3SFI5H5ANCNFSM4OAFR52A > . >

Thanks, can you also check Venus (/usrx/local/nceplibs/dev/NCEPLIBS/cmake/install/NCEPLIBS-v1.2.0). I see the libraries. But I don't see any module files. Thanks.

GeorgeGayno-NOAA · 2020-09-11T18:32:44Z

Just tested c649d9c on WCOSS-Cray. Everything built and all regression tests and utility scripts ran correctly.

One more machine to go: WCOSS-Dell.

Hang-Lei-NOAA · 2020-09-11T18:37:01Z

The NCO PM test got several library module build ended. George has pointed out. I am rebuilding them now on Dell.

…

On Fri, Sep 11, 2020 at 2:33 PM GeorgeGayno-NOAA ***@***.***> wrote: Just tested c649d9c <c649d9c> on WCOSS-Cray. Everything built and all regression tests and utility scripts ran correctly. One more machine to go: WCOSS-Dell. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKWSMFAALOU7V7IZTN6X6Z3SFJUNZANCNFSM4OAFR52A> .

This reverts commit 35e03ce.

GeorgeGayno-NOAA · 2020-09-14T17:34:57Z

Just tested 7e72c57 on WCOSS-Dell. Everything built OK. All regression tests and utility scripts ran fine.

The branch was successfully tested on Jet, Hera, Orion, WCOSS-Cray and WCOSS-Dell. Will merge this afternoon. Yeah!

climbfuji · 2020-09-14T17:40:11Z

For my limited understanding. Is this branch also required for the SRW App, or are the changes for SRW coming later?

kgerheiser · 2020-09-14T17:43:35Z

Both, this branch and then the other PR also.

climbfuji · 2020-09-14T17:46:38Z

Both, this branch and then the other PR also.

Ok, thanks for letting me know. Are the NCEPLIBS finalized? In other words, are all the changes required for UFS_UTILS ready in NCEPLIBS? If so, I will test that version with the ufs-weather-model to make sure it doesn't change the answer (it shouldn't, the differences I saw last week came from ESMF) and update the NCEPLIBS release/public-v2 branch to match the current hash of develop.

kgerheiser · 2020-09-14T17:49:27Z

NCEPLIBS is ready to go

Hang-Lei-NOAA · 2020-09-14T17:55:08Z

Make sure use the NCEPLIBS-v1.2.0 other old versions are not removed, since some people still want them.

…

On Mon, Sep 14, 2020 at 1:49 PM Kyle Gerheiser ***@***.***> wrote: NCEPLIBS is ready to go — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKWSMFCQWTTH7S2D3OKB4CTSFZJTNANCNFSM4OAFR52A> .

GeorgeGayno-NOAA

Branch was successfully tested on Jet, Hera, Orion, WCOSS-Cray and WCOSS-Dell. Will merge.

aerorahul added 6 commits June 16, 2020 18:24

move cmake to CMakeModules

a0f3c54

Change-Id: I8e7526ef1b796c3653f06d83d65ccc1d4df2483e

update pointer to CMakeModules

597f47f

Change-Id: I20a151f91202f78f4b125b6d3b99c8950c265ce7

use imported targets for NCEPLIBS

9a6bd56

Change-Id: I332a52f9a1ae79bfa5f05e8565b487f2672f5c7d

update pointer to CMakeModules

039d884

update pointer to CMakeModules

15d58e0

update gitmodule to point to correct branch of CMakeModules

08c4475

aerorahul requested review from GeorgeGayno-NOAA, DusanJovic-NOAA and kgerheiser June 17, 2020 02:58

GeorgeGayno-NOAA reviewed Jun 17, 2020

View reviewed changes

DusanJovic-NOAA reviewed Jun 17, 2020

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

DusanJovic-NOAA reviewed Jun 17, 2020

View reviewed changes

sorc/emcsfc_ice_blend.fd/CMakeLists.txt Outdated Show resolved Hide resolved

DusanJovic-NOAA reviewed Jun 17, 2020

View reviewed changes

sorc/emcsfc_snow2mdl.fd/CMakeLists.txt Outdated Show resolved Hide resolved

DusanJovic-NOAA reviewed Jun 17, 2020

View reviewed changes

sorc/nst_tf_chg.fd/CMakeLists.txt Outdated Show resolved Hide resolved

aerorahul closed this Jun 17, 2020

aerorahul reopened this Jun 17, 2020

aerorahul mentioned this pull request Jun 17, 2020

An umbrella build for NCEPLIBS NOAA-EMC/NCEPLIBS#76

Merged

removed explicit dependency on png, jasper and zlib as they are resol…

6c73bff

…ved in nceplibs. cleaned up nemsio utilities dependency list

aerorahul requested a review from DusanJovic-NOAA June 17, 2020 21:02

Update build.jet to point to OpenMP-enabled sp lib

1710db6

Update build.jet

cbd3a43

Update build.wcoss_cray

c649d9c

kgerheiser and others added 3 commits September 14, 2020 11:37

Update module_base.wcoss_dell_p3

35e03ce

Update build.wcoss_dell_p3

09f1baf

Revert "Update module_base.wcoss_dell_p3"

7e72c57

This reverts commit 35e03ce.

GeorgeGayno-NOAA self-assigned this Sep 14, 2020

GeorgeGayno-NOAA self-requested a review September 14, 2020 19:12

GeorgeGayno-NOAA approved these changes Sep 14, 2020

View reviewed changes

GeorgeGayno-NOAA merged commit 8404a4d into ufs-community:develop Sep 14, 2020

aerorahul deleted the feature/cmakepkgs branch September 14, 2020 20:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to use project configs from NCEPLIBS #118

Update to use project configs from NCEPLIBS #118

aerorahul commented Jun 17, 2020 •

edited

Loading

GeorgeGayno-NOAA left a comment

DusanJovic-NOAA commented Jun 17, 2020

GeorgeGayno-NOAA commented Jun 17, 2020

kgerheiser commented Jun 17, 2020 •

edited

Loading

GeorgeGayno-NOAA commented Jun 17, 2020

kgerheiser commented Jun 17, 2020 •

edited

Loading

aerorahul commented Jun 17, 2020

kgerheiser commented Jun 17, 2020

aerorahul commented Jun 17, 2020

GeorgeGayno-NOAA commented Jun 17, 2020

kgerheiser commented Jun 17, 2020

aerorahul commented Jun 17, 2020

Hang-Lei-NOAA commented Sep 10, 2020 via email

climbfuji commented Sep 10, 2020

Hang-Lei-NOAA commented Sep 10, 2020 via email

climbfuji commented Sep 10, 2020

kgerheiser commented Sep 10, 2020 •

edited

Loading

kgerheiser commented Sep 10, 2020

GeorgeGayno-NOAA commented Sep 11, 2020

GeorgeGayno-NOAA commented Sep 11, 2020

Hang-Lei-NOAA commented Sep 11, 2020 via email

Hang-Lei-NOAA commented Sep 11, 2020 via email

GeorgeGayno-NOAA commented Sep 11, 2020

GeorgeGayno-NOAA commented Sep 11, 2020

Hang-Lei-NOAA commented Sep 11, 2020 via email

GeorgeGayno-NOAA commented Sep 14, 2020

climbfuji commented Sep 14, 2020

kgerheiser commented Sep 14, 2020

climbfuji commented Sep 14, 2020

kgerheiser commented Sep 14, 2020

Hang-Lei-NOAA commented Sep 14, 2020 via email

GeorgeGayno-NOAA left a comment

Update to use project configs from NCEPLIBS #118

Update to use project configs from NCEPLIBS #118

Conversation

aerorahul commented Jun 17, 2020 • edited Loading

GeorgeGayno-NOAA left a comment

Choose a reason for hiding this comment

DusanJovic-NOAA commented Jun 17, 2020

GeorgeGayno-NOAA commented Jun 17, 2020

kgerheiser commented Jun 17, 2020 • edited Loading

GeorgeGayno-NOAA commented Jun 17, 2020

kgerheiser commented Jun 17, 2020 • edited Loading

aerorahul commented Jun 17, 2020

kgerheiser commented Jun 17, 2020

aerorahul commented Jun 17, 2020

GeorgeGayno-NOAA commented Jun 17, 2020

kgerheiser commented Jun 17, 2020

aerorahul commented Jun 17, 2020

Hang-Lei-NOAA commented Sep 10, 2020 via email

climbfuji commented Sep 10, 2020

Hang-Lei-NOAA commented Sep 10, 2020 via email

climbfuji commented Sep 10, 2020

kgerheiser commented Sep 10, 2020 • edited Loading

kgerheiser commented Sep 10, 2020

GeorgeGayno-NOAA commented Sep 11, 2020

GeorgeGayno-NOAA commented Sep 11, 2020

Hang-Lei-NOAA commented Sep 11, 2020 via email

Hang-Lei-NOAA commented Sep 11, 2020 via email

GeorgeGayno-NOAA commented Sep 11, 2020

GeorgeGayno-NOAA commented Sep 11, 2020

Hang-Lei-NOAA commented Sep 11, 2020 via email

GeorgeGayno-NOAA commented Sep 14, 2020

climbfuji commented Sep 14, 2020

kgerheiser commented Sep 14, 2020

climbfuji commented Sep 14, 2020

kgerheiser commented Sep 14, 2020

Hang-Lei-NOAA commented Sep 14, 2020 via email

GeorgeGayno-NOAA left a comment

Choose a reason for hiding this comment

aerorahul commented Jun 17, 2020 •

edited

Loading

kgerheiser commented Jun 17, 2020 •

edited

Loading

kgerheiser commented Jun 17, 2020 •

edited

Loading

kgerheiser commented Sep 10, 2020 •

edited

Loading