Add MET and METplus model verification tools to hpc-stack

[edwardhartnett]

See discussion in #83.

[edwardhartnett]

The first question is: what are the MET plus libraries? ;-)

Is there a URL or repo?

[jprestop]

Thank you for creating this issue.  METplus isn't quite a set of libraries. Regarding what METplus is, I invite you to take a look at the "METplus Concept" section of the METplus documentation.  I believe the work for hpc-stack will involve METplus wrappers and MET.  From our User's Guide, "MET is a suite of verification tools developed and supported to the community via the Developmental Testbed Center (DTC) for use by the numerical weather prediction community. METplus Wrappers are intended to be a suite of Python wrappers and ancillary scripts to enhance the user’s ability to quickly set-up and run MET. Over the next year, METplus Wrappers will become the authoritative repository for verification of the Unified Forecast System."

Links to the:
METplus GitHub repository
MET GitHub repository
The download page for the MET code

Please assign this task to me, Julie Prestopnik, jprestop.  I look forward to learning more about hpc-stack and to working with you.

[edwardhartnett]

Howdy Julie! ;-)

Do we have a requirement for when hpc-stack must be released with the MET/METplus changes? That is, is there a due date for this work?

[jprestop]

I just wanted to update this issue to let you know that I am starting to learn about hpc-stack now.

[edwardhartnett]

@jprestop you and me both! ;-)

[jprestop]

I have made changes in my GitHub repository to add MET and am ready to test on WCOSS, however, as you may have seen, there are currently devonprod access problems and I cannot access WCOSS. I will start on making changes to add METplus (which should go much more quickly now that I am more familiar with hpc-stack, and also the METplus installation has less work involved.

[aerorahul]

@jprestop
Do you have an update for us on this issue?

[jprestop]

@aerorahul
Hi Rahul. I have been working with Kyle to resolve some problems I experienced in trying to test the scripts I added. I had issues loading and building the necessary modules on WCOSS that were required for MET. With Kyle's (@kgerheiser) help, I was able to resolve those issue and am now able to work on testing the MET build. There are errors occurring in the make.log file for MET that I am working on resolving today. Due to the upcoming parallel production test and production switch, I am hoping to resolve these issues today.

There is one other issue that arose recently that impacts MET in hpc-stack. Mallory Row and I have been working with Boi Vuong regarding some issues we have had with the g2c library not working well on large FV2 files.  Boi said It is a known issue with g2c and g2 libraries where they can not handle files larger than 2GB.  We found that when removing the 64BIT flag, the large files process well. Boi planned to submit an RFC to NCO for g2c v1.6.3 on Wednesday, February 10th. MET should compile fine with the version 1.6.2 currently in hpc-stack, but will not work on large FV3 files. Once 1.6.3 is available, will hpc-stack be able to upgrade to use that version?  I will also need to remove some lines from build_met.sh in hpc-stack once the upgrade is made. Please let me know if you have any questions or concerns.

[aerorahul]

@jprestop
Thanks for the update. I was mainly making sure this is progressing.

[jprestop]

@aerorahul
I have submitted a pull request for this work: #169
Please note that, as stated above, for this installation, large files (e.g. FV3 files) over 2GB may not work with MET/METplus using the g2c library version 1.6.2. Once g2c 1.6.3 is integrated into hpc-stack and some lines are removed from build_met.sh in hpc-stack afterward, the g2c library and MET/METplus will handle large files well.

[aerorahul]

Closed by #324