====START OF HERA REGRESSION TESTING LOG====

UFSWM hash used in testing:
b5ae65b035142bee8c3d82b954fdabdda58f7992

Submodule hashes used in testing:
 37cbb7d6840ae7515a9a8f0dfd4d89461b3396d1 AQM (v0.2.0-37-g37cbb7d)
 1f9eaaa142c8b07ed6b788c9f44ea02cc86d0bae CDEPS-interface/CDEPS (cdeps0.4.17-42-g1f9eaaa)
 5e57a89cf533fe98352bcfd1a464b1d50713274d CICE-interface/CICE (CICE6.0.0-378-g5e57a89)
 24e9eed4ffe8138bef635c8f916f91b142595675 CMEPS-interface/CMEPS (cmeps_v0.4.1-2311-g24e9eed)
 cabd7753ae17f7bfcc6dad56daf10868aa51c3f4 CMakeModules (v1.0.0-28-gcabd775)
 1e10d62a971e4114faea90ae2f134664f3a9ef02 FV3 (remotes/origin/production/HREF.v3beta-341-g1e10d62)
 041422934cae1570f2f0e67239d5d89f11c6e1b7 GOCART (sdr_v2.1.2.6-119-g0414229)
 bcf7777bb037ae2feb2a8a8ac51aacb3511b52d9 HYCOM-interface/HYCOM (2.3.00-122-gbcf7777)
 c4d2337d8607ec994b3cd61179eb974e0a237841 LM4-driver (baseline_change_240904-6-gc4d2337)
 5e0c21f64fa5b20efc8f29f8709766e1e6793a79 MOM6-interface/MOM6 (dev/master/repository_split_2014.10.10-10230-g5e0c21f64)
 3ac32f0db7a2a97d930f44fa5f060c983ff31ee8 NOAHMP-interface/noahmp (v3.7.1-436-g3ac32f0)
 06c20998e59082e79529a8e728c7c3a43a050911 WW3 (6.07.1-385-g06c20998)
 05cad173feeb598431e3ef5f17c2df6562c8d101 fire_behavior (v0.2.0-1-g05cad17)
 fad2fe9f42f6b7f744b128b4a2a9433f91e4296f stochastic_physics (ufs-v2.0.0-219-gfad2fe9)


NOTES:
[Times](Memory) are at the end of each compile/test in format [MM:SS](Size).
The first time is for the full script (prep+run+finalize).
The second time is specifically for the run phase.
Times/Memory will be empty for failed tests.

BASELINE DIRECTORY: /scratch1/NCEPDEV/stmp4/Matthew.Masarik/FV3_RT/REGRESSION_TEST
COMPARISON DIRECTORY: /scratch1/NCEPDEV/stmp2/Matthew.Masarik/FV3_RT/rt_2575456

RT.SH OPTIONS USED:
* (-a) - HPC PROJECT ACCOUNT: marine-cpu
* (-m) - COMPARE AGAINST CREATED BASELINES
* (-k) - KEEP RUN DIRECTORY
* (-e) - USE ECFLOW

PASS -- COMPILE 's2swa_32bit_intel' [14:12, 12:38] ( 1 warnings 9 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_p8_mixedmode_intel' [, ]( MB)

PASS -- COMPILE 's2swa_32bit_pdlib_intel' [18:13, 16:44] ( 1 warnings 9 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_gfsv17_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_control_gfsv17_iau_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_restart_gfsv17_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_mpi_gfsv17_intel' [, ]( MB)

PASS -- COMPILE 's2swa_32bit_pdlib_sfs_intel' [17:14, 15:35] ( 1 warnings 9 remarks )
PASS -- TEST 'cpld_control_sfs_intel' [18:48, 16:42](1968 MB)

PASS -- COMPILE 's2swa_32bit_pdlib_debug_intel' [08:12, 06:16] ( 1520 warnings 1986 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_debug_gfsv17_intel' [, ]( MB)

PASS -- COMPILE 's2swa_intel' [14:13, 12:38] ( 1 warnings 9 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_p8.v2.sfc_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_restart_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_qr_p8_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_restart_qr_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_2threads_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_decomp_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_mpi_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_ciceC_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_c192_p8_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_restart_c192_p8_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_bmark_p8_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_restart_bmark_p8_intel' [, ]( MB)
PASS -- TEST 'cpld_s2sa_p8_intel' [08:23, 05:27](3317 MB)

PASS -- COMPILE 's2sw_intel' [13:12, 11:42] ( 1 warnings 9 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_noaero_p8_intel' [, ]( MB)
PASS -- TEST 'cpld_control_nowave_noaero_p8_intel' [10:16, 07:25](2045 MB)

PASS -- COMPILE 's2swa_debug_intel' [08:12, 06:16] ( 1410 warnings 1216 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_debug_p8_intel' [, ]( MB)

PASS -- COMPILE 's2sw_debug_intel' [07:13, 05:52] ( 1410 warnings 1216 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_debug_noaero_p8_intel' [, ]( MB)

PASS -- COMPILE 's2s_aoflux_intel' [13:12, 11:19] ( 1 remarks )
PASS -- TEST 'cpld_control_noaero_p8_agrid_intel' [07:05, 04:44](1991 MB)

PASS -- COMPILE 's2s_intel' [12:13, 11:08] ( 1 warnings 1 remarks )
PASS -- TEST 'cpld_control_c48_intel' [09:47, 07:41](3046 MB)
PASS -- TEST 'cpld_warmstart_c48_intel' [04:43, 02:43](3035 MB)
PASS -- TEST 'cpld_restart_c48_intel' [03:44, 01:32](2473 MB)

PASS -- COMPILE 's2swa_faster_intel' [18:15, 16:49] ( 1 warnings 9 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_p8_faster_intel' [, ]( MB)

PASS -- COMPILE 's2sw_pdlib_intel' [17:14, 15:45] ( 1 warnings 9 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_pdlib_p8_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_restart_pdlib_p8_intel' [, ]( MB)
FAILED: UNABLE TO START TEST -- TEST 'cpld_mpi_pdlib_p8_intel' [, ]( MB)

PASS -- COMPILE 's2sw_pdlib_debug_intel' [07:12, 06:04] ( 1520 warnings 1986 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_debug_pdlib_p8_intel' [, ]( MB)

PASS -- COMPILE 'atm_dyn32_intel' [12:12, 10:32] ( 1 warnings 1 remarks )
PASS -- TEST 'control_flake_intel' [05:30, 03:20](707 MB)
PASS -- TEST 'control_CubedSphereGrid_intel' [05:52, 03:11](1582 MB)
PASS -- TEST 'control_CubedSphereGrid_parallel_intel' [05:59, 03:15](1598 MB)
PASS -- TEST 'control_latlon_intel' [05:46, 03:13](1582 MB)
PASS -- TEST 'control_wrtGauss_netcdf_parallel_intel' [05:54, 03:14](1588 MB)
PASS -- TEST 'control_c48_intel' [13:52, 11:20](1713 MB)
PASS -- TEST 'control_c48.v2.sfc_intel' [08:33, 06:28](848 MB)
PASS -- TEST 'control_c192_intel' [14:08, 11:51](1757 MB)
PASS -- TEST 'control_c384_intel' [17:01, 13:46](1982 MB)
PASS -- TEST 'control_c384gdas_intel' [13:20, 08:23](1374 MB)
PASS -- TEST 'control_stochy_intel' [03:27, 01:38](661 MB)
PASS -- TEST 'control_stochy_restart_intel' [03:32, 00:57](496 MB)
PASS -- TEST 'control_lndp_intel' [03:26, 01:31](657 MB)
PASS -- TEST 'control_iovr4_intel' [04:26, 02:26](652 MB)
PASS -- TEST 'control_iovr5_intel' [04:26, 02:26](654 MB)
PASS -- TEST 'control_p8_intel' [05:58, 03:47](1824 MB)
PASS -- TEST 'control_p8.v2.sfc_intel' [06:01, 03:10](1874 MB)
PASS -- TEST 'control_p8_ugwpv1_intel' [05:52, 03:37](1879 MB)
PASS -- TEST 'control_restart_p8_intel' [03:48, 02:02](1119 MB)
PASS -- TEST 'control_noqr_p8_intel' [05:48, 03:34](1846 MB)
PASS -- TEST 'control_restart_noqr_p8_intel' [04:47, 01:59](1145 MB)
PASS -- TEST 'control_decomp_p8_intel' [06:46, 03:39](1874 MB)
PASS -- TEST 'control_2threads_p8_intel' [05:46, 03:27](1957 MB)
PASS -- TEST 'control_p8_lndp_intel' [08:42, 06:28](1876 MB)
PASS -- TEST 'control_p8_rrtmgp_intel' [05:50, 03:59](1944 MB)
PASS -- TEST 'control_p8_mynn_intel' [04:51, 03:02](1893 MB)
PASS -- TEST 'merra2_thompson_intel' [05:47, 03:22](1896 MB)
PASS -- TEST 'regional_control_intel' [07:39, 05:15](1091 MB)
PASS -- TEST 'regional_restart_intel' [04:40, 02:49](1086 MB)
PASS -- TEST 'regional_decomp_intel' [07:38, 05:31](1086 MB)
PASS -- TEST 'regional_2threads_intel' [05:38, 03:46](1036 MB)
PASS -- TEST 'regional_noquilt_intel' [07:41, 05:11](1387 MB)
PASS -- TEST 'regional_netcdf_parallel_intel' [07:45, 05:15](1091 MB)
PASS -- TEST 'regional_2dwrtdecomp_intel' [07:40, 05:18](1102 MB)
PASS -- TEST 'regional_wofs_intel' [08:39, 06:49](1894 MB)

PASS -- COMPILE 'rrfs_intel' [11:14, 09:40] ( 3 warnings 9 remarks )
PASS -- TEST 'rap_control_intel' [09:45, 07:42](1090 MB)
PASS -- TEST 'regional_spp_sppt_shum_skeb_intel' [07:07, 04:07](1239 MB)
PASS -- TEST 'rap_decomp_intel' [09:35, 08:06](1034 MB)
PASS -- TEST 'rap_2threads_intel' [10:42, 07:19](1153 MB)
PASS -- TEST 'rap_restart_intel' [05:42, 04:01](1073 MB)
PASS -- TEST 'rap_sfcdiff_intel' [10:43, 07:44](1084 MB)
PASS -- TEST 'rap_sfcdiff_decomp_intel' [10:37, 08:08](1033 MB)
PASS -- TEST 'rap_sfcdiff_restart_intel' [07:44, 05:51](1100 MB)
PASS -- TEST 'hrrr_control_intel' [06:43, 03:57](1039 MB)
PASS -- TEST 'hrrr_control_decomp_intel' [06:42, 04:03](1028 MB)
PASS -- TEST 'hrrr_control_2threads_intel' [06:33, 03:37](1102 MB)
PASS -- TEST 'hrrr_control_restart_intel' [04:29, 02:06](979 MB)
PASS -- TEST 'rrfs_v1beta_intel' [09:52, 07:43](1066 MB)
PASS -- TEST 'rrfs_v1nssl_intel' [11:27, 09:22](1979 MB)
PASS -- TEST 'rrfs_v1nssl_nohailnoccn_intel' [11:27, 08:57](2053 MB)

PASS -- COMPILE 'csawmg_intel' [11:14, 09:31]
PASS -- TEST 'control_csawmg_intel' [08:36, 06:09](1026 MB)
PASS -- TEST 'control_ras_intel' [05:22, 03:13](741 MB)

PASS -- COMPILE 'csawmg_gnu' [06:13, 04:12]
PASS -- TEST 'control_csawmg_gnu' [10:44, 08:32](733 MB)

PASS -- COMPILE 'wam_intel' [11:13, 09:31] ( 1 remarks )
PASS -- TEST 'control_wam_intel' [12:56, 11:04](1647 MB)

PASS -- COMPILE 'atm_faster_dyn32_intel' [11:14, 09:49] ( 1 remarks )
PASS -- TEST 'control_p8_faster_intel' [04:56, 02:54](1855 MB)
PASS -- TEST 'regional_control_faster_intel' [06:39, 04:48](1074 MB)

PASS -- COMPILE 'atm_debug_dyn32_intel' [09:14, 07:57] ( 882 warnings 9 remarks )
PASS -- TEST 'control_CubedSphereGrid_debug_intel' [04:41, 02:10](1607 MB)
PASS -- TEST 'control_wrtGauss_netcdf_parallel_debug_intel' [03:41, 02:02](1603 MB)
PASS -- TEST 'control_stochy_debug_intel' [04:23, 02:58](828 MB)
PASS -- TEST 'control_lndp_debug_intel' [04:23, 02:41](837 MB)
PASS -- TEST 'control_csawmg_debug_intel' [06:36, 04:08](1091 MB)
PASS -- TEST 'control_ras_debug_intel' [04:23, 02:45](813 MB)
PASS -- TEST 'control_diag_debug_intel' [04:45, 02:42](1639 MB)
PASS -- TEST 'control_debug_p8_intel' [04:39, 02:33](1856 MB)
PASS -- TEST 'regional_debug_intel' [19:41, 17:11](1055 MB)
PASS -- TEST 'rap_control_debug_intel' [06:24, 04:52](1192 MB)
PASS -- TEST 'hrrr_control_debug_intel' [06:24, 04:49](1173 MB)
PASS -- TEST 'hrrr_gf_debug_intel' [06:21, 04:46](1202 MB)
PASS -- TEST 'hrrr_c3_debug_intel' [06:23, 04:48](1174 MB)
PASS -- TEST 'rap_unified_drag_suite_debug_intel' [06:22, 04:51](1174 MB)
PASS -- TEST 'rap_diag_debug_intel' [07:39, 05:03](1246 MB)
PASS -- TEST 'rap_cires_ugwp_debug_intel' [06:23, 04:53](1200 MB)
PASS -- TEST 'rap_unified_ugwp_debug_intel' [06:22, 04:51](1200 MB)
PASS -- TEST 'rap_lndp_debug_intel' [06:23, 04:51](1203 MB)
PASS -- TEST 'rap_progcld_thompson_debug_intel' [06:23, 04:48](1200 MB)
PASS -- TEST 'rap_noah_debug_intel' [06:23, 04:49](1205 MB)
PASS -- TEST 'rap_sfcdiff_debug_intel' [06:24, 04:48](1198 MB)
PASS -- TEST 'rap_noah_sfcdiff_cires_ugwp_debug_intel' [09:26, 07:54](1171 MB)
PASS -- TEST 'rrfs_v1beta_debug_intel' [06:23, 04:43](1164 MB)
PASS -- TEST 'rap_clm_lake_debug_intel' [07:24, 05:51](1170 MB)
PASS -- TEST 'rap_flake_debug_intel' [06:23, 04:43](1171 MB)
PASS -- TEST 'gnv1_c96_no_nest_debug_intel' [10:40, 08:21](1178 MB)

PASS -- COMPILE 'atm_debug_dyn32_gnu' [06:14, 04:34]
PASS -- TEST 'control_csawmg_debug_gnu' [04:36, 02:15](702 MB)

PASS -- COMPILE 'wam_debug_intel' [06:12, 04:41] ( 837 warnings 1 remarks )
PASS -- TEST 'control_wam_debug_intel' [15:52, 13:10](1638 MB)

PASS -- COMPILE 'rrfs_dyn32_phy32_intel' [11:14, 09:16] ( 3 warnings 8 remarks )
PASS -- TEST 'regional_spp_sppt_shum_skeb_dyn32_phy32_intel' [07:01, 03:50](1098 MB)
PASS -- TEST 'rap_control_dyn32_phy32_intel' [08:39, 06:25](1025 MB)
PASS -- TEST 'hrrr_control_dyn32_phy32_intel' [05:33, 03:22](959 MB)
PASS -- TEST 'rap_2threads_dyn32_phy32_intel' [08:34, 06:04](1064 MB)
PASS -- TEST 'hrrr_control_2threads_dyn32_phy32_intel' [04:35, 03:06](932 MB)
PASS -- TEST 'hrrr_control_decomp_dyn32_phy32_intel' [05:31, 03:33](917 MB)
PASS -- TEST 'rap_restart_dyn32_phy32_intel' [06:39, 04:50](1003 MB)
PASS -- TEST 'hrrr_control_restart_dyn32_phy32_intel' [03:26, 01:49](902 MB)

PASS -- COMPILE 'rrfs_dyn32_phy32_faster_intel' [11:13, 09:23] ( 3 warnings 8 remarks )
PASS -- TEST 'conus13km_control_intel' [03:53, 01:57](1165 MB)
PASS -- TEST 'conus13km_2threads_intel' [02:39, 00:52](1084 MB)
PASS -- TEST 'conus13km_restart_mismatch_intel' [03:41, 01:11](1072 MB)

PASS -- COMPILE 'rrfs_dyn64_phy32_intel' [11:14, 09:27] ( 3 warnings 8 remarks )
PASS -- TEST 'rap_control_dyn64_phy32_intel' [06:39, 04:10](960 MB)

PASS -- COMPILE 'rrfs_dyn32_phy32_debug_intel' [06:14, 04:21] ( 785 warnings 8 remarks )
PASS -- TEST 'rap_control_debug_dyn32_phy32_intel' [06:24, 04:41](1056 MB)
PASS -- TEST 'hrrr_control_debug_dyn32_phy32_intel' [06:25, 04:37](1045 MB)
PASS -- TEST 'conus13km_debug_intel' [15:50, 13:08](1178 MB)
PASS -- TEST 'conus13km_debug_qr_intel' [15:47, 13:13](898 MB)
PASS -- TEST 'conus13km_debug_2threads_intel' [09:40, 07:36](1095 MB)
PASS -- TEST 'conus13km_radar_tten_debug_intel' [15:42, 13:43](1262 MB)

PASS -- COMPILE 'rrfs_dyn64_phy32_debug_intel' [06:14, 04:33] ( 785 warnings 8 remarks )
PASS -- TEST 'rap_control_dyn64_phy32_debug_intel' [06:24, 04:53](1098 MB)

PASS -- COMPILE 'hafsw_intel' [12:14, 11:00] ( 1 warnings 9 remarks )
PASS -- TEST 'hafs_regional_atm_intel' [08:17, 04:45](695 MB)
PASS -- TEST 'hafs_regional_atm_thompson_gfdlsf_intel' [08:34, 05:38](1075 MB)
PASS -- TEST 'hafs_regional_atm_ocn_intel' [09:25, 06:46](782 MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'hafs_regional_atm_wav_intel' [, ]( MB)
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'hafs_regional_atm_ocn_wav_intel' [, ]( MB)
PASS -- TEST 'hafs_regional_1nest_atm_intel' [07:54, 05:19](466 MB)
PASS -- TEST 'hafs_regional_telescopic_2nests_atm_intel' [09:32, 06:18](479 MB)
PASS -- TEST 'hafs_global_1nest_atm_intel' [04:43, 02:33](359 MB)
PASS -- TEST 'hafs_global_multiple_4nests_atm_intel' [11:40, 07:15](457 MB)
PASS -- TEST 'hafs_regional_specified_moving_1nest_atm_intel' [05:46, 03:34](489 MB)
PASS -- TEST 'hafs_regional_storm_following_1nest_atm_intel' [05:51, 03:21](494 MB)
PASS -- TEST 'hafs_regional_storm_following_1nest_atm_ocn_intel' [06:01, 04:07](553 MB)
PASS -- TEST 'hafs_global_storm_following_1nest_atm_intel' [03:31, 01:09](381 MB)
PASS -- TEST 'gnv1_nested_intel' [06:41, 04:01](1715 MB)

PASS -- COMPILE 'hafsw_debug_intel' [06:13, 05:05] ( 1462 warnings 1489 remarks )
PASS -- TEST 'hafs_regional_storm_following_1nest_atm_ocn_debug_intel' [14:56, 12:31](546 MB)

PASS -- COMPILE 'hafsw_faster_intel' [12:13, 10:31] ( 8 remarks )
PASS -- TEST 'hafs_regional_storm_following_1nest_atm_ocn_wav_intel' [11:03, 08:49](606 MB)
PASS -- TEST 'hafs_regional_storm_following_1nest_atm_ocn_wav_inline_intel' [12:05, 08:35](696 MB)

PASS -- COMPILE 'hafs_mom6w_intel' [12:15, 10:47] ( 8 remarks )
PASS -- TEST 'hafs_regional_storm_following_1nest_atm_ocn_wav_mom6_intel' [09:47, 06:33](693 MB)

PASS -- COMPILE 'hafs_all_intel' [12:15, 10:22] ( 8 remarks )
PASS -- TEST 'hafs_regional_docn_intel' [09:24, 06:17](791 MB)
PASS -- TEST 'hafs_regional_docn_oisst_intel' [09:24, 06:24](776 MB)
PASS -- TEST 'hafs_regional_datm_cdeps_intel' [18:56, 16:22](1222 MB)

PASS -- COMPILE 'datm_cdeps_intel' [08:13, 06:26]
PASS -- TEST 'datm_cdeps_control_cfsr_intel' [04:20, 02:56](1156 MB)
PASS -- TEST 'datm_cdeps_restart_cfsr_intel' [03:26, 01:40](1108 MB)
PASS -- TEST 'datm_cdeps_control_gefs_intel' [04:19, 02:36](1031 MB)
PASS -- TEST 'datm_cdeps_iau_gefs_intel' [04:19, 02:41](1017 MB)
PASS -- TEST 'datm_cdeps_stochy_gefs_intel' [04:20, 02:43](1009 MB)
PASS -- TEST 'datm_cdeps_ciceC_cfsr_intel' [04:19, 02:37](1142 MB)
PASS -- TEST 'datm_cdeps_bulk_cfsr_intel' [04:20, 02:41](1134 MB)
PASS -- TEST 'datm_cdeps_bulk_gefs_intel' [04:19, 02:35](1013 MB)
PASS -- TEST 'datm_cdeps_mx025_cfsr_intel' [09:55, 06:16](1061 MB)
PASS -- TEST 'datm_cdeps_mx025_gefs_intel' [09:53, 06:16](1040 MB)
PASS -- TEST 'datm_cdeps_multiple_files_cfsr_intel' [04:18, 02:43](1152 MB)
PASS -- TEST 'datm_cdeps_3072x1536_cfsr_intel' [06:20, 04:35](2442 MB)
PASS -- TEST 'datm_cdeps_gfs_intel' [05:21, 03:55](2506 MB)

PASS -- COMPILE 'datm_cdeps_debug_intel' [05:13, 03:19] ( 2 warnings )
PASS -- TEST 'datm_cdeps_debug_cfsr_intel' [08:20, 06:35](1069 MB)

PASS -- COMPILE 'datm_cdeps_faster_intel' [08:13, 06:40]
PASS -- TEST 'datm_cdeps_control_cfsr_faster_intel' [04:19, 02:44](1162 MB)

PASS -- COMPILE 'datm_cdeps_land_intel' [03:12, 01:08] ( 1 remarks )
PASS -- TEST 'datm_cdeps_lnd_gswp3_intel' [02:31, 00:51](258 MB)
PASS -- TEST 'datm_cdeps_lnd_era5_intel' [02:26, 00:55](314 MB)
PASS -- TEST 'datm_cdeps_lnd_era5_rst_intel' [02:29, 00:32](314 MB)

PASS -- COMPILE 'datm_cdeps_lm4_intel' [03:12, 01:15]
PASS -- TEST 'datm_cdeps_lm4_gswp3_intel' [02:32, 00:35](564 MB)
PASS -- TEST 'datm_cdeps_lm4_gswp3_rst_intel' [02:39, 00:20](454 MB)

PASS -- COMPILE 'atm_ds2s_docn_pcice_intel' [11:12, 09:46] ( 1 remarks )
PASS -- TEST 'atm_ds2s_docn_pcice_intel' [06:29, 03:40](1965 MB)

PASS -- COMPILE 'atm_ds2s_docn_dice_intel' [11:13, 09:43] ( 1 remarks )
PASS -- TEST 'atm_ds2s_docn_dice_intel' [06:14, 03:35](1931 MB)

PASS -- COMPILE 'atml_intel' [11:13, 09:56] ( 8 warnings 2 remarks )
PASS -- TEST 'control_p8_atmlnd_sbs_intel' [07:13, 04:18](1826 MB)
PASS -- TEST 'control_p8_atmlnd_intel' [07:03, 04:10](1842 MB)
PASS -- TEST 'control_restart_p8_atmlnd_intel' [04:45, 02:17](1088 MB)

PASS -- COMPILE 'atml_debug_intel' [07:13, 05:17] ( 880 warnings 2 remarks )
PASS -- TEST 'control_p8_atmlnd_debug_intel' [07:55, 05:30](1844 MB)

PASS -- COMPILE 'atmw_intel' [12:13, 10:19] ( 9 remarks )
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'atmwav_control_noaero_p8_intel' [, ]( MB)

PASS -- COMPILE 'atmaero_intel' [11:13, 09:47] ( 1 remarks )
PASS -- TEST 'atmaero_control_p8_intel' [05:58, 04:08](3180 MB)
PASS -- TEST 'atmaero_control_p8_rad_intel' [07:03, 04:45](3065 MB)
PASS -- TEST 'atmaero_control_p8_rad_micro_intel' [06:47, 04:53](3087 MB)

PASS -- COMPILE 'atmaq_debug_intel' [06:14, 04:24] ( 882 warnings 6 remarks )

PASS -- COMPILE 'atm_gnu' [06:13, 04:38]
PASS -- TEST 'control_c48_gnu' [11:49, 09:35](1503 MB)
PASS -- TEST 'control_stochy_gnu' [05:26, 03:30](478 MB)
PASS -- TEST 'control_ras_gnu' [07:24, 04:52](494 MB)
PASS -- TEST 'control_p8_gnu' [07:58, 05:14](1452 MB)
PASS -- TEST 'control_p8_ugwpv1_gnu' [07:54, 05:11](1440 MB)
PASS -- TEST 'control_flake_gnu' [13:32, 10:39](537 MB)

PASS -- COMPILE 'rrfs_gnu' [06:13, 04:19]
PASS -- TEST 'rap_control_gnu' [13:33, 11:14](798 MB)
PASS -- TEST 'rap_decomp_gnu' [13:31, 11:29](832 MB)
PASS -- TEST 'rap_2threads_gnu' [12:39, 10:16](906 MB)
PASS -- TEST 'rap_restart_gnu' [07:38, 05:46](555 MB)
PASS -- TEST 'rap_sfcdiff_gnu' [13:38, 11:17](801 MB)
PASS -- TEST 'rap_sfcdiff_decomp_gnu' [13:34, 11:31](802 MB)
PASS -- TEST 'rap_sfcdiff_restart_gnu' [10:39, 08:18](561 MB)
PASS -- TEST 'hrrr_control_gnu' [07:39, 05:50](801 MB)
PASS -- TEST 'hrrr_control_noqr_gnu' [07:33, 05:50](783 MB)
PASS -- TEST 'hrrr_control_2threads_gnu' [07:35, 05:16](903 MB)
PASS -- TEST 'hrrr_control_decomp_gnu' [07:35, 05:47](830 MB)
PASS -- TEST 'hrrr_control_restart_gnu' [04:28, 03:00](539 MB)
PASS -- TEST 'hrrr_control_restart_noqr_gnu' [04:26, 02:59](634 MB)
PASS -- TEST 'rrfs_v1beta_gnu' [12:45, 11:00](798 MB)

PASS -- COMPILE 'atm_dyn32_debug_gnu' [08:13, 06:23]
PASS -- TEST 'control_diag_debug_gnu' [03:48, 01:32](1267 MB)
PASS -- TEST 'regional_debug_gnu' [12:36, 10:30](736 MB)
PASS -- TEST 'rap_control_debug_gnu' [04:24, 02:27](814 MB)
PASS -- TEST 'hrrr_control_debug_gnu' [04:24, 02:23](812 MB)
PASS -- TEST 'hrrr_gf_debug_gnu' [04:22, 02:29](821 MB)
PASS -- TEST 'hrrr_c3_debug_gnu' [04:22, 02:29](821 MB)
PASS -- TEST 'rap_diag_debug_gnu' [04:39, 02:41](898 MB)
PASS -- TEST 'rap_noah_sfcdiff_cires_ugwp_debug_gnu' [05:24, 03:48](815 MB)
PASS -- TEST 'rap_progcld_thompson_debug_gnu' [04:22, 02:34](816 MB)
PASS -- TEST 'rrfs_v1beta_debug_gnu' [04:22, 02:26](808 MB)
PASS -- TEST 'control_ras_debug_gnu' [03:23, 01:29](448 MB)
PASS -- TEST 'control_stochy_debug_gnu' [03:21, 01:36](446 MB)
PASS -- TEST 'control_debug_p8_gnu' [03:39, 01:33](1436 MB)
PASS -- TEST 'rap_flake_debug_gnu' [04:23, 02:31](845 MB)
PASS -- TEST 'rap_clm_lake_debug_gnu' [04:23, 02:42](818 MB)
PASS -- TEST 'gnv1_c96_no_nest_debug_gnu' [05:38, 04:05](822 MB)

PASS -- COMPILE 'wam_debug_gnu' [04:12, 02:44]

PASS -- COMPILE 'rrfs_dyn32_phy32_gnu' [05:12, 03:57]
PASS -- TEST 'rap_control_dyn32_phy32_gnu' [11:32, 09:43](693 MB)
PASS -- TEST 'hrrr_control_dyn32_phy32_gnu' [06:37, 05:04](694 MB)
PASS -- TEST 'rap_2threads_dyn32_phy32_gnu' [10:35, 09:00](738 MB)
PASS -- TEST 'hrrr_control_2threads_dyn32_phy32_gnu' [06:33, 04:47](737 MB)
PASS -- TEST 'hrrr_control_decomp_dyn32_phy32_gnu' [07:31, 05:18](689 MB)
PASS -- TEST 'rap_restart_dyn32_phy32_gnu' [09:40, 07:08](532 MB)
PASS -- TEST 'hrrr_control_restart_dyn32_phy32_gnu' [06:27, 02:37](519 MB)
PASS -- TEST 'conus13km_control_gnu' [05:58, 03:09](850 MB)
PASS -- TEST 'conus13km_2threads_gnu' [10:46, 06:11](851 MB)
PASS -- TEST 'conus13km_restart_mismatch_gnu' [05:44, 01:47](545 MB)

PASS -- COMPILE 'atm_dyn64_phy32_gnu' [11:12, 09:47]
PASS -- TEST 'rap_control_dyn64_phy32_gnu' [07:37, 05:51](719 MB)

PASS -- COMPILE 'atm_dyn32_phy32_debug_gnu' [08:12, 06:12]
PASS -- TEST 'rap_control_debug_dyn32_phy32_gnu' [04:27, 02:19](686 MB)
PASS -- TEST 'hrrr_control_debug_dyn32_phy32_gnu' [04:24, 02:19](678 MB)
PASS -- TEST 'conus13km_debug_gnu' [08:44, 06:11](847 MB)
PASS -- TEST 'conus13km_debug_qr_gnu' [08:42, 06:11](547 MB)
PASS -- TEST 'conus13km_debug_2threads_gnu' [10:40, 08:03](855 MB)
PASS -- TEST 'conus13km_radar_tten_debug_gnu' [08:39, 06:17](913 MB)

PASS -- COMPILE 'atm_dyn64_phy32_debug_gnu' [08:12, 06:21]
PASS -- TEST 'rap_control_dyn64_phy32_debug_gnu' [04:24, 02:27](706 MB)

PASS -- COMPILE 's2swa_gnu' [17:12, 15:45]

PASS -- COMPILE 's2s_gnu' [17:12, 15:29]
PASS -- TEST 'cpld_control_nowave_noaero_p8_gnu' [15:07, 12:20](1489 MB)

PASS -- COMPILE 's2swa_debug_gnu' [04:12, 03:04]

PASS -- COMPILE 's2sw_pdlib_gnu' [17:13, 15:22]
FAILED: UNABLE TO COMPLETE COMPARISON -- TEST 'cpld_control_pdlib_p8_gnu' [, ]( MB)

PASS -- COMPILE 's2sw_pdlib_debug_gnu' [04:13, 02:51]

PASS -- COMPILE 'datm_cdeps_gnu' [16:13, 15:04]
PASS -- TEST 'datm_cdeps_control_cfsr_gnu' [04:18, 03:00](691 MB)

PASS -- COMPILE 'atm_fbh_intel' [11:12, 09:12] ( 3 warnings 8 remarks )
PASS -- TEST 'cpld_regional_atm_fbh_intel' [13:29, 11:48](1062 MB)

SYNOPSIS:
Starting Date/Time: 20241023 15:21:53
Ending Date/Time: 20241023 17:37:01
Total Time: 02h:15m:27s
Compiles Completed: 59/59
Tests Completed: 215/246
Failed Tests:
* TEST cpld_control_p8_mixedmode_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_p8_mixedmode_intel.log
* TEST cpld_control_gfsv17_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_gfsv17_intel.log
* TEST cpld_control_gfsv17_iau_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_restart_gfsv17_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_mpi_gfsv17_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_mpi_gfsv17_intel.log
* TEST cpld_debug_gfsv17_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_debug_gfsv17_intel.log
* TEST cpld_control_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_p8_intel.log
* TEST cpld_control_p8.v2.sfc_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_p8.v2.sfc_intel.log
* TEST cpld_restart_p8_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_control_qr_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_qr_p8_intel.log
* TEST cpld_restart_qr_p8_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_2threads_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_2threads_p8_intel.log
* TEST cpld_decomp_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_decomp_p8_intel.log
* TEST cpld_mpi_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_mpi_p8_intel.log
* TEST cpld_control_ciceC_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_ciceC_p8_intel.log
* TEST cpld_control_c192_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_c192_p8_intel.log
* TEST cpld_restart_c192_p8_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_bmark_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_bmark_p8_intel.log
* TEST cpld_restart_bmark_p8_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_control_noaero_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_noaero_p8_intel.log
* TEST cpld_debug_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_debug_p8_intel.log
* TEST cpld_debug_noaero_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_debug_noaero_p8_intel.log
* TEST cpld_control_p8_faster_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_p8_faster_intel.log
* TEST cpld_control_pdlib_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_pdlib_p8_intel.log
* TEST cpld_restart_pdlib_p8_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_mpi_pdlib_p8_intel: FAILED: UNABLE TO START TEST
-- LOG: N/A
* TEST cpld_debug_pdlib_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_debug_pdlib_p8_intel.log
* TEST hafs_regional_atm_wav_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_hafs_regional_atm_wav_intel.log
* TEST hafs_regional_atm_ocn_wav_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_hafs_regional_atm_ocn_wav_intel.log
* TEST atmwav_control_noaero_p8_intel: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_atmwav_control_noaero_p8_intel.log
* TEST cpld_control_pdlib_p8_gnu: FAILED: UNABLE TO COMPLETE COMPARISON
-- LOG: /scratch1/NCEPDEV/climate/Matthew.Masarik/projs/code_mgmt/ufs/ufs-weather-model_003/tests/logs/log_hera/run_cpld_control_pdlib_p8_gnu.log

NOTES:
A file 'test_changes.list' was generated with list of all failed tests.
You can use './rt.sh -c -b test_changes.list' to create baselines for the failed tests.
If you are using this log as a pull request verification, please commit 'test_changes.list'.

Result: FAILURE

====END OF HERA REGRESSION TESTING LOG====