[5pts] Create tools and code adjustments to assist in alpha testing analysis of GMS outputs.

[RobHanna-NOAA]

During large scale testing of GMS for alpha testing (synthesize_test_cases.py) was attempted, some challenges were discovered.

When the full GMS BED was run a few months back, due to output sizes and processing times, ~2,180 HUCs were processed in sets of 100 across two servers. By default, synthesize_test_cases.py is looking for a centralized single output folder to do alpha testing. Further, they needed to be processed with a valid test_cases folders in a specific relational path. Due to overall total size of GMS outputs, which was ~ 3.9 TB, it was very difficult, and some times impossible, to put them into a central location. Quick pre-tests also strongly suggested that processing of all ~2,180 GMS folders resulted in very slow alpha testing processing.

Tools and processes need to be created that can extract only HUCs that have valid test data then alpha test process just related HUC (unit and branch) folders that are applicable. Creating a list of alpha test applicable HUCs is needed, then use that list to extract correct GMS processed HUC folders. synthesize_test_cases.py and its child python files rely on a key file in a key location called gms_inputs.csv to exist and match folders in it's directory. Without a valid gms_inputs.csv which matches only applicable alpha tested common folder HUCs (units and branches), processing was slow with thousands of errors. This is due the total of ~ 2,180 total HUC files and hundreds of thousands of branch files. Only 216 HUCs are valid for alpha testing.

All January GMS folders are on production servers and not in locations where most staff can access them. Large volumes and processing times result in it being best to process in production servers instead of dev machines which were often previously used.

Tools, enhanced command line outputs / tracking and new processes are required to fully process all alpha testing applicable valid GMS folders / files.