PR: Refactor NHDPlus HR preprocessing workflow

[BrianAvant]

Refactor NHDPlus HR preprocessing workflow. Resolves issue #238

Changes

• NHD streams, NWM catchments, and headwaters input layers are consolidated for MS and FR. HUC8 intersections are now included in the input headwaters layer and there are attributes that define the type of each point. Both streams and headwater points have a mainstem attribute.

• clip_vectors_to_wbd.py checks stream segments that intersect with the WBD boundary and removes incoming streams from the selected layers. This is to prevent them from being routed out of the HUC during the flow direction calculation.

• Some gaps/overlap in coverage exist at the confluences of HUC8s. This is not likely to be fixed. FIM4 may provide a solution to this issue if we process FIM on a reach scale instead of HUC8s.

Screenshots

FIM v3.0.15.5 MS HUC 03150107

FIM v3.0.15.8 MS HUC 03150107

Fixes MS deviation

Testing

Marginal improvements are observed at BLE and NWS sites whereas a slight regression is observed across all thresholds of the USGS sites.

Looking at metrics of individual USGS sites, it seems the regression in aggregate CSI score may be caused by two specific sites. These sites show a reduction in related to the adjustment in catchment size of the incoming stream segments.
For example, NWS LID IOWI4 in HUC 07080209 shows a ~0.2 drop in CSI across all thresholds:

Further inspections reveals that the changes in the new version result in more complete mapping of the upstream boundary.

So even thought there is a regression in USGS CSI scores, the subsetting method produces a better output from a hydrologic and visual perspective.

[TrevorGrout-NOAA]

When calculating aggregate scores across all ahps sites (nws & usgs) there are some small fluctuations but overall very similar scores:

	3_0_15_3	3_0_15_8
Action	0.450	0.447
Minor	0.483	0.483
Moderate	0.542	0.544
Major	0.603	0.602

It has been observed that odd, and often unrealistic mapping can occur when a stream segment doesn't traverse the entire catchment. FIM 3.0.15.2 exhibits this:

Note how the Agreement raster indicates overprediction for catchments that are fully traversed by the stream centerline. Note how there is underprediction for the upstream most basin.

This feature branch exhibits much more realistic catchments and mapping:

The overprediction noted in 3.0.15.2 on the downstream catchments is now evident throughout.

Despite this site have reduced CSI, I would argue the output is more realistic and consistent. Altering Manning "n" would likely produce more agreeable results.

[TrevorGrout-NOAA]

I was able to check out the feature branch (dev-local-headwaters) and successfully run HUC 03150107 and verified that the catchments appear as indicated. I also inspected IOWI4 and agree that this feature branch, although a reduction in CSI, produces more consistent and realistic results by changing how the catchments are made. Additionally, when viewing the aggregate CSI of all AHPS sites (NWS & USGS) they are only vary by up to 0.3% between 3.0.15.3 vs this feature branch.

[TrevorGrout-NOAA]

Brian, Can you please elaborate why this change was added? Was this an oversight from earlier that is also fixed?

[BrianAvant]

The changes in PR #358 make this code redundant.

[TrevorGrout-NOAA]

@BrianAvant-NOAA , please make sure this FB is up to date with dev prior to merging.