Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1pt] PR: Added features to aggregate_outputs.py #296

Merged
merged 23 commits into from
Mar 11, 2021

Conversation

BrianAvant
Copy link

Adding features to aggregate_outputs.py to resolve issue #295 and issue #294

Additions

  • reprojecting grids to Web Mercator
  • parallelize HUC6 aggregation
  • changing grid blocksize from 256 to 1024 for faster postprocessing

Note

This feature branch is derived from dev-catfim-workflow so it needs to be merged in after PR #292

@BrianAvant BrianAvant added the enhancement New feature or request label Mar 4, 2021
@BrianAvant BrianAvant added this to the FIM v3.X milestone Mar 4, 2021
@BrianAvant BrianAvant changed the title PR: Added features to aggregate_outputs.py [1pt] PR: Added features to aggregate_outputs.py Mar 4, 2021
@BrianAvant BrianAvant self-assigned this Mar 4, 2021
@BradfordBates-NOAA BradfordBates-NOAA requested review from BradfordBates-NOAA and removed request for nickchadwick-noaa March 9, 2021 15:37
Base automatically changed from dev-catfim-workflow to dev March 9, 2021 16:30
@BradfordBates-NOAA
Copy link
Member

@BrianAvant-NOAA It looks like this branch has conflicts with dev. Could you please merge and then I'll review

@BradfordBates-NOAA
Copy link
Member

After the last patch, this branch is behind dev. Can you please merge again

@BrianAvant
Copy link
Author

merged

@BradfordBates-NOAA
Copy link
Member

@BrianAvant-NOAA, a couple questions:

  • Is there any change in usage in fim_run.sh? Any extra args?
  • Is there any change to the metrics after projecting grids to Web Mercator?

@BrianAvant
Copy link
Author

@BradfordBates-NOAA

  • you can check the changes in the Files changed section; I set the jobs arg for aggregate_fim_outputs.py to 4 with no user access because we need to optimize on the API. We can do that later.
  • metrics are run on the HUC8s currently. Web Mercator is used to reproject the HUC6s Viz use.

@BradfordBates-NOAA BradfordBates-NOAA merged commit 4364dc4 into dev Mar 11, 2021
@BradfordBates-NOAA BradfordBates-NOAA deleted the dev-agg-blocksize branch March 11, 2021 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants