-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MOBT-797: Coordinate retention in percentile generation #2087
Conversation
…ntile generation from coordinate collapse. This change also ensures that any scalar time coordinates that are created in this way adhere to the improver metadata standards.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2087 +/- ##
==========================================
+ Coverage 98.39% 98.40% +0.01%
==========================================
Files 124 134 +10
Lines 12212 13254 +1042
==========================================
+ Hits 12016 13043 +1027
- Misses 196 211 +15 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested this with a few coordinates to collapse and retain. It works as intended but I found that the bounds/points it puts on non-time related coordinates aren't always as useful. That's nothing to do with this PR though so happy to approve.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @bayliffe, I've added a question about whether the CLI change requires an acceptance test to go along with it.
* Enable retention of scalar coordinates that are produced during percentile generation from coordinate collapse. This change also ensures that any scalar time coordinates that are created in this way adhere to the improver metadata standards. * style fixes.
* MOBT-797: Coordinate retention in percentile generation (#2087) * Enable retention of scalar coordinates that are produced during percentile generation from coordinate collapse. This change also ensures that any scalar time coordinates that are created in this way adhere to the improver metadata standards. * style fixes. * Setting percentiles with as_iterable --------- Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk>
* MOBT-797: Coordinate retention in percentile generation (#2087) * Enable retention of scalar coordinates that are produced during percentile generation from coordinate collapse. This change also ensures that any scalar time coordinates that are created in this way adhere to the improver metadata standards. * style fixes. * Setting percentiles with as_iterable --------- Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk>
…ad_time using it (#2088) * radius_by_lead_time improved input handling * Make is_iterable changes applicable to percentiles option (#2090) * MOBT-797: Coordinate retention in percentile generation (#2087) * Enable retention of scalar coordinates that are produced during percentile generation from coordinate collapse. This change also ensures that any scalar time coordinates that are created in this way adhere to the improver metadata standards. * style fixes. * Setting percentiles with as_iterable --------- Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> * TEDST: is_iterable * Update improver/nbhood/__init__.py Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk> * Update __init__.py * Update nbhood.py --------- Co-authored-by: Phil <phil.relton@metoffice.gov.uk> Co-authored-by: bayliffe <benjamin.ayliffe@metoffice.gov.uk> Co-authored-by: Stephen Moseley <stephen.moseley@metoffice.gov.uk>
This PR enables the retention of scalar coordinates that are generated when collapsing coordinates to generate percentiles. It also ensures that if these retained coordinates are time related (time, forecast_period, forecast_reference_time), they are modified to ensure that the point is aligned to the upper bound, adhering with IMPROVER's metadata standards.
The purpose of this change is to enable the production of typical wind speed diagnostics with our existing tooling. The suite chain will become:
This change ensures that the time period covered by the diagnostic is available on the final diagnostic. Note that we will recycle the midnight validity time for adjacent days, e.g.
Day 1: 20250115T0000Z, 03, 06, 09, 12, 15, 18, 21, 20250116T0000Z
Day 2: 20250116T0000Z, 03, 06, 09, 12, 15, 18, 21, 20250117T0000Z
The wind speed diagnostics do no cover a period, so we use as many samples as we can for a given day. This means that, in the example shown here the 20250116T0000Z forecast is included in the forecast for day 1 and day 2. The generate_percentiles CLI will create bounds from the time points (as there are no bounds here), which results in bounds that span the whole 24-hours on the output cube.
Testing: