-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GRIB table updates #601
GRIB table updates #601
Conversation
NCEP GRIB2 tables still at v23.0.0
The last modified time for one of the GOES 16 objects stored in the Open Science Data Cloud changed and was causing a test to fail. The actual file represented in the blob and its associated data didn't change, so perhaps the OSDC bucket was migrated to a different system?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wow, @lesserwhirls this seems like it was really tedious 😬
Looks good to me, as long as the Python checks out with @dopplershift !
Reviewed 32 of 32 files at r1.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on @dopplershift)
Yeah, the python is a bit of a hot mess, but not too terribly so. @dopplershift - I modified what you had before to work with python 3, and to provide the option to interactively look at the differences between MRMS tables when there are differences between tables for a given |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The Python is complicated, but it's by no means an abomination.
I've made some comments to varying degrees from nitpicky to some good improvements. Feel free to disregard anything you don't feel like dealing with--the important thing is that it produces the file you want.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
selection -= 1 | ||
valid_selection = True | ||
|
||
if not valid_selection: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could just be the else
clause to the if
block right above, since the only way valid_selection
is set to True
is if you get in one of those. You could also make the whole while
block a while True:
and use break
to leave it and completely eliminate valid_selection
.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py
Outdated
Show resolved
Hide resolved
Backport from netCDF-Java 5 (Unidata/netcdf-java#601)
a5a7209
to
6a01d08
Compare
Thanks for the feedback @dopplershift! Should have those taken care of in the latest changes. |
Backport from netCDF-Java 5 (Unidata/netcdf-java#601)
6a01d08
to
917fa23
Compare
Backport from netCDF-Java 5 (Unidata/netcdf-java#601)
917fa23
to
ac06509
Compare
ac06509
to
b7f296c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 28 of 32 files reviewed, 11 unresolved discussions (waiting on @dopplershift, @haileyajohnson, and @lesserwhirls)
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 142 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
I think this should do the same:
for version in range(1, len(param_versions)): if not parameter_comparison(first, param_versions[version]): break else: # Didn't break unique_params.append(param_versions[-1]) continue dupes += 1 ...I'm not sure if that's actually clearer, but it eliminates the
has_diffs
flag.
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 164 at r1 (raw file):
Previously, lesserwhirls (Sean Arms) wrote…
sigh got it now, missed it before. Thanks for the catch!
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 179 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
This could just be the
else
clause to theif
block right above, since the only wayvalid_selection
is set toTrue
is if you get in one of those. You could also make the wholewhile
block awhile True:
and usebreak
to leave it and completely eliminatevalid_selection
.
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 206 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
pathlib
may make things easier here:from pathlib import Path tables = Path('tables/').iterdir()
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 211 at r1 (raw file):
Previously, lesserwhirls (Sean Arms) wrote…
I was trying to support these three cases:
convert_mrms_table.py --tables path/to/table1
convert_mrms_table.py --tables path/to/table1 --tables path/to/table2
convert_mrms_table.py --tables path/to/table1 path/to/table2
which required that
.add_argument
useaction='append', nargs='+'
. That resulted inargparse
creating alist
oflist
s. However, that's just overcomplicating things really, so I've simplified it by only supporting the case of multiple--table
flags (see update).
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 213 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
Usually no spaces around
=
here.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 28 of 32 files reviewed, 11 unresolved discussions (waiting on @dopplershift and @haileyajohnson)
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 3 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
Usually
from datetime import datetime
, but not a huge deal either.
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 7 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
exists
isn't being used
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 61 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
Could use f-strings and parentheses to replace with:
return (f'{param.Discipline}, {param.Category}, {param.Parameter}, "{param.Name}", ' f'"{param.Description}", "{param.Unit}", {param.No_Coverage}, ' f'{param.Missing}')That's purely stylistic though.
Done.
grib/src/main/resources/resources/grib2/mrms/convert_mrms_table.py, line 69 at r1 (raw file):
Previously, dopplershift (Ryan May) wrote…
Same thing here.
Done.
Updated the following GRIB tables:
Fixed the following new test failures:
Finally, bumped the version of the apache xmlbeans dependency.
The full test run against this PR is passing on Jenkins.
This change is