Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot pickle cubes with GRIB data #202

Open
pp-mo opened this issue Feb 24, 2020 · 0 comments
Open

Cannot pickle cubes with GRIB data #202

pp-mo opened this issue Feb 24, 2020 · 0 comments
Labels
T-Shirt: Medium Type: Bug Type: Technical Debt known issues left over from incomplete work

Comments

@pp-mo
Copy link
Member

pp-mo commented Feb 24, 2020

(#201 moves the grib-specific part of the iris 'test-pickle' from iris into iris-grib.
In principle, a cube should be picklable, but at present this doesn't work with grib data.
In fact a GribMessage itself cannot be successfully pickled + brought back.
So that needs fixing.
)

Actually, I think this issue was fixed long ago and obsolete
But now I think it is back ! See comment

Update again

I was wrong, this problem never went away.
Regardless, I do think this is still an open problem.

As posted elsewhere ...

Isn't the whole point of our interest in pickling that it is essential to the use of dask.distributed ?

In which case...

  • the fact that you can't successfully reconstruct a pickled cube derived from a grib load is still very much a problem.
  • No idea how important though ... or what it takes to fix ... but I believe this has proved awkward in the past
  • possibly something to do with the way ecCodes (and gribapi before it) use "handles" for messages ?
    • -- so presumably needs (at least) a getstate/setstate solution -- or a fix, since DataProxy at least already has one
  • possibly just needs a smarter definition of 'equality' to fix it

HINT:

I think latest dask problems seen in Iris have highlighted problems specifically with "__slots__",
so that might affect DataProxy ??
Note that _DataProxy doesn't seem fit the bill for that error message -- but possibly _DataProxy.recreate_raw, which is a _MessageLocation does ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-Shirt: Medium Type: Bug Type: Technical Debt known issues left over from incomplete work
Projects
Status: No status
Development

No branches or pull requests

1 participant