Fix dask2 proxies #190
Fix dask2 proxies #190
Conversation
iris_grib/__init__.py
Outdated
| @@ -25,6 +25,73 @@ | |||
| import iris.coord_systems as coord_systems | |||
| from iris.exceptions import TranslationError, NotYetImplementedError | |||
|
|
|||
| try: | |||
| from iris.util import _array_slice_ifempty | |||
There was a problem hiding this comment.
I'm pretty uncomfortable about this using a private function from iris.
I don't have a better suggestion, but I do think we seriously need to consider how we will manage this sort of thing, as we will encounter this problem more and more as we try to break iris up
There was a problem hiding this comment.
🤔 I really see what you mean. I think this is a big problem for the future.
Some thoughts ...
-
We may want to share common functionality with multiple dependent projects , but retain some ability to change interface and/or behaviour without it forcing a major version increment.
-
For example, I can imagine we might want common code for things like : a file-loader plugin class; a common DataProxy concept; DataManager objects;, 'lazy' data functions. (all of which currently exist as private apis within Iris)
-
The key problem is : At times we will want to change such things, in ways which would force updates in the dependent projects. But, we don't want this to have the same implications as breaking changes in user-api, and it certainly isn't appropriate to apply SemVer rules in the same way.
-
The key driver of this is that projects like iris-grib (e.g. file format interfaces) want to share common functions, but these may be lower-level and not user-relevant. It is a flexibility/stability tradeoff. There is no benefit in making all shared function rigidly defined + controlled in the same way as user API : it needs to be more flexible than that.
-
we could put such functions in their own independent projects (not just in Iris), but I don't think that actually adds anything useful.
I think it's also worth noting that, in the specific case of this function, not only is it too low-level, without user-level relevance, to publish it as a controlled API, but also it is highly specific to what Dask is doing, and that might even change, which makes it a bit "ad hoc".
There was a problem hiding this comment.
I think this is pretty much ready to go.
I don't have any other comments than what we discussed about:
- removing the try/except, and only have the import of
_array_slice_ifemptyfrom iris - update the environment.yml with the iris >= 2.4 dependency
But these depend on the Iris release
|
Thanks @lbdreyer
I think it's close. Let's wait ... |
Latest does that. |
Done + ready ! |
pp-mo
force-pushed
the
fix_proxies_dask2
branch
from
408d4da to
e23229f
Compare
|
Had to rebase. |
Change that parallels SciTools/iris#3659
NOTE:
This contains a temporary copy of the key Iris routine that enables this fix.
This can be removed when Iris 2.4 is out.
Not clear if we should merge this as-is, or wait for that ?