-
-
Notifications
You must be signed in to change notification settings - Fork 316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ABSStore ImportError #618
Comments
Should we consider deprecating ABSStore in favor of https://github.com/dask/adlfs, which has matured a lot recently? That would bring our treatment of ABS more in line with the other cloud object stores. |
@jhamman, the BlockBlobService is no longer available in recent versions of the Azure storage module. BlobServiceClient replaces this in azure.storage.blob, and transitioning to this new approach in ABSStore should be relatively straightforward but it's not clear how easy it would be to maintain backward compatibility. The fsspec mapper interface does work and it may be the best thing to rely on going forward. |
@rabernat, if adlfs, despite the name, is able to access Azure Blob storage, then this might also be a viable way forward. Data Lake is not Blob, but there is some indication in the adlfs documentation that it can access Blob. |
|
FWIW, I haven't been able to get from adlfs import AzureBlobFileSystem
path = 'carbonplan-data/raw/terraclimate/4000m/raster.zarr'
fs = AzureBlobFileSystem(
account_name="carbonplan",
account_key=os.environ["BLOB_ACCOUNT_KEY"]
)
mapper = fs.get_mapper(path)
xr.open_zarr(mapper, consolidated=True) # hangs!!! While I can get a mapper object, accessing keys from the mapper ends up hanging indefinitely. Also, since I know you're wondering, read-only access over http is working just fine for these stores. |
@jhamman as a stop-gap, if you want to use ABSStore you can install a pre-12 (2.1) version of azure-storage-blob:
Updating ABSStore to work with v12 versions of the azure-storage SDK would likely be just a few lines of code. Maintaining backward compatibility would be more difficult but almost certainly doable. I would be happy to continue supporting ABSStore if other Zarr devs support this approach, but I am also happy for us to move in a different direction if that's what we would like to do. This would be a great time and maybe even a great place to have this discussion. |
I'm finding that current versions of
azure-storage-blob
(12.5.0) are not importable when using theABSStore
. This was actually warned in this comment: #528 (comment). Is anyone using this store anymore? Presumably, fsspec's mapper interface should workMinimal, reproducible code sample, a copy-pastable example if possible
Problem description
Simply creating the
ABSStore
raises anImportError
:ABSStore is unusable in its current state.
Version and installation information
zarr.__version__
: '2.4.0'numcodecs.__version__
: '0.7.2'conda env export
:cc @tjcrone
The text was updated successfully, but these errors were encountered: