Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update reader.py to v0.3 #89

Merged
merged 32 commits into from
Aug 25, 2021
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
10df6d4
Update reader.py to v0.3
constantinpape Jun 11, 2021
0e5dc42
Merge remote-tracking branch 'ome/master' into patch-1
constantinpape Jun 14, 2021
36e37cf
Remove opencv dependency and use skimage.resize for nearest-neighbor …
constantinpape Jun 14, 2021
cfe03e3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 14, 2021
eac41be
Merge remote-tracking branch 'ome/master' into patch-1
constantinpape Jun 15, 2021
f28b051
Add FormatV03
constantinpape Jun 15, 2021
32aaa47
Merge branch 'patch-1' of https://github.com/constantinpape/ome-zarr-…
constantinpape Jun 15, 2021
49017d7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 15, 2021
1129149
Merge branch 'master' into patch-1
constantinpape Jul 7, 2021
5740ba1
Fix tests
constantinpape Jul 7, 2021
41a6c6a
Merge branch 'patch-1' of https://github.com/constantinpape/ome-zarr-…
constantinpape Jul 7, 2021
d86787f
Merge branch 'master' into patch-1
constantinpape Aug 10, 2021
e07194d
Add test for scaler
constantinpape Aug 10, 2021
e971fac
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 10, 2021
b986523
Update test writer
constantinpape Aug 10, 2021
46c8e6e
Merge branch 'patch-1' of https://github.com/constantinpape/ome-zarr-…
constantinpape Aug 10, 2021
221c0e4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 10, 2021
ebdeab2
Add test for ome_zarr info cli info with s3 address
constantinpape Aug 10, 2021
306d293
Merge branch 'patch-1' of https://github.com/constantinpape/ome-zarr-…
constantinpape Aug 10, 2021
ebc0ed9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 10, 2021
a6ea341
Fix black
constantinpape Aug 10, 2021
0ba85e7
Update ome_zarr/reader.py
constantinpape Aug 11, 2021
329c852
Fix typo in ZarrLocation
constantinpape Aug 11, 2021
5dfd973
Mark expected failures with xfail
constantinpape Aug 11, 2021
d9f994d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 11, 2021
0fc3318
Merge remote-tracking branch 'ome/master' into patch-1
constantinpape Aug 11, 2021
a60d040
Merge branch 'patch-1' of https://github.com/constantinpape/ome-zarr-…
constantinpape Aug 11, 2021
84d8a54
Cherry-pick isory fix: 561df25dc77e758864bab94470213c8cc8df8166
joshmoore Aug 12, 2021
79e53c0
Add v0.2 test data
joshmoore Aug 12, 2021
cf1d243
Remove xfail after dim separator fix (Close #104)
joshmoore Aug 25, 2021
741e952
Skip fsspec 2021.7.0
joshmoore Aug 25, 2021
93d0964
Use official upload of v0.3 data
joshmoore Aug 25, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .isort.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[settings]
known_third_party = cv2,dask,numpy,pytest,scipy,setuptools,skimage,vispy,zarr
known_third_party = dask,numpy,pytest,scipy,setuptools,skimage,vispy,zarr
multi_line_output=6
include_trailing_comma=False
force_grid_wrap=0
Expand Down
2 changes: 0 additions & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ dependencies:
- ipython
- mypy
- omero-py
- opencv
- pip
- py-opencv
- pytest
- requests
- s3fs
Expand Down
14 changes: 13 additions & 1 deletion ome_zarr/format.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ def format_implementations() -> Iterator["Format"]:
"""
Return an instance of each format implementation, newest to oldest.
"""
yield FormatV03()
yield FormatV02()
yield FormatV01()

Expand Down Expand Up @@ -124,4 +125,15 @@ def init_store(self, path: str, mode: str = "r") -> FSStore:
return store


CurrentFormat = FormatV02
class FormatV03(FormatV02): # inherits from V02 to avoid code duplication
"""
Changelog: variable number of dimensions (up to 5),
introduce axes field in multiscales (June 2021)
"""

@property
def version(self) -> str:
return "0.3"


CurrentFormat = FormatV03
12 changes: 9 additions & 3 deletions ome_zarr/reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -262,10 +262,17 @@ def matches(zarr: ZarrLocation) -> bool:
def __init__(self, node: Node) -> None:
super().__init__(node)

axes_values = {"t", "c", "z", "y", "x"}
try:
multiscales = self.lookup("multiscales", [])
version = multiscales[0].get("version", "0.1")
version = multiscales[0].get(
"version", "0.1"
) # should this be matched with Format.version?
datasets = multiscales[0]["datasets"]
# axes field was introduced in 0.3, before all data was 5d
axes = tuple(multiscales[0].get("axes", ["t", "c", "z", "y", "x"]))
if len(set(axes) - axes_values) > 0:
raise RuntimeError(f"Invalid axes names: {set(axes) - axes_values}")
datasets = [d["path"] for d in datasets]
self.datasets: List[str] = datasets
LOGGER.info("datasets %s", datasets)
Expand All @@ -274,14 +281,13 @@ def __init__(self, node: Node) -> None:
return # EARLY EXIT

for resolution in self.datasets:
# data.shape is (t, c, z, y, x) by convention
data: da.core.Array = self.array(resolution, version)
chunk_sizes = [
str(c[0]) + (" (+ %s)" % c[-1] if c[-1] != c[0] else "")
for c in data.chunks
]
LOGGER.info("resolution: %s", resolution)
LOGGER.info(" - shape (t, c, z, y, x) = %s", data.shape)
LOGGER.info(" - shape %s = %s", axes, data.shape)
LOGGER.info(" - chunks = %s", chunk_sizes)
LOGGER.info(" - dtype = %s", data.dtype)
node.data.append(data)
Expand Down
20 changes: 13 additions & 7 deletions ome_zarr/scale.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,15 @@
from dataclasses import dataclass
from typing import Callable, Iterator, List

import cv2
import numpy as np
import zarr
from scipy.ndimage import zoom
from skimage.transform import downscale_local_mean, pyramid_gaussian, pyramid_laplacian
from skimage.transform import (
downscale_local_mean,
pyramid_gaussian,
pyramid_laplacian,
resize,
)

from .io import parse_url

Expand Down Expand Up @@ -123,19 +127,21 @@ def __create_group(

def nearest(self, base: np.ndarray) -> List[np.ndarray]:
"""
Downsample using :func:`cv2.resize`.
Downsample using :func:`skimage.transform.resize`.

The :const:`cvs2.INTER_NEAREST` interpolation method is used.
"""
return self._by_plane(base, self.__nearest)

def __nearest(self, plane: np.ndarray, sizeY: int, sizeX: int) -> np.ndarray:
"""Apply the 2-dimensional transformation."""
return cv2.resize(
return resize(
plane,
dsize=(sizeY // self.downscale, sizeX // self.downscale),
interpolation=cv2.INTER_NEAREST,
)
output_shape=(sizeY // self.downscale, sizeX // self.downscale),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@will-moore : can you check that this matches your cv2 use cases?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will double check this and see if we can cover it by some test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added tests for the scaler in e07194d. Judging from the results the current implementation is correct.

order=0,
preserve_range=True,
anti_aliasing=False,
).astype(plane.dtype)

def gaussian(self, base: np.ndarray) -> List[np.ndarray]:
"""Downsample using :func:`skimage.transform.pyramid_gaussian`."""
Expand Down
1 change: 0 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ def read(fname):
install_requires += (["scikit-image"],)
install_requires += (["toolz"],)
install_requires += (["vispy"],)
install_requires += (["opencv-contrib-python-headless"],)


setup(
Expand Down
2 changes: 1 addition & 1 deletion tests/test_ome_zarr.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

def log_strings(idx, t, c, z, y, x, ct, cc, cz, cy, cx, dtype):
yield f"resolution: {idx}"
yield f" - shape (t, c, z, y, x) = ({t}, {c}, {z}, {y}, {x})"
yield f" - shape ('t', 'c', 'z', 'y', 'x') = ({t}, {c}, {z}, {y}, {x})"
yield f" - chunks = ['{ct}', '{cc}', '{cz}', '{cx}', '{cy}']"
yield f" - dtype = {dtype}"

Expand Down