Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: low-level buffer allocation utilities for new storages #1303

Merged
merged 34 commits into from
Aug 9, 2023

Conversation

egparedes
Copy link
Contributor

@egparedes egparedes commented Jul 27, 2023

Description

Add new buffer allocation utilities and concepts definitions.
Main changes:

  • New definition of core concepts dealing with data: DType, Device, ScalarTypes
  • New BullerAllocator protocol to support different buffer allocation strategies depending on the device type.
  • Refactoring of existing cartesian storages allocation tools using the new buffer allocators

Pending

  • Make current unit tests pass
  • Add unit tests for new buffer allocation and core definitions
  • Fix typing with python 3.8 compatibility and code style errors

@egparedes egparedes changed the title Storages next feat: new low-level buffer allocation utilities for next generation of storages Jul 27, 2023
@egparedes egparedes changed the title feat: new low-level buffer allocation utilities for next generation of storages feat: low-level buffer allocation utilities for new storages Jul 27, 2023
Comment on lines 428 to 431
NDArrayObject = Union[npt.NDArray, "CuPyNDArray", "JaxNDArray", NDArrayObjectProto]
NDArrayObjectT = TypeVar(
"NDArrayObjectT", npt.NDArray, "CuPyNDArray", "JaxNDArray", NDArrayObjectProto
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this if mypy works for

tmp: NDArrayObjectProto = np.asarray([])

@havogt havogt marked this pull request as ready for review August 4, 2023 17:10
@havogt
Copy link
Contributor

havogt commented Aug 4, 2023

cscs-ci run

@havogt
Copy link
Contributor

havogt commented Aug 4, 2023

cscs-ci run

1 similar comment
@havogt
Copy link
Contributor

havogt commented Aug 4, 2023

cscs-ci run

@havogt
Copy link
Contributor

havogt commented Aug 5, 2023

cscs-ci run

3 similar comments
@havogt
Copy link
Contributor

havogt commented Aug 5, 2023

cscs-ci run

@havogt
Copy link
Contributor

havogt commented Aug 5, 2023

cscs-ci run

@havogt
Copy link
Contributor

havogt commented Aug 5, 2023

cscs-ci run

Comment on lines 110 to 111
class UnsignedIntegral(numbers.Integral):
"""Abstract base class for unsigned integral types."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
class UnsignedIntegral(numbers.Integral):
"""Abstract base class for unsigned integral types."""
class PositiveIntegral(numbers.Integral):
"""Abstract base class representing positive integral numbers."""

Comment on lines 54 to 71
# class _NDArrayLike(Protocol):
# def __array__(self, dtype: Optional[np.dtype] = None) -> np.ndarray:
# ...

# def __getitem__(self, item: core_defs.SliceLike) -> np.ndarray:
# ...


# class _NDArrayLikeArrayInterface(xtyping.ArrayInterface, _NDArrayLike):
# ...


# class _NDArrayLikeCUDAArrayInterface(xtyping.CUDAArrayInterface, _NDArrayLike):
# ...


# class _NDArrayLikeDLPackBuffer(xtyping.DLPackBuffer, _NDArrayLike):
# ...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

havogt and others added 2 commits August 8, 2023 08:27
Co-authored-by: Stefano Ubbiali <subbiali@phys.ethz.ch>
@FlorianDeconinck
Copy link
Contributor

Can confirm that gt4py.storage.cartesian.dace_descriptor is not use in Pace.

@havogt havogt merged commit 054e4e6 into GridTools:main Aug 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants