Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow zarr writer to do incremental writes of single timepoints #91

Merged
merged 1 commit into from
Feb 11, 2025

Conversation

toloudis
Copy link
Contributor

@toloudis toloudis commented Feb 8, 2025

Link to Relevant Issue

Chantelle had a use case where she wanted to write out a time series zarr but could only prepare the source array sequentially, one timestep at a time. She wanted to use the writer to write sub-arrays incrementally.

Description of Changes

So I am adding a toffset parameter to the zarr writer. The writer will then attempt to write the given time series starting at the given t offset.

In this way you can call write_t_batches_from_array more than once, in a loop, passing in smaller amounts of t at a time.

A unit test is added to show how this should be used. You can just call write_t_batches_array in a loop now.

A future generalization of this could be to pass in even smaller parts of the array and give the writer a tczyx offset of where to start writing the input data.

Copy link
Contributor

@evamaxfield evamaxfield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!! Only comment would be that it might be nice to have an example of this in the top level writer docstring

@toloudis toloudis merged commit 3411983 into main Feb 11, 2025
20 checks passed
@toloudis toloudis deleted the zarr-writer-single-time branch February 11, 2025 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants