Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StructBuilder::append_null should recursively append_null to children #7192

Open
the80srobot opened this issue Feb 25, 2025 · 0 comments
Open

StructBuilder::append_null should recursively append_null to children #7192

the80srobot opened this issue Feb 25, 2025 · 0 comments
Labels
enhancement Any new improvement worthy of a entry in the changelog

Comments

@the80srobot
Copy link

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

At the moment, it's impossible to generically append_null to a StructBuilder, because each nested column must also be appended to. Going through field_builder is not an option, because it requires knowing the type of the field ahead of time.

Describe the solution you'd like

Calling append_null on a StructBuilder should probably always append_null to all children with exactly n-1 slots. Alternatively, we could have a second function, called append_nulls or something similar.

Describe alternatives you've considered

If there was a way to generically iterate over the field builders, callers could implement it themselves, using a match on the type of the builder.
@the80srobot the80srobot added the enhancement Any new improvement worthy of a entry in the changelog label Feb 25, 2025
@the80srobot the80srobot changed the title StructBuilder::apend_null should recursively append_null to children StructBuilder::append_null should recursively append_null to children Feb 25, 2025
the80srobot added a commit to the80srobot/pedro that referenced this issue Feb 25, 2025
@the80srobot
rednose: work around some Arrow bugs
3132aaf 
This works around the fact that `append_null` is
non-recursive in Rust (even though it is in C++?)
by providing a recursive append_null. (See
apache/arrow-rs#7192).

Unfortunately, providing a recursive append_null
requires codegen, because of
apache/arrow-rs#7193.

Finally, it seems that "item" fields in Lists must
always be nullable, because Arrow will
occasionally make them so in a copy of the schema
it uses for validation. (I don't yet have a
minimal repro for this bug, so I haven't filed it
yet.)
the80srobot added a commit to the80srobot/pedro that referenced this issue Feb 25, 2025
@the80srobot
rednose: work around some Arrow bugs
29ca925 
This works around the fact that `append_null` is
non-recursive in Rust (even though it is in C++?)
by providing a recursive append_null. (See
apache/arrow-rs#7192).

Unfortunately, providing a recursive append_null
requires codegen, because of
apache/arrow-rs#7193.

Finally, it seems that "item" fields in Lists must
always be nullable, because Arrow will
occasionally make them so in a copy of the schema
it uses for validation. (I don't yet have a
minimal repro for this bug, so I haven't filed it
yet.)
the80srobot added a commit to the80srobot/pedro that referenced this issue Feb 25, 2025
@the80srobot
rednose: work around some Arrow bugs
80bebbd 
This works around the fact that `append_null` is
non-recursive in Rust (even though it is in C++?)
by providing a recursive append_null. (See
apache/arrow-rs#7192).

Unfortunately, providing a recursive append_null
requires codegen, because of
apache/arrow-rs#7193.

Finally, it seems that "item" fields in Lists must
always be nullable, because Arrow will
occasionally make them so in a copy of the schema
it uses for validation. (I don't yet have a
minimal repro for this bug, so I haven't filed it
yet.)
the80srobot added a commit to the80srobot/pedro that referenced this issue Feb 25, 2025
@the80srobot
rednose: work around some Arrow bugs
6e94aaa 
This works around the fact that `append_null` is
non-recursive in Rust (even though it is in C++?)
by providing a recursive append_null. (See
apache/arrow-rs#7192).

Unfortunately, providing a recursive append_null
requires codegen, because of
apache/arrow-rs#7193.

Finally, it seems that "item" fields in Lists must
always be nullable, because Arrow will
occasionally make them so in a copy of the schema
it uses for validation. (I don't yet have a
minimal repro for this bug, so I haven't filed it
yet.)
the80srobot added a commit to wowsignal-io/pedro that referenced this issue Feb 26, 2025
@the80srobot
rednose: work around some Arrow bugs
830a975 
This works around the fact that `append_null` is
non-recursive in Rust (even though it is in C++?)
by providing a recursive append_null. (See
apache/arrow-rs#7192).

Unfortunately, providing a recursive append_null
requires codegen, because of
apache/arrow-rs#7193.

Finally, it seems that "item" fields in Lists must
always be nullable, because Arrow will
occasionally make them so in a copy of the schema
it uses for validation. (I don't yet have a
minimal repro for this bug, so I haven't filed it
yet.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@the80srobot