Efficiently map array to a (possibly) different type

[benkay86]

ArrayBase::mapv_into() consumes self and reuses its memory allocation to compute a new array with the same shape and type. Avoiding unnecessary allocations is important when mapping very large arrays.

When writing generic methods, it is often necessary to assume that two arrays may be of different types even though we might expect them to be the same type most of the time. For example, a method that is generic over complex numbers may be called with all real arguments, all complex arguments, or a mixture of real and complex arguments. This precludes direct use of mapv_into() since we cannot assume the arguments are the same type.

Would it be worth adding something like this trans_mapv_into(), which maps an array with elements of one type to a (possibly) different type? If the types are the same it delegates to mapv_into(), and if the types are different it falls back to using additional allocations. Type comparison is performed at runtime pending specialization.

[bluss]

So for this use case, the element type is something with non-trivial Clone or not? Or is it just a Copy element type?

[benkay86]

For my use case trans_mapv_into() is meant to be a drop-in replacement for mapv_into() with possibly worse performance if the types are different. The trait bounds are as close as possible to mapv_into(), which means the mapped type is Clone just like in mapv_into():

pub fn mapv_into<F>(mut self, f: F) -> Self
    where
        S: DataMut,
        F: FnMut(A) -> A,
        A: Clone,
        // ...

[benkay86]

@bluss, it's been a few days, so just checking again to see if trans_mapv_into() or something like it fits into your vision of what ndarray should offer. If so, I would welcome feedback on what the interface (e.g. Clone vs Copy) and implementation (whether to use unsafe or rely on the optimizer for efficient downcasting) should look like prior to opening a pull request.

If this just isn't a fit that's totally fine by me! I can focus on integrating this functionality into my own code or a separate, helper crate.

[bluss]

I'm not currently working on ndarray, so days are currently a too short time scale 🙂 . Last months I did a lot of work on it. The idea makes sense, but I'd probably tweak the signature and name if I had a go at it. About your current implementation I must point out that into_raw_vec is unfortunately hard to use correctly, and it has an error even here (for arrays that are sliced in place).

Fortunately ndarray has just added an actual .into_iter().

[benkay86]

@bluss, fair enough! As the most active committer I thought maybe you were the official maintainer of ndarray 😅. I guess I'll turn this into a pull request and solicit feedback that way.

I'm not sure that I see the error in into_raw_vec() -- I thought it was always safe to use if the array is in standard memory layout? But I am definitely rejoicing the introduction of into_iter()!

[bluss]

Well you're not wrong, I wrote most of ndarray and am one of the maintainers