Skip to content

Alternative to np.concatenate given iterator of arrays, dtype, and final shape #93

@flexatone

Description

@flexatone

In all of SF's usage of np.concatenate, the following pattern is used

        array = np.empty((rows, columns), dtype=dtype)
        np.concatenate(blocks_norm, axis=1, out=array)
        array.flags.writeable = False

As np.concatenate does not support generators, and has additional overhead for more general cases, an optimized version might be implemented in ArrayKit.

NumPy's PyArray_AssignArray seems to be pretty close to what is needed:

https://github.com/numpy/numpy/blob/891ab8ee00755a539bd118b1bbe4e0a237a2d844/numpy/core/src/multiarray/array_assign_array.c#L295

But given that the destination is guaranteed to be a newly created, contiguous array, there might be a more efficient route.

We might be able to use this lower level interface directly:

https://github.com/numpy/numpy/blob/891ab8ee00755a539bd118b1bbe4e0a237a2d844/numpy/core/src/multiarray/array_assign_array.c#L80

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions