Description
Let's focus on Neo as this will cover a lot of ground and then we can discuss the rest as edge cases / enchancements. For neo segments the traces look like this:
def get_traces(
self,
start_frame: Union[int, None] = None,
end_frame: Union[int, None] = None,
channel_indices: Union[List, None] = None,
) -> np.ndarray:
raw_traces = self.neo_reader.get_analogsignal_chunk(
block_index=self.block_index,
seg_index=self.segment_index,
i_start=start_frame,
i_stop=end_frame,
stream_index=self.stream_index,
channel_indexes=channel_indices,
)
return raw_traces
I think we should follow numpy ufuncs API and add and extra argument called out where we can pass a previously allocated memory array where the traces will be written. Example in numpy:
import numpy as np
# Pre-allocate an array of the desired shape
# Generate some data
data = np.array([1, 2, 3])
# Use numpy.square to fill 'out'
result = np.square(data, out=data)
np.shares_memory(data, result)Reasons to do this:
- for our (power) users: we will give them more control over their memory allocations outsourcing optimization tricks to their side.
- Within our own code: this will help us avoid situations like the following where we have to sacrifice sequential readiability to avoid an undesirable memory allocation.
I think this makes the API more powerful and we already have numpy doing the work of providing a standard way of doing stuff. Most our users should be familiar with numpy and we can leverage on that for discoverability / usability.
As for implementation, for base recording get_traces is a read operation that ideally should return a memmap object or something of the like. With the out parameter we just make an assignation. A minimal check of shape can be added if we desire to add meaningul assertions for example.
What do you think guys? Are there downsides of this that I am not seeing?