Skip to content

Switch to append_columns method for adding records #41

@ashander

Description

@ashander

per discussion elsewhere, it's much faster but only if you avoid lists. as @molpopgen summarized:

passing lists to append_columns is a performance killer, too. One solution for ftprime will be to use numpy arrays (with the correct dtypes, which I can write down in the README for my project when I get a moment...) and act like you're in C:

x = np.zeroes([INITIAL_SIZE], dtype = the_right_type)
x_size = 0 #dummy variable

# fill X with new data, and x_size += 1 each time

When appropriate, send the slice x[:x_size] to the correct version of append_columns.

When filling, if x_size will become len(x) after adding data, create a new array that's the concatenation of x and a new empty array of size INITIAL_SIZE.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions