Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zero-copy deserialisation #31

Open
clbarnes opened this issue Jan 20, 2024 · 1 comment
Open

Zero-copy deserialisation #31

clbarnes opened this issue Jan 20, 2024 · 1 comment

Comments

@clbarnes
Copy link
Owner

clbarnes commented Jan 20, 2024

Newly-default spatial tree library kiddo supports zero-copy serialisation/ deserialisation with rkyv, which means that the structs which are practically thin wrappers around it (i.e. this whole crate) should be able to as well without too much hassle. This would allow for larger-than-RAM arenas, or at least extremely efficient caching. Currently CATMAID uses .rda objects to cache resampled neurons between runs; this should be much better.

Currently, kiddo's neuron objects are much bigger than necessary because the points are duplicated, but that will not be necessary once sdd/kiddo#135 is merged.

@clbarnes
Copy link
Owner Author

clbarnes commented Jan 26, 2024

At present, just recreating the tree from points, tangents, alphas is much faster than deserialising from CBOR, at least [1]. That isn't zero-copy, but it's not the copying which is the slow bit here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant