Skip to content

Conversation

@PyDataBlog
Copy link
Owner

@PyDataBlog PyDataBlog commented Jun 29, 2025

Description

A complete (and breaking) refactor of API and internal data structures which allows for native parallelism.

Fixes #8

Type of change
Switching to string interning and the adoption of atomic referencing allows enables true parallel searches. Initial benchmarks suggest a significant improvement (2-6x faster) in search speeds as well as database indexing compared to old API and other implementations.

Please choose the type of change that best describes your PR.

  • feat!(api): Switch to string interning and atomic reference to avoid unnecessary allocations.
  • perf(api): native parallel searches

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

  • test_measures
  • test_database
  • test_features
  • test_search

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added an entry to the CHANGELOG.md file
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@PyDataBlog PyDataBlog self-assigned this Jun 29, 2025
@PyDataBlog PyDataBlog added the enhancement New feature or request label Jun 29, 2025
@PyDataBlog PyDataBlog merged commit f10972f into main Jun 30, 2025
5 checks passed
@PyDataBlog PyDataBlog deleted the feat/optimise-data-structure branch July 4, 2025 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant