Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event Store Initial Implementation #2507

Merged
merged 11 commits into from
Jun 15, 2022
Merged

Conversation

velvia
Copy link
Contributor

@velvia velvia commented Jun 9, 2022

Initial implementation of the EventStore trait (and some modifications to it) based on SQLite and the sqlx crate.
Methods are enough for basic event store queries by event type and module/package ID, per events implementation plan.

NOTE: The choice to have a JSON field is to allow for easy filtering of any field within Move events and other events. However, it would take up more space and duplicate the binary data in a BCS field.
Also it gives flexibility to add more fields to other events in the future without having to dedicate a column to each.
The idea is that the separate columns are the main ones used for filtering things down.

To be added:

  • A test for Move events (would love hints for implementing event generation)
  • Benchmarks
  • Add in column to store the binary BCS bytes for Move events

@velvia velvia requested review from longbowlu and patrickkuo June 9, 2022 21:49
Copy link
Collaborator

@longbowlu longbowlu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work!

crates/sui-storage/src/event_store/mod.rs Show resolved Hide resolved
crates/sui-storage/src/event_store/mod.rs Show resolved Hide resolved
crates/sui-storage/src/event_store/mod.rs Outdated Show resolved Hide resolved
crates/sui-types/src/event.rs Outdated Show resolved Hide resolved
crates/sui-storage/src/event_store/sql.rs Outdated Show resolved Hide resolved
crates/sui-storage/src/event_store/sql.rs Show resolved Hide resolved
// to execute all inserts in a single statement?
// TODO: See https://kerkour.com/high-performance-rust-with-sqlite
for event in events {
// If batching, turn off persistent to avoid caching as we may fill up the prepared statement cache
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious when does the cache get flushed?

/// Will return at most limit of the most recent events within the window, sorted in ascending time.
fn events_by_module_id(
/// Will return at most limit of the most recent events within the window, sorted in descending time.
async fn events_by_module_id(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

function_name?

crates/sui-storage/src/event_store/sql.rs Show resolved Hide resolved
crates/sui-storage/src/event_store/sql.rs Outdated Show resolved Hide resolved
@velvia
Copy link
Contributor Author

velvia commented Jun 13, 2022

@longbowlu pushed a new commit to address most of your comments. Move event test also coming.

Copy link
Collaborator

@longbowlu longbowlu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great

crates/sui-storage/src/event_store/sql.rs Outdated Show resolved Hide resolved
crates/sui-storage/src/event_store/sql.rs Outdated Show resolved Hide resolved
crates/sui-storage/src/event_store/sql.rs Show resolved Hide resolved
crates/sui-storage/src/event_store/sql.rs Show resolved Hide resolved
@velvia velvia enabled auto-merge (squash) June 15, 2022 06:14
@velvia velvia merged commit 74d519b into MystenLabs:main Jun 15, 2022
@velvia velvia deleted the ec/event-store-impl1 branch June 15, 2022 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants