Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create storage functions for handling reorgs #90

Merged
merged 1 commit into from
Oct 10, 2024
Merged

Create storage functions for handling reorgs #90

merged 1 commit into from
Oct 10, 2024

Conversation

iuwqyir
Copy link
Collaborator

@iuwqyir iuwqyir commented Oct 4, 2024 •
edited
Loading

TL;DR

Added support for chain reorganization detection and handling in the storage layer.

What changed?

  • Introduced a new BlockHeader struct in block.go
  • Added methods to ClickHouseConnector for managing reorg checks and block headers
  • Implemented GetLastReorgCheckedBlockNumber and SetLastReorgCheckedBlockNumber in storage connectors
  • Added LookbackBlockHeaders and DeleteDataForBlocks methods to IMainStorage interface
  • Updated MemoryConnector to support new reorg-related operations
  • Added reorg check functionality to RedisConnector
  • Created a new SQL script for a cursors table in ClickHouse
  • Modified the staging table creation script to remove the primary key

@iuwqyir iuwqyir mentioned this pull request Oct 4, 2024
Merged
@iuwqyir Graphite App
Copy link
Collaborator Author

iuwqyir commented Oct 4, 2024 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @iuwqyir and the rest of your teammates on Graphite Graphite

This was referenced Oct 4, 2024
Merged
Merged
@iuwqyir iuwqyir changed the title create new storage functions for handling reorgs Create storage functions for handling reorgs Oct 4, 2024
@iuwqyir iuwqyir requested a review from AmineAfia October 4, 2024 16:40
@iuwqyir iuwqyir marked this pull request as ready for review October 4, 2024 16:40
@iuwqyir iuwqyir force-pushed the 10-04-change_main_storage_interface_to_handle_inserts_all_at_once branch from d0e0f6f to 4773497 Compare October 4, 2024 16:46
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch from 81f8a60 to 2e3c10b Compare October 4, 2024 16:46
@iuwqyir iuwqyir force-pushed the 10-04-change_main_storage_interface_to_handle_inserts_all_at_once branch from 4773497 to d06c5d1 Compare October 7, 2024 11:04
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch from 2e3c10b to 487685c Compare October 7, 2024 11:05
@iuwqyir iuwqyir force-pushed the 10-04-change_main_storage_interface_to_handle_inserts_all_at_once branch from d06c5d1 to d0961cb Compare October 7, 2024 11:44
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch 2 times, most recently from dab5d24 to 56646cf Compare October 7, 2024 12:12
@iuwqyir iuwqyir requested a review from AmineAfia October 7, 2024 12:17
@iuwqyir iuwqyir force-pushed the 10-04-change_main_storage_interface_to_handle_inserts_all_at_once branch from d0961cb to b11b079 Compare October 7, 2024 13:19
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch 2 times, most recently from a4e9d3e to b591820 Compare October 7, 2024 13:22
@iuwqyir iuwqyir force-pushed the 10-04-change_main_storage_interface_to_handle_inserts_all_at_once branch from b11b079 to dd391a7 Compare October 7, 2024 20:01
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch from b591820 to ddcfcbb Compare October 7, 2024 20:01
@iuwqyir iuwqyir mentioned this pull request Oct 8, 2024
Merged
@iuwqyir iuwqyir mentioned this pull request Oct 8, 2024
Merged
@iuwqyir iuwqyir force-pushed the 10-04-change_main_storage_interface_to_handle_inserts_all_at_once branch from dd391a7 to ccac684 Compare October 8, 2024 23:32
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch from ddcfcbb to f4c4f06 Compare October 8, 2024 23:32
@iuwqyir iuwqyir force-pushed the 10-04-change_main_storage_interface_to_handle_inserts_all_at_once branch from ccac684 to 6ead9a7 Compare October 10, 2024 12:48
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch from f4c4f06 to d96a682 Compare October 10, 2024 12:48
@iuwqyir Graphite App
Copy link
Collaborator Author

iuwqyir commented Oct 10, 2024 •
edited
Loading

Merge activity

  • Oct 10, 10:01 AM EDT: A user started a stack merge that includes this pull request via Graphite.
  • Oct 10, 10:06 AM EDT: Graphite rebased this pull request as part of a merge.
  • Oct 10, 10:07 AM EDT: A user merged this pull request with Graphite.

@iuwqyir iuwqyir changed the base branch from 10-04-change_main_storage_interface_to_handle_inserts_all_at_once to graphite-base/90 October 10, 2024 14:02
@iuwqyir iuwqyir changed the base branch from graphite-base/90 to main October 10, 2024 14:04
@iuwqyir iuwqyir force-pushed the 10-04-create_new_storage_functions_for_handling_reorgs branch from d96a682 to 0f70f2e Compare October 10, 2024 14:05
@iuwqyir iuwqyir merged commit 80fcbb5 into main Oct 10, 2024
4 checks passed
@iuwqyir iuwqyir deleted the 10-04-create_new_storage_functions_for_handling_reorgs branch October 10, 2024 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AmineAfia AmineAfia AmineAfia approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@iuwqyir @AmineAfia