transaction sync #2718
Merged
transaction sync #2718
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
update non-relay request periodically.
Improve late bloom timing accuracy
…1948) * create a msgStat object so that we don't have to sscanf(sprintf()) * reorg to local Logger interface
// algofix allow sync // algofix require deadlock
cleanup
use comments to locally allow sync.Mutex instead of rewriting it to the much slower deadlock.Mutex
Avoid using the `deadlock.Mutex` in the emulator, and use a channel instead. The `deadlock.Mutex` was consuming considerable amount of memory, due to it's internally allocated timer. edit: following on rare data races found on travis, I avoided using the main state's clock in the async methods. The clock is being updated once every round, so we can't really control that unless we add a synchronization primitive or find a better way.
* add unit test for txn cache * rename transactionLru -> transactionCache
Ensure the timing for the late bloom filter for outgoing relays is calculated correctly : it will now default to use a single window offset, or use the bandwidth & previous bloom filter to estimate sending time.
rebase feature/txnsync with master
Rebase transaction sync with master
Refactor the bloom filter implementation to have two "implementations" : testableBloomFilter and bloomFilter. testableBloomFilter is used exclusively when being received from the network; it does not contain any encoded fields, nor does it include transactionsRange which is used for generating new bloom filters. bloomFilter on the flip side is used only for outgoing bloom filters. It doesn't support testing, and it contains only the encoded bloom filter ( and not the underlying bloom filter needed for testing, for instance ). This separation allows us to ensure we don't store bloom filter object beyond the required scope, improving the memory utilization.
…o feature/txnsync
Add an in-code comment.
Existing short term cache in the transaction sync was not supporting promoting updated cache entries. In this PR, I have replaced the circulating buffer with a dual linked list to allow promoting cached entries.
Improve the emulator setup of the transactions that would be exchanged.
On non-relays, exclude txid that were sent from the relay from the bloom filter sent to the relay. This is expected to reduce the size of the bloom filters by 25%.
merge master into txnsync
In onTransactionPoolChangedEvent, there is an action taken on every peer which is both in s.interruptablePeers and in s.scheduler.peers.
This is accomplished in the loop:
for _, peer := range peers {
peerNext := s.scheduler.peerDuration(peer)
where for each peer, the list of s.scheduler.peers is traversed when peerDuration is called.
This is len(peers) * len(s.scheduler.peers) complexity.
The task here is to make this operation linear instead of potentially a quadratic operation.
merge master into txnsync branch
Only send bloom/xor filter for txns that came in after the last filter was sent.
ghost
approved these changes
Oct 4, 2021
tsachiherman
added a commit
to tsachiherman/go-algorand
that referenced
this pull request
Nov 2, 2021
This reverts commit 69aace5.
tmc
pushed a commit
to tmc/go-algorand
that referenced
this pull request
Mar 7, 2025
## Summary This PR implements the transaction sync 2.0. More details can be found in the design document and the feature presentation . ## Test Plan Unit tests, e2e tests and performance tests were executed against this branch successfully. ## Reviewer notes This PR is pretty large. Consider reviewing it in browsers other than Safari.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements the transaction sync 2.0.
More details can be found in the design document and the feature presentation .
Test Plan
Unit tests, e2e tests and performance tests were executed against this branch successfully.
Reviewer notes
This PR is pretty large. Consider reviewing it in browsers other than Safari.