Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Topics #126

Closed
wants to merge 407 commits into from
Closed
Show file tree
Hide file tree
Changes from 250 commits
Commits
Show all changes
407 commits
Select commit Hold shift + click to select a range
df74c4b
Run cargo fmt
emhane Jun 21, 2022
28188d0
Move hashing of topic up one layer to app
emhane Jun 21, 2022
8c8b1ff
Remove unused deps
emhane Jun 21, 2022
bbe4329
Fix send_register_topics logic
emhane Jun 21, 2022
622ca2b
Add warning
emhane Jun 21, 2022
1746aaf
Fix regtopics
emhane Jun 21, 2022
066e2a5
Fix regtopics
emhane Jun 21, 2022
dd12411
Allow NODES repsonse for RETOPIC
emhane Jun 21, 2022
b914ac4
Match NODES response for REGTOPIC req
emhane Jun 21, 2022
ed26c95
Add hack for topic query double nodes response
emhane Jun 21, 2022
ea80eab
Add warning to topic query double nodes response logic
emhane Jun 21, 2022
f817aee
Add debug info
emhane Jun 21, 2022
741c19a
Add debug info
emhane Jun 21, 2022
df6fdb1
Add debug info
emhane Jun 21, 2022
1b9e9c2
Add debug messages
emhane Jun 21, 2022
003b816
Update regtopic active requests for addtional NODES response
emhane Jun 21, 2022
505aff4
Amplify distances reqeusted for topics
emhane Jun 21, 2022
fe5a5d0
Add trace for responses
emhane Jun 21, 2022
c3311e6
Add trace messages
emhane Jun 22, 2022
65cc66c
Add trace message
emhane Jun 22, 2022
3916017
Fix reinsertion of active req handler
emhane Jun 22, 2022
b139738
Ignore wait time zero tickets
emhane Jun 22, 2022
fdea8b2
Add trace
emhane Jun 22, 2022
888f9fc
Git increase request timeout
emhane Jun 22, 2022
42879b3
Increase request timeout to receive REGCONFIRMATION
emhane Jun 22, 2022
2fe218a
Fix distance amplification bug
emhane Jun 22, 2022
1cf0aa5
Increase timeout handler
emhane Jun 22, 2022
dfb51a9
Add trace for TOPICQUERY
emhane Jun 22, 2022
f905849
Increase trace messages in buggy area
emhane Jun 22, 2022
e36e2a5
Fix endless while loop bug
emhane Jun 22, 2022
995b450
Remove trace messages
emhane Jun 22, 2022
f804e9a
Add trace for topic query ad nodes not sending
emhane Jun 22, 2022
3290184
Add extra ADNODE response body
emhane Jun 22, 2022
a212173
Receive AdNodes response on service level
emhane Jun 22, 2022
f5b5ef8
Reuse code for adding batches of new nodes to topics kbuckets
emhane Jun 22, 2022
97c42e7
Fix carg clippy
emhane Jun 22, 2022
024f548
Add debug for request service level
emhane Jun 23, 2022
b10885c
Fix reinsertion of active request bug
emhane Jun 23, 2022
55815d3
Fix topic query response state bug
emhane Jun 23, 2022
53c0dab
Fix topic query response state bug
emhane Jun 23, 2022
e688e71
Introduce state for topic query responses in handler
emhane Jun 23, 2022
e913344
Filter nodes in responses before insertion in kbuckets
emhane Jun 23, 2022
f46e694
Constrain response combo topics in handler
emhane Jun 23, 2022
ff5bb1d
Fix clippy warnings
emhane Jun 23, 2022
6b2f3c9
Add trace messages
emhane Jun 24, 2022
ef768e4
Add trace message
emhane Jun 24, 2022
88977c2
Rearrange imperative order to fix empty topic kbuckets bug
emhane Jun 24, 2022
89aa724
Send ADNODES responses
emhane Jun 24, 2022
24b3d1e
Remove TOPICQUERY query if no more peers
emhane Jun 24, 2022
36dd5a1
Fix missing message decode AdNodes response
emhane Jun 24, 2022
cb7c336
Increase timeout for testing regconfirmation from 2 nodes
emhane Jun 24, 2022
dd21d02
Add trace info for TOPICQUERY dry state
emhane Jun 24, 2022
fdf3d5a
Add trace messages in buggy zone
emhane Jun 25, 2022
d5cabd9
Add trace in buggy zone
emhane Jun 25, 2022
5607e96
Set queried peer upon return of result
emhane Jun 25, 2022
31a82f8
Pass query through dry state before remove
emhane Jun 25, 2022
4aa52f2
Set regconfirmation time out spearately in handler
emhane Jun 27, 2022
da33ffc
Handle timeout for topic requests
emhane Jun 27, 2022
b959a0e
Proceed as normal in handler if register confirmation does not come
emhane Jun 27, 2022
e6b8a11
Solve merge conflicts
emhane Jun 28, 2022
c21f8df
Run cargo fmt
emhane Jun 28, 2022
6054069
Assign wait time based on similarity score
emhane Jun 28, 2022
b673eb0
Add discovered topic enrs to topic kbuckets
emhane Jun 29, 2022
5e879c9
Add discovered topic nodes as trusted enrs after filter
emhane Jun 29, 2022
6f7ee38
Add new enrs from pong to topic kbuckets also
emhane Jun 29, 2022
1655dad
fmt
divagant-martian Jun 29, 2022
9173669
Fix reception of pong response service
emhane Jun 29, 2022
37396b2
Track ads by subnet
emhane Jun 29, 2022
139fcd2
Fix clippy warnings
emhane Jun 29, 2022
95f2a07
Add tests for subnet ticket wait time
emhane Jun 30, 2022
a482724
Filter tickets before selection
emhane Jun 30, 2022
054520b
Filter returned ad nodes before inserting in query
emhane Jun 30, 2022
d68d0a7
Insert nodes with correct connection direction
emhane Jul 1, 2022
3d13dac
Fix bug
emhane Jul 1, 2022
0b6ddff
Remove old logic
emhane Jul 4, 2022
7ece2b0
Remove block upon awaiting regconfirmation
emhane Jul 4, 2022
d2f2744
fixup! Remove block upon awaiting regconfirmation
emhane Jul 4, 2022
983819b
Ban nodes that send invalid ticket and fix bug
emhane Jul 4, 2022
42ee4e4
Run cargo fmt
emhane Jul 4, 2022
c107493
Check distances of returned nodes for topic reqs
emhane Jul 4, 2022
9012e26
Fixup comments
emhane Jul 4, 2022
fbcdfdf
Comment code and fix misc bugs
emhane Jul 4, 2022
a2186b9
Update rust before running clippy and check-rustdoc-links
emhane Jul 4, 2022
57a831e
Merge branch 'master' of github.com:sigp/discv5
emhane Jul 4, 2022
c38e2d0
Fix typo
emhane Jul 4, 2022
c760f41
Fix comment
emhane Jul 5, 2022
cf6fd4d
Put repeated code in sync closure
emhane Jul 6, 2022
01ad49d
Fix retain statements and unsafe addition of discovered nodes to topi…
emhane Jul 11, 2022
e3e8126
Add bounds to discovered topic peers storage
emhane Jul 11, 2022
84dfb3f
remove workflow difference
divagant-martian Jul 12, 2022
941de59
Merge branch 'master' into tickets-v2
divagant-martian Jul 12, 2022
17156b2
Update method for getting active ads
emhane Jul 13, 2022
0a1d65f
Solve merge conflicts
emhane Jul 13, 2022
d5e5e95
fixup! Update method for getting active ads
emhane Jul 13, 2022
a78cf3f
Ensure topics republish
emhane Jul 13, 2022
f0a0e1e
Fix copy paste bug
emhane Jul 13, 2022
b9a53be
fixup! Ensure topics republish
emhane Jul 13, 2022
2f9223a
Proceed with topic lookup upon newly discovered peers for topic
emhane Jul 13, 2022
d4ac5fc
Run cargo fmt
emhane Jul 13, 2022
5cafcb4
Fix uncontacted peers first in topic request bug
emhane Jul 13, 2022
8c38d10
Add trace message
emhane Jul 13, 2022
f1f6a94
fixup! Fix uncontacted peers first in topic request bug
emhane Jul 13, 2022
31ff35f
Fix bug of proceeding with query if discovered called with empty enrs…
emhane Jul 13, 2022
1595444
Fix discovery bug
emhane Jul 13, 2022
62a0e39
Fix bug using uncontacted peers first for topic requests
emhane Jul 14, 2022
eeeaa33
Fix bug upon new discovered peers
emhane Jul 14, 2022
4936e03
Fix find new topic query peers bug
emhane Jul 14, 2022
64e287f
Add trace message for republish
emhane Jul 14, 2022
8cca202
Add trace messages for re-registration on same node after ad-lifetime up
emhane Jul 14, 2022
b818e30
Fix sending double nodes in NODES response to topic request
emhane Jul 15, 2022
fb690be
Fix bug overriding reg attempt and speed up topic query in case of no…
emhane Jul 15, 2022
4ac8600
Debug
emhane Jul 15, 2022
0dfcba9
Fix replace ticket with regconfirmation bug
emhane Jul 15, 2022
924c986
Debug distance mapping reg attempts and kbuckets
emhane Jul 15, 2022
e4596e1
Fix distance mapping bug
emhane Jul 15, 2022
ef4be68
Fix re-publishing topics on nodes have reached ticket limit bug
emhane Jul 15, 2022
7ab9ae3
Fix bug sending the recipient's node in NODES response to a topic req…
emhane Jul 15, 2022
586a5bf
Set ad lifetime
emhane Jul 15, 2022
4497957
Update docs
emhane Jul 18, 2022
e2ced6e
Run cargo fmt
emhane Jul 18, 2022
f19d2bb
Change name to be meanigful to new logic
emhane Jul 18, 2022
02ed526
Retrieve hosted ads for a topic from app level
emhane Jul 18, 2022
5b3364e
Retrieve active registration attempts from app for debug
emhane Jul 19, 2022
075e186
Retrieve table entries for a topic from app
emhane Jul 19, 2022
a67515e
Fix async bug
emhane Jul 19, 2022
00669ec
Change trace to debug messages
emhane Jul 19, 2022
fcf5549
Only send DiscoveredTopic event for new peers
emhane Jul 19, 2022
09fce24
Limit triggering TICKET, NODES and REGCONFIRMATION responses to a max…
emhane Jul 20, 2022
880f63f
Set max regtopis per registration interval
emhane Jul 20, 2022
3eff0ac
Fix bug of triggering reg-topics outside of interval
emhane Jul 20, 2022
4b2dac3
Correct return type
emhane Jul 21, 2022
a8bdc31
Clarify docs
emhane Jul 22, 2022
c15c500
Correct docs
emhane Jul 22, 2022
215467e
Correct docs
emhane Jul 22, 2022
17a05fc
Correct docs
emhane Jul 22, 2022
9fe32be
Fix typo
emhane Jul 22, 2022
d8a229c
Add missing docs for advertisement crate
emhane Jul 22, 2022
9fbf099
Solve merge conflict
emhane Jul 22, 2022
6130cb7
Fix typo
emhane Jul 22, 2022
58217ad
Fix typo
emhane Jul 22, 2022
043c2fe
Clarify bounds
emhane Jul 22, 2022
6bad729
Follow Rust pattern of new and default
emhane Jul 22, 2022
b2a97cf
Merge branch 'tickets-v2' of github.com:emhane/discv5 into tickets-v2
emhane Jul 22, 2022
080131b
Constrain ad insertion and deactivate pool tickets
emhane Jul 22, 2022
1392646
Fix bug reinserting entry which cannot be immutably borrowed
emhane Jul 22, 2022
3a70ac3
Add missing docs in discv app interface
emhane Jul 23, 2022
30210cb
Move method to associated function
emhane Jul 23, 2022
23af87e
fixup! Add missing docs in discv app interface
emhane Jul 23, 2022
d014b94
fixup! Move method to associated function
emhane Jul 23, 2022
c65debc
Fix broken docs
emhane Jul 23, 2022
9636ea4
Fix clippy warning
emhane Jul 24, 2022
fc26548
Fix const function
emhane Jul 24, 2022
a0a1a26
Export global helper function
emhane Jul 24, 2022
b47b455
Remove static lifetime
emhane Jul 24, 2022
c5c158e
Correct poll behaviour for ticket stream
emhane Jul 25, 2022
4757816
Attempt to simplify the handler
AgeManning Jul 27, 2022
0b19596
Merge pull request #3 from sigp/simplify-handler
emhane Jul 27, 2022
8617437
Remove ADNODES response
emhane Jul 27, 2022
ba15a64
Rlp encode topics for enr
emhane Jul 27, 2022
f88d89f
Use ticket wait time to communicate a successful registration instead…
emhane Jul 27, 2022
a5829af
Remove unused TicketsPool due to replacing comparison of ad candidate…
emhane Jul 27, 2022
43376e2
Remove multiple responses to topic requests in Handler
emhane Jul 28, 2022
7cd0fdf
Mix regtopic requests with normal active requests at service level as…
emhane Jul 28, 2022
e2c4551
Remove multiple responses to topic requests in service
emhane Jul 28, 2022
9d02c67
Add ads returned in NODES responses to topic lookup query
emhane Jul 28, 2022
c90b338
fixup! Remove multiple responses to topic requests in service
emhane Jul 28, 2022
9d1a376
Add nodes to topic kbuckets by FINDNODE iterative query
emhane Jul 29, 2022
db89688
Query for more peers upon dry topic lookup
emhane Jul 29, 2022
d2b5339
Fix bug in stream of topic lookups
emhane Jul 29, 2022
1d7ef49
Verify enrs of ads returned by TOPICQUERY
emhane Jul 29, 2022
813af26
Verify topics field in ENR upon incoming registration attempt
emhane Jul 29, 2022
fe8a335
Simplify code
emhane Jul 30, 2022
0a26668
Minimize LOC
emhane Jul 30, 2022
deb42fd
Remove function relevant for potential future implementation of mutli…
emhane Aug 1, 2022
50c462d
Use strings as topics in REGTOPICS
emhane Aug 1, 2022
9359d19
Add trace message
emhane Aug 2, 2022
8eed39f
Improve trace message
emhane Aug 2, 2022
f7221df
Fix bogus while loop logic
emhane Aug 2, 2022
7fb23a1
Add peers to topic's kbuckets when query is done not during
emhane Aug 2, 2022
bd9604b
Only ad nodes which support topics version to topic kbuckets
emhane Aug 3, 2022
e11c6dd
Advertise this node as supporting topics
emhane Aug 3, 2022
c0dca81
Run cargo fmt
emhane Aug 3, 2022
10d73be
Move enr version insertion to up a layer to discv5
emhane Aug 3, 2022
a6d57f2
Add debug
emhane Aug 3, 2022
2470410
If topic lookup runs dry, find more peers
emhane Aug 3, 2022
5a7f0e8
fixup! If topic lookup runs dry, find more peers
emhane Aug 3, 2022
d02bd7a
Allow peers which don't include version field in enr
emhane Aug 5, 2022
c830ed0
Fix typo
emhane Aug 30, 2022
679635f
Fix typo
emhane Aug 30, 2022
8bdd653
Use consants instead of string literals
emhane Aug 30, 2022
36a60cc
Fix non-backwards compatibale changes as introduced by new stable rust
emhane Aug 30, 2022
033cf82
Make name more verbose
emhane Aug 30, 2022
cfc837b
Clarify docs
emhane Aug 30, 2022
0271c7d
Fix typo
emhane Aug 30, 2022
fdec540
Make name more verbose
emhane Aug 30, 2022
7b10c0d
Merge branch 'tickets-v2' of github.com:emhane/discv5 into tickets-v2
emhane Sep 5, 2022
2a5942a
Make name more verbose
emhane Sep 5, 2022
21aa98d
Improve wording
emhane Sep 5, 2022
89928a4
Correct typo
emhane Sep 5, 2022
49ed28f
Correct typo
emhane Sep 5, 2022
36b9c43
Correct typo
emhane Sep 5, 2022
3bd661c
Correct typo
emhane Sep 5, 2022
610fd7f
Improve docs semantics
emhane Sep 5, 2022
830a29c
Correct typo
emhane Sep 5, 2022
7a23172
Correct typo
emhane Sep 5, 2022
a72e750
Correct typo
emhane Sep 5, 2022
00a7d8f
Correct typo
emhane Sep 5, 2022
f46d9da
Link constant in docs
emhane Sep 5, 2022
58491e0
fixup! Merge branch 'tickets-v2' of github.com:emhane/discv5 into tic…
emhane Sep 5, 2022
090a361
Correct typo
emhane Sep 5, 2022
96af7f7
Correct typo
emhane Sep 5, 2022
b1a8440
Correct typo
emhane Sep 5, 2022
ef31700
Correct typo
emhane Sep 5, 2022
642a962
Use custom type to make code more descriptive
emhane Sep 5, 2022
6905c7b
Merge branch 'tickets-v2' of github.com:emhane/discv5 into tickets-v2
emhane Sep 5, 2022
a524876
Improve code structure
emhane Sep 5, 2022
bdb5974
Fix bug of replacing a topics registration attempts
emhane Sep 5, 2022
1e8e28d
Make proper use of callbacks to ServiceRequests and fix misc typos
emhane Sep 5, 2022
a4e4d0b
Stop some continued registrations for a removed topic
emhane Sep 5, 2022
eb92b9b
fixup! Stop some continued registrations for a removed topic
emhane Sep 5, 2022
c79db97
Fix typo
emhane Sep 5, 2022
a142ad2
Misc fixes
emhane Sep 5, 2022
1140597
Merge branch 'tickets-v2' of github.com:emhane/discv5 into tickets-v2
emhane Sep 5, 2022
ace89ab
Ignore VS Code custom spelling settings file
emhane Sep 5, 2022
a745f8e
Merge branch 'sigp:master' into tickets-v2
emhane Sep 5, 2022
2068253
Properly implement version check
emhane Sep 6, 2022
ec1c50e
fixup! Ignore VS Code custom spelling settings file
emhane Sep 6, 2022
7f32c61
Merge branch 'tickets-v2' of github.com:emhane/discv5 into tickets-v2
emhane Sep 6, 2022
8dca222
Encapsulate RLP en-/decoding for topics enr field
emhane Sep 6, 2022
08de697
fixup! Encapsulate RLP en-/decoding for topics enr field
emhane Sep 6, 2022
bcd51f8
Add test for en-/decoding topics enr field
emhane Sep 6, 2022
39aa82a
Lay ground for uncostly subscribing (repeated look up) of a topic and…
emhane Sep 7, 2022
82c6257
Fix bug of trying to register removed topic and misc grammar fixes
emhane Sep 7, 2022
a69a655
Move decryption of tickets to Hanlder
emhane Sep 7, 2022
35201e3
Make use of the fact that nodes can only register themselves as ads
emhane Sep 7, 2022
4baeafe
Move repeated code to closure
emhane Sep 7, 2022
ef1d76a
Clean up and comment
emhane Sep 7, 2022
ad01569
Remove unused methods
emhane Sep 8, 2022
0b09f60
Correct name
emhane Sep 8, 2022
3111592
Run cargo fmt
emhane Sep 8, 2022
8386fba
Merge branch 'master' of github.com:sigp/discv5 into tickets-v2
emhane Sep 9, 2022
f2caf4a
Change name
emhane Sep 9, 2022
2d5954d
Fix merge conflict
emhane Sep 9, 2022
66f43ae
Fix bad return bug
emhane Sep 9, 2022
b699290
fixup! Fix bad return bug
emhane Sep 9, 2022
86b7491
Set features in ENR
emhane Sep 9, 2022
5b2209a
Return whole topic for human readable name for logs
emhane Sep 9, 2022
a18ae8c
Only log events for topics to maintain backwards compatibility
emhane Sep 9, 2022
b37e1b1
fixup! Only log events for topics to maintain backwards compatibility
emhane Sep 9, 2022
69cd507
fixup! Only log events for topics to maintain backwards compatibility
emhane Sep 9, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,4 @@ jobs:
- name: Get latest version of stable rust
run: rustup update stable
- name: Check rustdoc links
run: RUSTDOCFLAGS="--deny broken_intra_doc_links" cargo doc --verbose --workspace --no-deps --document-private-items
run: RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links" cargo doc --verbose --workspace --no-deps --document-private-items
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ Cargo.lock

# VIM swap files
*.sw[op]

# VS Code settings files
/.vscode/
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ lru = "0.7.1"
hashlink = "0.7.0"
delay_map = "0.1.1"
more-asserts = "0.2.2"
base64 = "0.13.0"

[dev-dependencies]
rand_07 = { package = "rand", version = "0.7" }
Expand Down
4 changes: 3 additions & 1 deletion examples/find_nodes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ async fn main() {
match discv5.find_node(target_random_node_id).await {
Err(e) => warn!("Find Node result failed: {:?}", e),
Ok(v) => {
// found a list of ENR's print their NodeIds
// found a list of ENRs print their NodeIds
let node_ids = v.iter().map(|enr| enr.node_id()).collect::<Vec<_>>();
info!("Nodes found: {}", node_ids.len());
for node_id in node_ids {
Expand All @@ -191,8 +191,10 @@ async fn main() {
}
match discv5_ev {
Discv5Event::Discovered(enr) => info!("Enr discovered {}", enr),
Discv5Event::DiscoveredPeerTopic(enr, topic_hash) => info!("Enr discovered {} for topic {}", enr, topic_hash),
Discv5Event::EnrAdded { enr, replaced: _ } => info!("Enr added {}", enr),
Discv5Event::NodeInserted { node_id, replaced: _ } => info!("Node inserted {}", node_id),
Discv5Event::NodeInsertedTopic { node_id, replaced: _, topic_hash } => info!("Node inserted {} in topic hash {} kbucket", node_id, topic_hash),
Discv5Event::SessionEstablished(enr, _) => info!("Session established {}", enr),
Discv5Event::SocketUpdated(addr) => info!("Socket updated {}", addr),
Discv5Event::TalkRequest(_) => info!("Talk request received"),
Expand Down
Loading