p2p: make dial faster by streamlined discovery process #31678
+198
−15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding an optional name to the dial sources allows better diagnostics of FairMix and the whole dial process. Signed-off-by: Csaba Kiraly <[email protected]>
Simple version that does the filtering, but misses pipelining, waiting for ENRs to be retrieved one-by-one. Signed-off-by: Csaba Kiraly <[email protected]>
Signed-off-by: Csaba Kiraly <[email protected]>
Signed-off-by: Csaba Kiraly <[email protected]>
It is not guaranteed that Next will be called until exhaustion after Close was called. Hence, we need to empty override the passed channel. Signed-off-by: Csaba Kiraly <[email protected]>
Signed-off-by: Csaba Kiraly <[email protected]>
Signed-off-by: Csaba Kiraly <[email protected]>
BufferIter wraps an iterator and prefetches up to a given number of nodes from it. Signed-off-by: Csaba Kiraly <[email protected]>
Signed-off-by: Csaba Kiraly <[email protected]>
Signed-off-by: Csaba Kiraly <[email protected]>
|
First hour of dial progress before this PR: number of outgoing peers First hour of dial progress after this PR (and #31592): number of outgoing peers |
Signed-off-by: Csaba Kiraly <[email protected]>
|
Discovery traffic before: And after: This PR increases the initial UDP traffic, mostly caused by the pre-fetch buffer. We might dial that down a bit, although I don't think the amount of traffic generated (~70 KB/s) is too much. On the long term, average discovery traffic should be the same. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR improves the speed of Disc/v4 and Disc/v5 based discovery. It builds on #31592. To be rebased after merging that PR.
Our dial process is rate-limited, but until now, during startup, our discovery was too slow to serve dial. So the bottleneck was discovery, not the rate limit in dial, resulting in slow ramp-up of outgoing peer count.
This PR making discovery fast enough to serve dial's needs. The rate-limit of dial is still limiting the discovery process, so we are not risking being too aggressive in our discovery.
The PR adds: