Clarify the purpose of periodically calling `Kademlia::get_closest_peer`

[lyulka]

Kademlia::get_closest_peer is periodically called (so-called "random walk"), seemingly in the belief that this will help "peer discovery".

However, there is a lack of explanation as to how exactly periodically calling Kademlia::get_closest_peer helps with peer discovery, and an in-depth online search only finds vague references to how Kademlia does, or is supposed to do peer discovery. This lack of documentary material (a widespread problem with libp2p) + the 'hacky' nature of the solution (periodically calling a function) concerns me, so I want to do more research and writing on this.

This issue defines peer discovery, and then collects some evidence I found online against, for, and partially for "random walk" being the appropriate solution for peer discovery.

What is "peer discovery"

initially, a peer that wants to join the network (the "joining peer") may only know the address information of a single peer (the "bootstrap peer"). The joining peer uses this address information to connect with the bootstrap peer, and then uses peer discovery to get the address information of other peers that it should have in its routing table/k-buckets.

More tersely, understand "peer discovery" as: the mechanism by which a joining peer fills its k-buckets, starting with just a single entry.

Evidence

Against

• This section in the libp2p docs suggests that "random walk" is a built-in feature of Kad in libp2p, and we don't have to trigger it explicitly.
• The result of the get_closets_peers query (QueryResult::GetClosestPeers) is ignored in engine's event handling loop.

For

• The Kademlia protocol specification in libp2p states that: "Any time a relevant Peer record is encountered, the associated multiaddrs are stored in the node's peerbook."
  • The way I interpret this is that any time peer information is received by a Kademlia handler through a message, it (in the background) tries to add it to its local node's routing table. This justifies why the GetClosestPeer result does not have to be explicitly used.

Partially for

• The Wikipedia page on Kademlia describes a process outwardly similar to "random walk". However, instead of picking a global random key to FIND_NODES on, it suggests picking a random key inside each k-bucket range.
• As "for" mentions, the protocol specifies that a peer's routing table is updated any time a relevant Peer record is encountered. However, there may be other things that can cause the Kademlia handler to receive new peer information. For example, content provider advertisement, which by default happens periodically (by default, every 12 hours). Maybe this mechanism renders "random walk" unnecessary.

Action points

I need input, research, and testing by members to reach a conclusion on "random walk".

• If it is concluded that another existing mechanism already does peer discovery (e.g., content provider advertisement), then we will mark "random walk" as an unnecessary feature and remove it from pchain-network in v0.4.3.
• If it is concluded that "random walk" is the only existing mechanism for peer discovery:
  • ...and it is the best mechanism we can think of, then it shall remain in pchain-network and prescribed in the P2P section of the ParallelChain protocol.
  • but there is a less hacky or better mechanism to do it, then we shall mark it in breaking idiosyncrasies and remove it from the next version of the ParallelChain protocol.

[lyulka]

Note that libp2p Kademlia has a bootstrap method.

This is worth considering to supplant random walk, at least when a node starts up.