Dev Release 20

Today was a long hectic day that included various internal meetings and discussions around gossip protocol which would help us in implementation of P2P networks in blockDAG. One of the available option is the gossip protocol. Now, let's understand why this protocol came into picture.

Gossip protocols, often likened to rumor spreading in a social network, are powerful tools for distributed systems. They enable peer-to-peer communication and state management, fostering scalability and resilience. This post dives into the world of gossip protocols, exploring rumor mongering (information dissemination) and anti-entropy (data consistency) mechanisms, along with some technical algorithms to bring it all to life.

Imagine a network of nodes, each with a piece of information. Gossip protocols establish communication channels where nodes exchange data with randomly chosen peers. This "rumor mongering" ensures information propagates throughout the network, eventually reaching all nodes.

Here are some popular rumor mongering strategies:

• Push Model: Nodes with new information proactively push it to a random subset of peers. This is efficient for small updates but can overload the network with frequent pushes.

       Here's a code snippet:
function PushGossip(rumor):
     peers = select_random_peers(k) # Choose k random peers
     for peer in peers:
         send_message(peer, rumor)
 

• Pull Model: Nodes periodically pull updates from randomly chosen peers. This reduces network traffic but might lead to slower information dissemination.

       function PullGossip():
          rumors = []
          peers = select_random_peers(k) # Choose k random peers
          for peer in peers:
             rumors += receive_message(peer)
           return rumors
 

• Push-Pull Model: A hybrid approach combining push and pull strategies, offering a balance between efficiency and information spread.

    function PushPullGossip(rumor):
     # Push phase
      peers = select_random_peers(k) # Choose k random peers
      for peer in peers:
          send_message(peer, rumor)
      # Pull phase (can be integrated into a separate periodic function)
       rumors = receive_message_from_all_peers()
       return rumors

Algorithms for Effective Rumor Mongering:

• Random Gossiping: Nodes randomly select peers to exchange information with. Simple and scalable, but convergence (all nodes having the same data) can be slow.

       function RandomGossip():
          if has_new_rumor():
            peer = select_random_peer()
            send_message(peer, get_rumor())
 

• Fanout Gossiping: Each node pushes information to a fixed number of random peers in each gossip round. Faster convergence than random gossiping, but requires tuning the fanout value for optimal performance.

       function FanoutGossip(fanout):
          if has_new_rumor():
             peers = select_random_peers(fanout)
             for peer in peers:
                send_message(peer, get_rumor())
 

• Age-based Gossiping: Nodes prioritize sharing the newest information. This ensures faster propagation of critical updates.

        function AgeBasedGossip():
          rumor = get_oldest_rumor()
             if rumor is not None:
               peer = select_random_peer()
               send_message(peer, rumor)

As discussed internally, blockDAG (Directed Acyclic Graph) projects, gossip protocols play a crucial role in ensuring all nodes possess a consistent view of the ever-growing ledger. These protocols, akin to rumor-mongering in a social network, facilitate data dissemination and state management across the distributed network. But with various gossip protocols at our disposal, selecting the optimal one becomes paramount.

Following Factors can Guide for Gossip Protocol Choice in BlockDAGs:

• Network Dimensions: The sheer size of the blockDAG network significantly impacts the gossip protocol choice. For smaller networks with sporadic updates, a simpler approach might suffice.
• Data Update Frequency: How often new data (transactions or blocks) gets added to the blockDAG is another critical factor. Frequent updates necessitate a more efficient gossip protocol to maintain consistency.
• Desired Consistency Level: The level of consistency required between nodes in the blockDAG network dictates the complexity of the gossip protocol. Higher consistency demands a more robust approach.

• Small Networks with Infrequent Updates: For fledgling blockDAGs with a limited number of nodes and infrequent data additions, a basic random gossiping protocol combined with periodic state exchange for anti-entropy can be adequate. This approach ensures all nodes eventually receive the updates, albeit at a slower pace.
• Larger Networks or Frequent Updates: As the blockDAG scales or experiences a surge in data updates, a more efficient strategy becomes necessary. Here, fanout gossiping shines. By sending information to a fixed number of randomly chosen peers in each gossip round, fanout gossiping achieves faster information propagation compared to random gossiping. To further optimize network traffic, consider employing digest-based anti-entropy. Nodes exchange digests (checksums) of their data first. If digests differ, only the changed portions are exchanged, reducing bandwidth consumption.
• High Consistency Scenarios: For blockDAG applications demanding a high degree of consistency between nodes, a more sophisticated approach is recommended. Merkle tree-based anti-entropy paired with age-based gossiping provides a potent combination. Merkle trees allow efficient identification of data inconsistencies, while age-based gossiping prioritizes the spread of the latest information, ensuring rapid convergence to a consistent state.

By tomorrow we'll be able to finalize the gossip protocol that we'll be using and also soon after that we'll delve into the networking between nodes which includes the following concepts; streams , MPSC(multi producer single consumer ) channels, yamux : multiplexing. 

So, stay tuned for more conclusions around the same.