Announcesub spec: pubsub protocol with no duplicates

[ppopth]

DEPRECATED

Announcesub is a new pubsub protocol over libp2p, heavily based on gossipsub.

The protocol is designed in a way to reduce the number of duplicates to zero, trading off with more latency.

It is actually the same as Gossipsub except that instead of sending the messages right away to mesh peers, it sends IANNOUNCE instead and waits for INEED before sending the actual messages.

This guarantees that each node will receives exactly one copy of messages.

---

Authored by: @ppopth, @nisdas, @chirag-parmar

I made this PR as a draft first just to gain some visibility. We need to do simulations and further analysis to compare it with Gossipsub.

Our next step would be to implement it in go-libp2p-pubsub and do simulations.

[vyzo]

I think it might make more sense to propose this as a gossipsub extension and not an entirely new protocol.

One way it could work is to have two levels of mesh, one for eager propagation and one for announce based lazy propagation.

Set the eager degree to 0, and you get pure announcesub, set the lazy degree to 0, and you get current gossipsub.

[nisdas]

@vyzo Initially it was supposed to be an extension to gossipsub but we figured that because it was a big change to gossipsub having it as an entirely new protocol would be better.

One way it could work is to have two levels of mesh, one for eager propagation and one for announce based lazy propagation.

Yeah, this was something we discussed where there would be a hybrid approach of having both fast and slow links to your mesh rather than a fully announcement based protocol. We can take a look on how to make it an extension rather a new protocol.

[vyzo]

The thing is, this is really where we have been going for gossipaub v2.0; the interplay between eager and lazy.

Having a tunable parameter that can adjust the bandwidth/latency profile would be a big win imo. And on the same time we can further explore dynamic bandwidth tuning schemes that allow a peer (or group of peers and so on) to dynamically adjust (similar to the episub idea).

So really, announcesub seems like a very natural extension of gossipsub.
So I would very much like to see this a gossipsub v2.0 than a separate protocol. This would also solve deployment issues, by maintaining backwards compatibility.

[ppopth]

Alright, I'm writing the spec for gossipsub v2.0

Also note that this is not backward-compatible with gossipsub v1.2, right? because you don't expect a control message from mesh peers.

In fact, I also would like to introduce new control messages (IANNOUNCE/INEED) in order to avoid confusion with the (IHAVE/IWANT) emitted at heartbeats.

[vyzo]

We should keep backwards compat in terms of interoperability with older clients, this will help a lot with deployment.

[ppopth]

this will help a lot with deployment

For Ethereum, we have hard forks anyway which require all the clients to update their software, how about other applications?

Introducing (IANNOUNCE/INEED) will break backward-compatibility I guess. In fact, if we don't force nodes to update, Ethereum will not scale more with full potential, so I think we should force them to update.

[vyzo]

mixed environments are quite common in non blockchain applications i beleieve.

[raulk]

Taking a step back, I see an opportunity to evolve gossipsub v2 into a modular and extensible framework split into two layers.

The system space of gossipsub v2 would be in charge of maintaining one or many named meshes, discovering peers through active or passive mechanisms, tracking metadata and metrics (which would be fed into the scoring engine), tracking subscriptions, etc.

The user space would create and drive mesh instances, implement specialized dissemination policies, evaluate peers as they're being offered from the system, drive scoring based on system input, exchange specialised control messages, adjust system parameters dynamically, etc.

Gossipsub could even become multi-tenant, i.e. a single app could have multiple gossipsub instances/domains (reusing underlays) to cater to the needs of various subsystems co-existing within a process, eg. in an Eth client: mempool, PeerDAS, attestations, Beam-related flows, etc. Each domain can have its specialized policies while reusing the gossipsub system plumbing.

IMO, it's the right time to re-imagine gossipsub under the lens of a framework. This kind of architectural layering is consistent with the original modular vision. Executing on it now creates new degrees of freedom for experimentation and innovation at the app layer, which various teams may welcome to accelerate their roadmap. After all, no one size fits all, and overloading a single protocol with tons of parameters/knobs in order to support all possible variants and characteristics introduces too much complexity and slows everyone down.

[ppopth]

Closed as superseded by #653