# Causal delivery mailbox for pub/sub

**URL:** <https://forum.anoma.net/t/causal-delivery-mailbox-for-pub-sub/1631>\
**Category:** P2P Protocols\
**Created:** [June 18, 2025, 7:13pm UTC](https://forum.anoma.net/t/causal-delivery-mailbox-for-pub-sub/1631 "2025-06-18T19:13:56Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![tg-x](https://dub1.discourse-cdn.com/flex013/user_avatar/forum.anoma.net/tg-x/32/11_2.png) [@tg-x](https://forum.anoma.net/u/tg-x)\
**Post date:** [June 18, 2025, 7:13pm UTC](https://forum.anoma.net/t/causal-delivery-mailbox-for-pub-sub/1631/1 "2025-06-18T19:13:56Z")

</div>

This post describes details of message delivery for the [Distributed Publish/Subscribe Protocol](https://forum.anoma.net/t/distributed-publish-subscribe-protocol/1405) and how this can be implemented with mailboxes, and serves as an example of using mailboxes.

## Message delivery

Subscribers of pub/sub topics deliver messages locally in causal order, buffering messages with missing dependencies.  
It is independent of the dissemination protocol that forwards messages without buffering.  
A message synchronization protocol ensures that missing dependencies eventually arrive.  
Once a missing message arrives, it triggers delivery of deliverable buffered messages.

Causal delivery maintains the causal order according to the message DAG, which is the result of a _topological sort_ of the DAG.

## Causal delivery with mailboxes

A mailbox responsible for causal delivery buffers all messages  
which contain any dependency that has not been delivered yet.  
Since message dependencies are referenced by a _signed message content hash_, the mailbox needs to keep track of message hashes that it has delivered so far.  
In case of a pub/sub topic, _epochs_ are used to limit causal history, where relays and subscribers only need to maintain history for the current epoch.

Subscribers of the pub/sub have different roles, a node may have one or more roles:

- _publisher_: publishes messages
- _subscriber_: delivers messages locally
- _relay_: forwards messages to other relays
- _broker_: a relay with remote subscribers

Let’s take a look at an example of a relay with local subscribers.  
The implementation consists of two engines responsible for handling pub/sub messages:

- _Topic Relay_: responsible for message forwarding between remote relays (_Topic 1_ in the figure)
- _Local Broker_: responsible for causal delivery to local subscribers (_Local Topic 1_ in the figure)

The _Topic Relay_ receives messages from remote relays, stores them in _Local Storage_ in order to be able to serve message requests from other nodes, then forwards them to the _Local Broker_ and other remote relays. It uses FIFO delivery without buffering with a validity predicate that checks whether the message signature comes from a valid publisher.

The _Local Broker_ delivers messages to local subscriber engines in causal order with the help of a mailbox.  
The mailbox uses the _Local Storage_ for message buffering, but since the _Topic Relay_ already stores all messages there, it only needs to keep track the set of buffered message hashes, and when they become deliverable retrieve them from _Local Storage_ and forward them to the _Local Broker_.  
The mailbox has a validity predicate that perform further checks, such as further permissions and validity of the payload.

The two layers of validation checks allows filtering out messages by non-publishers to avoid spam and message amplification attacks, and ensures that all messages with a valid signature are disseminated even if their contents are malicious or invalid, which serves as a proof of misbehaviour for Byzantine nodes. On the second layer the delivery predicate ensures only valid messages are delivered.

 ![image](https://europe1.discourse-cdn.com/flex013/uploads/anoma1/original/2X/4/4bfd409b449e2845815d179aea931a8cbbe6f072.png)

## Message type

```ruby
type TopicMsg := mkTopicMsg {
  publisher : PublisherID;
  deps : Set TopicMsgID;
  tags : Set String;
  content : TopicMsgContent;
  hsig : Signature;
};

type TopicMsgID := mkTopicMsgID {
  publisher : PublisherID;
  hash : Digest;
  hsig : Signature;
};

```

cc @jonathan