Exon: An Oblivious Exactly-Once Messaging Protocol With Reliable Delegation

TCP is the default transport protocol of choice, namely for message-oriented middleware protocols (e.g., ZMTP, AMQP, MQTT) or distributed language runtimes (e.g., distributed Erlang), where exactly-once (EO) messaging is paramount. However, EO is only guaranteed within the TCP session, since reality shows that TCP connections can fail under many circumstances. Ensuring EO delivery ends up at the middleware layer, at the cost of higher complexity and lack of obliviouness—due to the use of permanent per-peer state. Moreover, using TCP at scale in highly concurrent systems leads to the need for TCP connection multiplexing, and possibly drastic performance loss due to head-of-line blocking. This paper introduces Exon, an oblivious exactly-once messaging protocol, and a corresponding lightweight (requiring no persistent storage, minimal memory, and low computation) library implementation over UDP. Exon uses a novel strategy of a per-message four-way protocol to ensure oblivious exactly-once messaging, with on-demand protocol-level “soft half-connections”, established when needed and safely discarded. Obliviousness here refers to the protocol’s ability to discard connection-specific state between incarnations, although some global information is retained. Exon achieves simultaneously: correctness with no timing assumptions, obliviousness, and performance through merging and pipelining basic protocol messages. Exon also employs a reliable delegation technique to handover the sending responsibility to a mediating node, without violating EO, when the sender the receiver are directly unreachable to each other and even if the message had already been delivered. The empirical evaluation of Exon demonstrates significant improvements (40%) over TCP in throughput and latency under packet loss, while maintaining a negligible (8%) overhead in healthy networks.