Unbeatable consensus

Abstract

The unbeatability of a consensus protocol, introduced by Halpern et al. (SIAM J Comput 31:838–865, 2001), is a stronger notion of optimality than the accepted notion of early stopping protocols. Using a novel knowledge-based analysis, this paper derives the first explicit unbeatable consensus protocols in the literature, for the standard synchronous message-passing model with crash failures. These protocols strictly dominate the best kno-wn protocols for uniform and for Nonuniform Consensus, in some cases improving on them by a large margin. The analysis provides a new understanding of the logical structure of consensus, and of the distinction between uniform and nonuniform Consensus. All protocols presented in this paper have very concise descriptions, and are shown to be efficiently implementable.

Appendices

A Additional Proofs of Sect. 4

Proof of Lemma 1

Let P be a consensus protocol and let $R_P=R(P,{\gamma ^{{{\varvec{t}}}}_{\mathrm {cr}}})$. Let ${\mathtt {v}}\in {\mathtt{V}}$, let $r\in R_P$ and let $\langle {i,m}\rangle$ be a node s.t. i decides on ${\mathtt {v}}$ at time m in r.

We commence by proving (a). Assume for contradiction that no process has initial value ${\mathtt {v}}$ in r. By definition of ${\gamma ^{{{\varvec{t}}}}_{\mathrm {cr}}}$, there exists a run $r'$ of P, s.t.  1) $r'_i(m)\!=\!r_i(m)$, 2) i does not fail in $r'$, and 3) The initial values in $r'$ are the same as in r. As $r'_i(m)=r_i(m)$, we have that i decides on ${\mathtt {v}}$ at time m in $r'$ as well. As the initial values in $r'$ are the same as in r, we have that no process has initial value ${\mathtt {v}}$ in $r'$. As i does not fail in $r'$, we therefore have that Validity does not hold regarding the decision of i in $r'$—a contradiction.

We move on to proving (b). Assume for contradiction that some process j decides $\bar{\mathtt {v}}$ at some time $m'\le m$ in r, and that j is active at m in r. Once again by definition of ${\gamma ^{{{\varvec{t}}}}_{\mathrm {cr}}}$, there exists a run $r'$ of P, s.t. 1) $r'_i(m)\!=\!r_i(m)$, 2) $r'_j(m')\!=\!r_j(m')$, and 3) neither i nor j fail in $r'$. As $r'_i(m)=r_i(m)$, we have that i decides on ${\mathtt {v}}$ at time m in $r'$ as well; as $r'_j(m')=r_j(m')$, we have that j decides on $\bar{\mathtt {v}}$ at time $m'$ in $r'$ as well. As neither i not j fail in $r'$, we therefore have that Agreement does not hold in $r'$ — a contradiction. $\square$

The proof of Lemma 3 is assisted by Definition 6 and Lemma 20:

Definition 6

Let P be a protocol in ${\gamma ^{{{\varvec{t}}}}_{\mathrm {cr}}}$ and let $r\in R_P=R(P,{\gamma ^{{{\varvec{t}}}}_{\mathrm {cr}}})$. Let ${\mathtt {v}}\in {\mathtt{V}}$ and let $\langle {i,m}\rangle$ be a node. We say that there is a ${\mathtt {v}}$-chain for $\langle {i,m}\rangle$ in the run r if, for some $d\le m$, there is a sequence $j_0,j_1,\ldots , j_d=i$ of distinct processes, such that $v_{j_0}={\mathtt {v}}$ and for all $1\le k\le d$, the process $j_k$ receives a message from $j_{k-1}$ at time k in r.

Lemma 20

Let P be a fip in ${\gamma ^{{{\varvec{t}}}}_{\mathrm {cr}}}$ and let $r\in R_P=R(P,{\gamma ^{{{\varvec{t}}}}_{\mathrm {cr}}})$. Then, for every processes i and time $m\ge 0$, it is the case that $(R_P,r,m)\models K_i{\exists 0}$ iff there is a 0-chain for $\langle {i,m}\rangle$ in r.

Proof

For the first direction, assume that there is a 0-chain $j_0,\ldots ,j_d=i$ for $\langle {i,m}\rangle$ in r. It is easy to show by induction that $K_{j_k}\exists 0$ at k in r for every k; therefore, $K_i\exists 0$ at d in r, and since P is a fip, $K_i\exists 0$ at m in r, as required. We prove the second direction for all i by induction on m.

Base ($m=0$): Since process i at time 0 knows no initial value but its own, we have that $v_i=0$ and so i (with $d=0$) is a 0-chain as required.

Inductive step ($m>0$): In a fip, $K_i\exists 0$ at m implies that either $K_i\exists 0$ at $m-1$ or $K_j\exists 0$ at $m-1$ for some $j \ne i$ that successfully sends a message at time $m-1$ to j. If $K_i\exists 0$ at $m-1$, then by the induction hypothesis there exists a 0-chain for $\langle {i,m-1}\rangle$ in r, and by definition this is also a 0-chain for $\langle {i,m}\rangle$ in r. It remains to consider the case in which $K_i\exists 0$ does not hold at $m-1$; therefore, $K_j\exists 0$ at $m-1$ for some j that successfully sends a message at time $m-1$ to j. By the induction hypothesis, there exists a 0-chain $j_0,\ldots ,j_d=j$ for $\langle {j,m-1}\rangle$. We first claim that i does not appear in that chain; indeed, if $j_{d'}=i$ for some $d'<d$, then by definition $j_0,\ldots ,j_{d'}$ would be a 0-chain for $\langle {i,m-1}\rangle$, and by the previous direction we would have $K_i\exists 0$ at $m-1$ in r. We now claim that $d=m-1$; indeed, if $d<m-1$, then $j_0,\ldots ,j_d$ would be a 0-chain for $\langle {j,d}\rangle$, and so we would have $K_j\exists 0$ at $d<m-1$. As j is active at all times earlier than $m-1$, we would have that $\langle {j,d}\rangle$ successfully sends a message to i, and so $K_i\exists 0$ at $d+1\le m-1$; as P is a fip, we would therefore have that $K_i\exists 0$ at $m-1$ — a contradiction. As i does not appear in $j_0,\ldots ,j_d$, and as $d=m-1$, by definition $j_0,\ldots ,d_j,i$ is a 0-chain for i, as required. $\square$

Proof of Lemma 3

Assume that $(R_P,r,{{\varvec{t}}}+1)\models K_i{\exists {\mathtt {v}}}$. By Lemma 20, there exists a 0-chain $j_0,\ldots ,j_d$ for $\langle {i,{{\varvec{t}}}+1}\rangle$. If j appears in $j_0,\ldots ,j_d$, then by Lemma 20 we are done; assume, therefore, that j does not appear in $j_0,\ldots ,j_d$. If $d<{{\varvec{t}}}+1$, then since i successfully sends all messages at times earlier than ${{\varvec{t}}}+1$, we have that $j_0,\ldots ,j_d,j$ is a 0-chain for $\langle {j,{{\varvec{t}}}+1}\rangle$; therefore, by Lemma 20, $K_j{\exists {\mathtt {v}}}$ at ${{\varvec{t}}}+1$, as required. Otherwise, $d={{\varvec{t}}}+1$, and so, as $j_0,\ldots ,j_{d-1}$ are ${{\varvec{t}}}+1$ distinct processes, there exists $0\le d'\le d-1$ s.t. $j_{d'}$ is nonfaulty throughout r. Therefore, $j_0,\ldots ,j_{d'},j$ is a 0-chain for $\langle {j,{{\varvec{t}}}+1}\rangle$, as required. $\square$

Proof of Lemma 5

$\Longrightarrow$: Assume that $(R_P,r,m)\not \models K_i{\mathsf {not}{\_}\mathsf {known}(\exists 0)}$. Therefore, by definition of $K_i$, there exists a run $r'\in R_P$ s.t. 1) $r'_i(m)\!=\!r_i(m)$, and 2) $(R_P,r',m)\not \models {\mathsf {not}{\_}\mathsf {known}(\exists 0)}$. As $(R_P,r',m)\not \models {\mathsf {not}{\_}\mathsf {known}(\exists 0)}$, there exists a process j s.t. $K_j\exists 0$ holds at m in $r'$ (and j is active at m in $r'$). By definition, $K_j\exists 0$ first holds at or before time m in $r'$, and so j decides 0 before or at time m in $r'$; therefore, $(R_P,r',m)\not \models {\mathsf {none}{\_}\mathsf {decided}(0)}$. As $r'_i(m)=r_i(m)$, we therefore have $(R_P,r,m)\not \models K_i{\mathsf {none{\_}decided}(0)}$, as required.

$\Longleftarrow$: We will show that $(R_P,r,m)\models {\mathsf {not}{\_}\mathsf {known}(\exists 0)}$ implies that $(R_P,r,m)\models {\mathsf {none{\_}decided}(0)}$; by definition of knowledge, it will then follow that $(R_P,r,m)\models K_i{\mathsf {not}{\_}\mathsf {known}(\exists 0)}$ implies $(R_P,r,m){\models } K_i \mathsf {none{\_}decided}(0)$. Assume, therefore, that $(R_P,r,m)\models {\mathsf {not}{\_}\mathsf {known}(\exists 0)}$, and let j be a process that is active at time m in r. As ${\mathsf {not}{\_}\mathsf {known}(\exists 0)}$ at m in r, we have that $K_j\exists 0$ does not hold at m in r. As P is a fip, we have that neither does $K_j\exists 0$ hold at any time prior to m in r. By definition, therefore j does not decide 0 before or at m in r, as required. $\square$

B Proof of Theorem 4

Decision: In some run of ${{{{u-Opt}}_0}}$, let i be a process and let m be a time s.t. i is active at m but has not decided until m, inclusive. Let $\tilde{m}\le m$ be the latest time not later than m s.t. a hidden path exists w.r.t. $\langle {i,\tilde{m}}\rangle$. We claim that as i is undecided at m, we have $\tilde{m} \ge m-1$; indeed, otherwise, by i being undecided at $\tilde{m}+1$ despite the absence of a hidden path w.r.t. $\langle {i,\tilde{m}+1}\rangle$, we would have $K_i\exists 0$ at $\tilde{m}+1$, and so, by Lemma 11, we would have $K_i{\exists \mathsf {correct}(0)}$ at $\tilde{m}+2\le m$—a contradiction to i being undecided at m.

In Definition 4, for a node $\langle {i,m}\rangle$, we denote by $F\langle {i,m}\rangle \in \{0,\ldots ,t\}$ the number of failures known to $\langle {i,m}\rangle$, i.e., the number of processes $j \ne i$ from which i does not receive a message at time m.

As a hidden path exists w.r.t. $\langle {i,\tilde{m}}\rangle$, we have, as in the proof of Lemma 8, that $\tilde{m}\le {{\varvec{f}}}$; in fact, the same proof shows the even stronger claim $\tilde{m}\le F\langle {i,\tilde{m}}\rangle$ — we we will later return to this inequality. As $\tilde{m}\le {{\varvec{f}}}$, we therefore have that $m\le \tilde{m}+1\le {{\varvec{f}}}+1$. We thus have that every process that is active at time ${{\varvec{f}}}+2$, decides by this time at the latest.

Before moving on to show Validity and Uniform Agreement, we first complete the analysis of stopping times. Assume that $m={{\varvec{f}}}+1$. (i is still a process that is active but undecided at m.) As ${{\varvec{f}}} = m-1 \le \tilde{m} \le F\langle {i,\tilde{m}}\rangle \le F\langle {i,m}\rangle \le {{\varvec{f}}}$, we have that both $\tilde{m}=m-1$ and $F\langle {i,m}\rangle ={{\varvec{f}}}$. As $\tilde{m}=m-1$, we have that no hidden path exists w.r.t. $\langle {i,m}\rangle$. As i is undecided at m, we thus have, by definition of ${{{{u-Opt}}_0}}$ and the fact that a time $\le m$ has been revealed to $\langle {i,m}\rangle$, that $K_i\exists 0$ while $\lnot K_i{\exists \mathsf {correct}(0)}$ at m. We therefore have that $K_i\exists 0$ at m for the first time. Therefore, as $m>\tilde{m}\ge 0$, there exists a process j such that $K_j\exists 0$ at $m-1$ and s.t. $\langle {j,m-1}\rangle$ is seen by $\langle {i,m}\rangle$. Thus, by Lemma 11 and since $\lnot K_i{\exists \mathsf {correct}(0)}$, we have $F\langle {i,m}\rangle <{{\varvec{t}}}-1$, and so ${{\varvec{f}}}= F\langle {i,m}\rangle <{{\varvec{t}}}-1$.

We thus have that if ${{\varvec{f}}}={{\varvec{t}}}-1$, then every process that completes round ${{\varvec{f}}}+2$ decides by time ${{\varvec{f}}}+1$ at the latest.

We move on to show Validity and Uniform Agreement. Henceforth, let i be a (possibly faulty) process that decides in some run of ${{{{u-Opt}}_0}}$, let m be the decision time of i, and let $\mathtt {v}$ be the value upon which i decides.

Validity: If $\mathtt {v}=0$, then by definition $K_i{\exists \mathsf {correct}(0)}$ at m, and so $K_i\exists 0$ at m, and in particular $\exists 0$. If $\mathtt {v}=1$, then by definition $\lnot K_i\exists 0$, and so the initial value of i is 1, and so $\exists 1$. Either way, we have ${\exists {{\mathtt {v}}}}$ as required.

Uniform Agreement: If $\mathtt {v}=0$ then i decides 0 at the first time m such that $K_i{\exists \mathsf {correct}(0)}$ holds. From Lemma 11 we get that $m \ge 1$ and for every process j that is active at time m, it holds that $K_j \exists 0$ at time m, at the latest. Therefore, no process decides 1. We now show that if $\mathtt {v}=1$, then 0 is never decided upon in the current run. For the rest of this proof we assume, therefore, that $\mathtt {v}=1$; therefore, by definition of ${{{{u-Opt}}_0}}$, we have that both $\lnot K_i\exists 0$ and no hidden path exists w.r.t. $\langle {i,m}\rangle$. By Lemma 7, we thus have that $K_i{\mathsf {not}{\_}\mathsf {known}(\exists 0)}$ at m, and in particular ${\mathsf {not}{\_}\mathsf {known}(\exists 0)}$ at m. By a trivial induction, as in the proof of Theorem 2, we have that ${\mathsf {not}{\_}\mathsf {known}(\exists 0)}$ at every time later than m. In particular, we have that no correct process ever learns of an initial value of 0 (as ${\mathsf {not}{\_}\mathsf {known}(\exists 0)}$ would never hold from that point on), and so ${\exists \mathsf {correct}(0)}$ never holds; therefore, $K_j{\exists \mathsf {correct}(0)}$ never holds for any j, and so by definition of ${{{{u-Opt}}_0}}$ no process ever decides upon 0, and the proof is complete.

C Proof of Lemma 19

We first prove Part 1; If $m\!=\!0$, then there exists a run $r'\!=\!Q[\beta ]$ of Q, s.t.  1) $r'_i(0)\!=\!r_i(0)$,  2) in $r'$ all initial values are 0, and  3) i never fails in $r'$. Hence, in $P_0[\beta ]$ all decisions are taken at time $m\!=\!0$, and therefore so is the last decision. Therefore, the last decision in $r'$ must be taken at time 0. As i never fails in $r'$, by Decision it must decide at some point during this run, and therefore must decide at 0 in $r'$. As $r_i(0)\!=\!r'_i(0)$, i decides at 0 in r as well, as required.

If $m\!>\!0$, then there exists a process j s.t. $K_j\exists 0$ at $m-1$ in r and $\langle {j,m-1}\rangle$ is seen by $\langle {i,m}\rangle$. Thus, there exists a run $r'\!=\!Q[\beta ]$ of Q, s.t.  1) $r'_i(m)\!=\!r_i(m)$, and  2) i and j never fail in $r'$. Thus, all processes that are active at m in $r'$ see $\langle {j,m-1}\rangle$ in $r'$ and therefore know $\exists 0$ in $r'$. Hence, in $P_0[\beta ]$ all decisions are taken by time m, and therefore so is the last decision. Therefore, the last decision in $r'$ must be taken no later than at time m. As i never fails in $r'$, by Decision it must decide at some point during this run, and therefore must decide by m in $r'$. As $r_i(m)\!=\!r'_i(m)$, i decides by m in r as well, as required.

We now prove Part 2. If $m\!=\!0$, then by Lemma 11, ${{\varvec{t}}}\!=\!0$. There exists a run $r'\!=\!Q[\beta ]$ of Q, s.t.  1) $r'_i(0)=r_i(0)$, and  2) in $r'$ all initial values are 0. Therefore, as ${{\varvec{t}}}\!=\!0$, we have by Lemma 11 that all processes know ${\exists \mathsf {correct}(0)}$ at $m\!=\!0$ in $r'$. Hence, in ${{{{u-}}P_0}}[\beta ]$ all decisions are taken at time $m\!=\!0$, and therefore so is the last decision. Therefore, the last decision in $r'$ must be taken at time 0 as well. Since ${{\varvec{t}}}\!=\!0$, i never fails in $r'$, and so by Decision it must decide at some point during this run, and therefore must decide at 0 in $r'$. As $r_i(0)\!=\!r'_i(0)$, i decides at 0 in r as well, as required.

If $m\!>\!0$, then there exists a process j s.t. $K_j\exists 0$ at $m\!-\!1$ in r and $\langle {j,m-1}\rangle$ is seen by $\langle {i,m}\rangle$ in r. Furthermore, as ${{\varvec{t}}}\!<\!n$, there exists a set of processes I s.t.  1) $i,j \notin I$,  2) $|I|=t\!-\! F\langle {i,m}\rangle \!-\!1$, and  3) $\langle {k,m\!-\!1}\rangle$ is seen by $\langle {i,m}\rangle$ for every $k \in I$. Thus, there exists a run $r'=Q[\beta ]$ of Q, s.t.  1) $r'_i(m)\!=\!r_i(m)$,  2) i and j never fail in $r'$,  3) all of I fail in $r'$ at $m\!-\!1$, successfully sending messages only to i, and  4) every process at $m\!-\!1$ in $r'$ that is not seen by $\langle {i,m}\rangle$, is not seen by any other process at m as well. We henceforth reason about $r'$. Every process $k \ne j$ that is active at m sees $\langle {j,m\!-\!1}\rangle$ and furthermore satisfies $F\langle {k,m}\rangle \ge F\langle {i,m}\rangle +|I|={{\varvec{t}}}-1$. Thus, by Lemma 11, $K_k{\exists \mathsf {correct}(0)}$ at m, and thus k decides at time m in ${{{{u-}}P_0}}[\beta ]$. Additionally, as $K_j\exists 0$ at $m\!-\!1$, by Lemma 11$K_j{\exists \mathsf {correct}(0)}$ at m, and thus j decides at time m in ${{{{u-}}P_0}}[\beta ]$. Hence, in ${{{{u-}}P_0}}[\beta ]$ all decisions are taken by time m, and therefore so is the last decision. Therefore, the last decision in $r'$ must be taken no later than at time m. As i never fails in $r'$, by Decision it must decide at some point during this run, and therefore must decide by m in $r'$. As $r_i(m)=r'_i(m)$, i decides by m in r as well, as required.