Lectures 10. Longest-chain consensus.

In this lecture we are working towards the two bottom results in this table:

Contents of the lecture:

• Two Protocol Paradigms
• Bitcoin’s innovations
• Longest-chain consensus (abstractly)
• Longest-chain consensus protocol:
• Assumptions:
• Honest vs Dishonest Behavior
• Goals of the longest-chain consensus: finality & liveness
• (0) Balanced leaders property: f < n/2 ⇒ random leaders are balanced
• (0) Balanced leaders property ⇒ (1) Common prefix property
• (0) Balanced leaders property ⇒ (2) finality
• (0) Balanced leaders property ⇒ (3) liveness
• Chain Quality: the fraction of honest blocks
• What about partial synchrony?
• Permissionless setting

Two Protocol Paradigms

1. BFT-type protocols (e.g. Tendermint)
• canonical tool: multiple rounds of voting to ensure consistency
• prefer consistency over liveness, and achieve consistency always (assuming f < n/3)
• failure mode under attack: stall
• very difficult to resolve forks (due to bugs, f ≥ n/3, etc) in-protocol
2. Longest-chain protocols (e.g. Bitcoin, Ethereum)
• no explicit voting, embrace forks as normal, resolve ambiguity in-protocol
• favor liveness over consistency
• failure mode under attack: big chain reorganizations (e.g. Ethereum Classic)
• enables double-spend attacks

Bitcoin’s innovations

Really, there are two innovations in the bitcoin whitepaper, which together form Nakamoto consensus:

• Longest-chain consensus - which blocks are confirmed. Can be studied in both permissioned and permissionless settings.
• Proof-of-work - who can propose blocks. Enables permissionless setting, given we use the longest-chain consensus. [In more detail, it’s a sybil resistance mechanism, which ensures that too many Byzantine nodes joining network cannot break it. Viewed yet differently, proof-of-work is one-CPU-one-vote.]

To reiterate: primary reason for using longest-chain consensus is to be able to promote it to the permissionless setting.

Longest-chain consensus (abstractly)

• This material is relevant for:

(PKI) permissioned + PKI setting (PoW) permissionless, proof-of-work [rmk: here “longest-chain” means chain with the highest total difficulty] (PoS) permissionless, proof-of-stake

• Longest-chain consensus is primarily studied in the synchronous model (where ∆ bounds msg delays)
• In the (i) PKI setting, this protocol is inferior to Dolev-Strong protocol. Still, it is very useful to study it, so that later transition to permissionless setting is smooth.

Longest-chain consensus protocol:

[for all three settings (PKI), (PoW), (PoS)]

(1) Hard-coded “genesis block” $B_0$

(2) In each round r=1,2,3,… [Definitions of round in each settting: (PKI): as before, rounds are governed by time; (PoW): rounds are event driven, by blocks getting mined. No shared clock needed! (PoS): same as in (PKI), governed by time]

(2a) One node is chosen as a leader [How chosen in each settting: (PKI): round-robin rotation OR randomly; (PoW): leader is the node that mined the block; (PoS): random selection, probability of being selected is proportional to the stake of the node]

(2b) Leader can create a set of round-r blocks, each with a predecessor block (could be $B_0$, forks allowed!) [In BFT-type protocols this doesn’t make sense, everyone knows which is the latest block, so there can only be one predecessor.]

Rmk. Blocks form an in-tree:

Assumptions:

(A4’) The leader of a round produces exactly one block, and this block claims as its predecessor some block that belongs to a previous round. [Thiis stronger assumption is not needed, but will make some of the proofs easier]

Rmk. (A2) and (A3) matter much more in PoW and PoS versions.

Honest vs Dishonest Behavior

Honest node behavior in block proposer step (2b):

(i) Form a block B := all known pending txs

(ii) Set predecessor := end of current longest chain (pick any if several)

(iii) Immediately broadcast B (+ specified predecessor) to all other nodes

Byzantine node: can deviate from any of (i-iii), but must commmit to B & predecessor at the time it’s selected as a leader. This is because “switching predecessors” can very quickly lead to rolling back txs.

Rolling blocks back: k Byzantine leaders in a row can “cancel” last k-1 blocks ⇒ blocks at the tip are “not confirmed”. How long do you have to wait to consider them confirmed? See below!

⇒ f ≥ n/2 ⇒ honest nodes are doomed: 51% Byz nodes ⇒ out of 1000 leaders, about 510 will be Byzantine and 490 honest, and due to 20 nodes excess those 510 nodes can eventually cancel the last 19 nodes on the chain (not as drawn on the right, there maybe a scenario when there are no 20 consecutive Byz leaders, but in total those 510 Byz nodes can outpace those 19 blocks…). Same thing for 10000 leaders, but the result is that 199 blocks can be canceled. Etc.

So here is the summary:

So here is the summary of the protocol with assumptions embedded:

Goals of the longest-chain consensus: finality & liveness

Assumption (A5) (“super-synchronous” model, all messages delivered instantaneously: ∆=0) ⇒ trivializes consistency between nodes, but not self-consistency (since blocks can be rolled back).

Rmk: this assumption reduces clutter that would otherwise block the key ideas. All results extend to the usual synchronous model assuming rate of block production is slow relative to ∆, since this implies that inadvertant honest forks (between honest nodes) rarely occur; see the last two papers in the resources section above]

Def. A block is considered finalized if it belongs to the longest chain and willl not be rolled back.

Ultimate goal for finality: there exists k > 0 such that [f < n/2 ⇒ all but last k blocks on the longest chain can be considered probabilistically finalized]

Notation: $B_k(G)$ := all but last k blocks on longest chain, where $G$ is the current in-tree:

Our intention is to consider blocks in $B_k(G)$ finalized.

Note 1: parameter k is not a part of the protocol description.

Note 2: not clear $B_k(G)$ is well-defined. Turns out that if f < n/2 and k>>1, then it is! See below.

Plan for the proofs: [f < n/2] ⇒ [(0) “sequence of leaders is $w$-balanced for $w$>>1”] ⇒ ⇒ (1) and (2) and (3)

Definition: (”A” stands for an adversarial node, “H” stands for honest node) A sequence $\ell_1,\ell_2,\ell_3,\ldots\in\{H,A\}$ is $w$-balanced ($w$ stands for “window”) if, in every window $\ell_i,\ell_{i+1},\ldots,\ell_j$ of length ≥$w$, strict majority are honest. Note 1. We want $w$ as small as possible (since this controls time of tx confirmation)] Note 2. Conceivably, sequence of leaders is $w$-balanced for $w$ sufficiently large, if <51% honest nodes.

Rmk. We wrote “for every possible” because the sequence is not fully determined because Byz nodes can do bad things, and honest nodes can break ties between longest chains arbitrarily.

(0) Balanced leaders property: f < n/2 ⇒ random leaders are balanced

From the viewpoint of $w$-balanced sequences, round-robin leader selection is not as good as random — so we focus on random choice!

Good news: reasonabe to expect randomly chosen leaders to be balanced.

Bad news: stuck with non-zero (but hopefully astronomically small) failure probability.

⇒ forced to settle for probabilistic guarantees (i.e., have probabilistic finality). [Aside: this an extremely powerful idea in cryptography and computer science: things that are completely impossible become very much possible if we are okay with having 0.9999999 probability of them happening. And this is completely okay for reality, because the chance that an asteroid hits the planet is also non-trivial…]

Notation: $\alpha$ = fraction of nodes that are Byzantine. Assume $\alpha<\frac 1 2$.

Note: for any window of consecutive leaders, expect $1-\alpha>\frac 1 2$ fraction to be honest (on average)

Intuition: bigger window length $w$ ⇒ bigger gap $(1-2\alpha)w$ between expected number of honest vs Byzantine nodes ⇒ bigger gap to absorb variation around expectation ⇒ less likely to see ≥50% Byzantine nodes.

Math: this probability is decreasing exponentially with $w$!

Pr[a given length-$w$ window is ≥50% Byzantine] ≤ $e^{-cw}$ (c = some constant) [This is due to Chernoff bound, see Problem 3 in HW, or this link…]

⇒ Pr[any window of length ≥$w$ is ≥50% Byzantine] ≤ $\sum_w$Pr[…] (by a union bound)

⇒ Pr[any window of length ≥$w$ is ≥50% Byzantine] ≤ $T^2$$e^{-c(\alpha)w}$ (where T is the length of a sequence; $T^2$ because window starts and ends somewhere)

⇒ get failure probability ≤$\delta$ as long as $w\geq \frac {1} {c(\alpha)} (2\ln T - \ln \delta)$

⇒ get failure probability ≤$\delta$ as long as $w\geq c_2(\alpha)(\ln T + \ln \frac 1 \delta)$ where constant $c_2(\alpha)$ depends only on $\alpha$

Since $\ln T$ and $\ln \frac 1 \delta$ grow very slowly, we can take $w$ that satisfies the above inequality for all conceivable $T$…

(0) Balanced leaders property ⇒ (1) Common prefix property

Rmk. We wrote “any possible in-tree” due to tie-breaking and Byzantine nodes.

Proof: assume the theorem is not true. Then there is an in-tree G that looks as follows:

Trace back to the first B$_0$ which is honest (possible due to A1). We are doing this since B* could have been produce by Byzantine node (who delayed its production, for example).

Suppose r$_0$ is the round of B$_0$’s creation and announcement.

Now, the claim is that the bad unbalanced window (with ≥50% Byzantine leaders) is the window between r$_0$ and the moment when the in-tree G is known to everyone.

(def: height of a block = the length of the chain from the block to the genesis block.)

Claims: (that finish the proof)

1. ≥2k+2 leaders after r$_0$ until G

All the blocks pointing to B$_0$ are created after r$_0$, since they needed to know about B$_0$ ⇒ 2k+2 purple blocks are created after r$_0$ ⇒ 2k+2 leaders appeared, by the assumption (A4’): qed.

2. ≤ h-h$_0$ honest leaders after r$_0$ until G

Proposition: the heights of honest blocks strictly increase over time; follows obv from (A5). The statement b. now follows immediately from the proposition.

3. ≥ h-h$_0$ Byzantine leaders after r$_0$ until G

Well, half of the purple blocks need to be produced by Byzantine nodes + red blocks need to be produced by Byzantine nodes ⇒ we get precisely at least h-h$_0$ Byzantine nodes.

qed.

(0) Balanced leaders property ⇒ (2) finality

Rmk. We can write $G_i$ as opposed to “$G_i$ known to a node $j$” because an in-tree known to one honest node is immediately propagated to the others, via assumption (A5).

Proof: suppose the opposite: a block b belongs to $B_k(G_i)$ but not $B_k(G_j)$ for some $j>i$.

⇒ b belongs to every longest chain of $G_i$ (we can even say that it is >k blocks deep there), but does not belong to any longest chains of $G_j$ (if it is >k blocks deep, then it is in $B_k(G_j)$ which is not true; if it is ≤k blocks deep, then this contradicts with b being in $B_k(G_i)$)

Consider now the sequence of in-trees $G_i,G_{i+1}, \ldots, G_j$, each time one block is added (if Byz node adds multiple blocks, we view it as multiple in-trees)

There needs to be the first (in the sequence $G_i,G_{i+1}, \ldots, G_j$,) in-tree $G_h$ when the block b is excluded from the longest chain:

(The content of Theorem 1 above was precisely the fact that such structures as above cannot arise…)

⇒ $G_h$ has two longest chains that disagree on last ≥k+1 blocks ⇒ $B_k(G_h)$ is not well-defined, contradicting Theorem 1.

qed.

(0) Balanced leaders property ⇒ (3) liveness

Proof: define an epoch as 2k consecutive rounds (+ their leaders).

By 2k-balancedness, ≥k+1 honest leaders ⇒ ≤k-1 Byzantine leaders in each epoch.

Note 1. Every honest leader adds 1 to the length of longest chain.

Note 2. Each Byzantine leader contributes ≤1 block to any given chain, by assumptions (A4).

⇒ after T epochs, longest chain contains ≥(k+1)T blocks, of which ≤(k-1)T are contributed by Byzantine nodes

⇒ ≥ 2T blocks on longest chain are produced by honest nodes (⇒ honest blocks are added to the longest chain infinitely often)

⇒ ≥ 2T-k of these blocks have been finalized

⇒ honest blocks are finalized inifinitely often

⇒ a tx* known to all honest nodes will eventually be finalized! [This is true since after each honest block is added the tx* will be in the longest chain (either in the past or in the new block)]

Chain Quality: the fraction of honest blocks

Possible stronger liveness guarantees:

(i) assume tx known to only one honest node; but then Theorem 3 becomes false!

(ii) quantitative bounds on time-till-finalization (research on this exists)

(iii) lower bound fraction on honestly produced bllocks on the longest chain

Revised analysis of randomly chosen leader, property (0)

As long as $w\geq c(\epsilon)(\ln T + \ln \frac 1 \delta)$, we have

Pr[any window of length ≥w has ≤($\alpha+\epsilon$) fraction of Byzantine leaders] ≤ $\delta$

⇒ leader sequence is ($w,\alpha+\epsilon$)-balanced with probability ≥1-$\delta$

(Thm 3 is a special case of above, where $\alpha+\epsilon$ is very close (and slightly less than) to 1/2, then $\frac{1-2\alpha-2\epsilon}{1-\alpha-\epsilon}$ tends to 0)

Proof: is very similar to Thm 3:

… ⇒ after T epochs, longest chain contains ≥2k(1 $-\alpha-\epsilon$)T blocks, of which ≤2k($\alpha+\epsilon$)T are contributed by Byzantine nodes

⇒ ≥ $\frac{2k(1-\alpha-\epsilon)T-2k(\alpha+\epsilon)T}{2k(1-\alpha-\epsilon)T}$=$\frac{1-2\alpha-2\epsilon}{1-\alpha-\epsilon}$ fraction of blocks is honestly produced.

qed.

What about partial synchrony?

We had synchronous model with ∆=0, but as wee said, this can be extended to ∆≠0 provided the average block time is >>∆, which guarantees that honest nodes don’t produce accidental forks too often.

If we want to impose partial synchrony and consider what happens to longest-chain consensus in an attack, some of the theorems above do not work:

Note: finality (Theorem 2) of longest-chain consensus breaks down in partial synchrony, even if all nodes are honest (f=0)! This is because the adversarial message delivery can model a network partition, where the sat of nodes A does not talk to the set of nodes C, and therefore they operate independently:

⇒ once the length of two forks exceeds k, we lose the common prefix property, and therefore it doesn’t even make sense to talk about finality. In particular, if one fork is shorter than the other (let’s say the one below, created by nodes C), when the network partition ends finality for nodes C truly breaks, since they have to throw out their whole chain.

Note: longest-chain consensus still makes progress with network partition!

In a nutshell, under attack we have lost consistency (finality, to be precise: the blocks can roll back, arbitrarily large number of them), but did not lose liveness (see below). In contrast to this, BFT-type protocol (e.g. Tendermint) in case of such a network partition, will just stall.

⇒ “longest-chain favors liveness over safety” (fail via reorgs) “BFT-type protocols favor safet over liveness” (fail by stalling)

Permissionless setting

BFT-type protocols wouldn’t quite work right away, since they are based on voting, and nodes wouldn’t know the number of nodes in advance, and therefore they wouldn’t know the threshold.

This problem can be circumvented by “wrapping” the BFT-type protocol via some sort of authentification of nodes (see ETH 2.0 design, for example, where validators do know about each other).

Longest-chain consensus, in contrast, extends to permissionless setting amazingly well: (Nakamoto probably invented longest-chain consensus presiccely to be able to extend it to permissionless setting.)