- time spent in class: 3 hours
- Original whitepaper: https://bitcoin.org/bitcoin.pdf
- Chapters 14 and 17 of Elaine Shi's book draft, Foundations of Distributed Consensus and Blockchains.
- Nakamoto's Longest-Chain Wins Protocol (Decentralized Thoughts)
- Lecture notes from David Tse's Internet-Scale Consensus in the Blockchain Era course
- I. Abraham and D. Malkhi, The Blockchain Consensus Layer and BFT, Bulletin of the EATCS, 2017.
- J. A. Garay, A. Kiayias, and N. Leonardos, The Bitcoin Backbone Protocol: Analysis and Applications, EUROCRYPT, 2015.
- R. Pass, L. Seeman, and a. shelat, Analysis of the Blockchain Protocol in Asynchronous Networks, EUROCRYPT, 2017.
Problems 2, 3, 5 here:
In this lecture we are working towards the two bottom results in this table:
✅ [Dolev-Strong] PKI, any f<n ⇒ BB protocol ❌ [PSL, FLM] no PKI, f≥n/3 ⇒ no BB protocol
✅ ❌ [DLS] f<n/3 SMR protocol, favoring safety over liveness in attack mode
❌ [FLP] f=1 ⇒ no BA protocol
✅ [Nakamoto] majority of hashrate is honest (f<n/2) ⇒ non-deterministic SMR protocol
✅ [Nakamoto] majority of hashrate is honest (f<n/2) ⇒ non-deterministic SMR protocol, favoring liveness over safety in attack mode
Contents of the lecture:
- Two Protocol Paradigms
- Bitcoin’s innovations
- Longest-chain consensus (abstractly)
- Longest-chain consensus protocol:
- Honest vs Dishonest Behavior
- Goals of the longest-chain consensus: finality & liveness
- (0) Balanced leaders property: f < n/2 ⇒ random leaders are balanced
- (0) Balanced leaders property ⇒ (1) Common prefix property
- (0) Balanced leaders property ⇒ (2) finality
- (0) Balanced leaders property ⇒ (3) liveness
- Chain Quality: the fraction of honest blocks
- What about partial synchrony?
- Permissionless setting
Two Protocol Paradigms
- BFT-type protocols (e.g. Tendermint)
- canonical tool: multiple rounds of voting to ensure consistency
- prefer consistency over liveness, and achieve consistency always (assuming f < n/3)
- failure mode under attack: stall
- very difficult to resolve forks (due to bugs, f ≥ n/3, etc) in-protocol
- no explicit voting, embrace forks as normal, resolve ambiguity in-protocol
- favor liveness over consistency
- failure mode under attack: big chain reorganizations (e.g. Ethereum Classic)
- enables double-spend attacks
Really, there are two innovations in the bitcoin whitepaper, which together form Nakamoto consensus:
- Longest-chain consensus - which blocks are confirmed. Can be studied in both permissioned and permissionless settings.
- Proof-of-work - who can propose blocks. Enables permissionless setting, given we use the longest-chain consensus. [In more detail, it’s a sybil resistance mechanism, which ensures that too many Byzantine nodes joining network cannot break it. Viewed yet differently, proof-of-work is one-CPU-one-vote.]
To reiterate: primary reason for using longest-chain consensus is to be able to promote it to the permissionless setting.
Longest-chain consensus (abstractly)
- This material is relevant for:
- Longest-chain consensus is primarily studied in the synchronous model (where ∆ bounds msg delays)
- In the (i) PKI setting, this protocol is inferior to Dolev-Strong protocol. Still, it is very useful to study it, so that later transition to permissionless setting is smooth.
(PKI) permissioned + PKI setting (PoW) permissionless, proof-of-work [rmk: here “longest-chain” means chain with the highest total difficulty] (PoS) permissionless, proof-of-stake
Longest-chain consensus protocol:
[for all three settings (PKI), (PoW), (PoS)]
(1) Hard-coded “genesis block”
(2) In each round r=1,2,3,… [Definitions of round in each settting: (PKI): as before, rounds are governed by time; (PoW): rounds are event driven, by blocks getting mined. No shared clock needed! (PoS): same as in (PKI), governed by time]
(2a) One node is chosen as a leader [How chosen in each settting: (PKI): round-robin rotation OR randomly; (PoW): leader is the node that mined the block; (PoS): random selection, probability of being selected is proportional to the stake of the node]
(2b) Leader can create a set of round-r blocks, each with a predecessor block (could be , forks allowed!) [In BFT-type protocols this doesn’t make sense, everyone knows which is the latest block, so there can only be one predecessor.]
Rmk. Blocks form an in-tree:
(A1) (trusted setup) not chosen by, or known in advance to Byzantine nodes.
- Really have to assume this
(A2) (to be enforced) Round-r leader can prove itself to other nodes + non-leaders cannot masquerade as a leader.
- How to impose in each setting: (PKI): since leaders are governed by time, everybody knows who the leader is at every moment; (PoW): leader can prove themselves by presenting the mined block (hash with many zeroes); (PoS): being a leader there = having a proof that you are the leader, by some randomized algorithm.
(A3) (to be enforced) Nodes cannot manipulate their probability of being chosen as leader in (2a)
- How to impose in each setting: (PKI): no problem; (PoW): “random oracle assumption” for hash functions guarantees this; (PoS): much harder here, since no source of randomness in-protocol. Fancy cryptography needed.
(A4) (to be enforced) Every round-r block’s predecessor was created in some previous round.
- Consequences: (i) excludes Byzantine nodes creating a lot of blocks pointing to each other at step (2b) (⇒ no cycles!) (ii) excludes Byzantine nodes doing the following “delaying into the future”: not announce their block at r=17, and then at r=27 announce the block from r=17 and specify the predecessor to be some blocks created between r=18 and r=26. Note: simple delaying is allowed for Byz nodes, ie revealing in 27th round the r=17 block which points to r=13 block. This will be important in selfish mining. (iii) implies that leader can only add one block to any given chain in the tree.
- How to impose in each setting: (PKI): You can have blocks “expire”, meaning honest nodes ignore delayed blocks from Byz nodes, to deal with (ii) above + honest nodes can detect the scenario (i) above; (PoS): Same ideas as in (PKI); (PoW): is automatically true, because node specifies the predecessor before it is notified that it is a leader. In other words, one can mine a block only with a previous block predecessor specified. In fact, PoW guarantees.
(A4’) The leader of a round produces exactly one block, and this block claims as its predecessor some block that belongs to a previous round. [Thiis stronger assumption is not needed, but will make some of the proofs easier]
(A5) (to be relaxed) At all times, all honest nodes know about the exact same set of blocks (and predecessors). [”instant communication mode” or “supersynchronous model”: ∆=0]
- Rmk. This implies consistency trivially, but does not imply finality. Wasn’t the whole point is to sync honest nodes? With this assumption, each time an honest node creates block, that block automatically propogated to all other honest nodes, so consistency trivially holds. BUT Consistency with a future-yourself is not guaranteed (basically that blocks do not get rolled back). We call this property “finality”
- How to impose in each setting: (PKI): (A5) is not big of a deal, each leader rotation we can run Dolev-Strong BB protocol to sync honest nodes… In this model then, our protocol would be very similar to Dolev-Strong protocol, and only will differ from it in the way it treats Byzantine nodes… (PoW): we will be able relax (A5) to the synchronous model, as long as round-time >> ∆ (PoS): we will be able relax (A5) to the synchronous model, as long as round-time >> ∆
Rmk. (A2) and (A3) matter much more in PoW and PoS versions.
Honest vs Dishonest Behavior
Honest node behavior in block proposer step (2b):
(i) Form a block B := all known pending txs
(ii) Set predecessor := end of current longest chain (pick any if several)
(iii) Immediately broadcast B (+ specified predecessor) to all other nodes
Byzantine node: can deviate from any of (i-iii), but must commmit to B & predecessor at the time it’s selected as a leader. This is because “switching predecessors” can very quickly lead to rolling back txs.
Rolling blocks back: k Byzantine leaders in a row can “cancel” last k-1 blocks ⇒ blocks at the tip are “not confirmed”. How long do you have to wait to consider them confirmed? See below!
⇒ f ≥ n/2 ⇒ honest nodes are doomed: 51% Byz nodes ⇒ out of 1000 leaders, about 510 will be Byzantine and 490 honest, and due to 20 nodes excess those 510 nodes can eventually cancel the last 19 nodes on the chain (not as drawn on the right, there maybe a scenario when there are no 20 consecutive Byz leaders, but in total those 510 Byz nodes can outpace those 19 blocks…). Same thing for 10000 leaders, but the result is that 199 blocks can be canceled. Etc.
So here is the summary:
(1) Hard-coded “genesis block”
(2) In each round r=1,2,3,…
(2a) One node is chosen as a leader
(2b) Leader can create a set of round-r blocks, each with a predecessor block (could be , forks allowed!). Honest blocks pick predecessors on the longest-chain. [PoW: longest according to total difficulty]
(A1) not chosen by, or known in advance to Byzantine nodes.
(A2) Round-r leader can prove itself to other nodes + non-leaders cannot masquerade as a leader.
(A3) Nodes cannot manipulate their probability of being chosen as a leader
(A4) Every round-r block’s predecessor was created in some previous round.
(A4’) The leader of a round produces exactly one block, and this block claims as its predecessor some block that belongs to a previous round
(A5) At all times, all honest nodes know about the exact same set of blocks (and predecessors): ∆=0
So here is the summary of the protocol with assumptions embedded:
(1) “Genesis block” hard-coded (A1) by an honest party.
(2) In each round r=1,2,3,…
(2a) One node is (A2+A3) randomly chosen as a leader.
(2b) Leader creates and broadcasts (A4’) one new round-r block, specifying a round-k predecessor block, where (A4) k<r and the predecessor is the tip of the current longest chain. [PoW setting: longest according to total difficulty]
Goals of the longest-chain consensus: finality & liveness
Assumption (A5) (“super-synchronous” model, all messages delivered instantaneously: ∆=0) ⇒ trivializes consistency between nodes, but not self-consistency (since blocks can be rolled back).
Rmk: this assumption reduces clutter that would otherwise block the key ideas. All results extend to the usual synchronous model assuming rate of block production is slow relative to ∆, since this implies that inadvertant honest forks (between honest nodes) rarely occur; see the last two papers in the resources section above]
Def. A block is considered finalized if it belongs to the longest chain and willl not be rolled back.
Ultimate goal for finality: there exists k > 0 such that [f < n/2 ⇒ all but last k blocks on the longest chain can be considered probabilistically finalized]
Notation: := all but last k blocks on longest chain, where is the current in-tree:
Our intention is to consider blocks in finalized.
Note 1: parameter k is not a part of the protocol description.
Note 2: not clear is well-defined. Turns out that if f < n/2 and k>>1, then it is! See below.
(1) Common prefix property: is well-defined
Plan for the proofs: [f < n/2] ⇒ [(0) “sequence of leaders is -balanced for >>1”] ⇒ ⇒ (1) and (2) and (3)
Definition: (”A” stands for an adversarial node, “H” stands for honest node) A sequence is -balanced ( stands for “window”) if, in every window of length ≥, strict majority are honest. Note 1. We want as small as possible (since this controls time of tx confirmation)] Note 2. Conceivably, sequence of leaders is -balanced for sufficiently large, if <51% honest nodes.
Moreover, under the property (0), for every possible sequence of in-trees we have: (1) common prefix property: for all , the prefix is well-defined; (2) finality: (once confirmed, always confirmed) (3) liveness: if a tx is known to all honest nodes, it will eventually be included in
Rmk. We wrote “for every possible” because the sequence is not fully determined because Byz nodes can do bad things, and honest nodes can break ties between longest chains arbitrarily.
(0) Balanced leaders property: f < n/2 ⇒ random leaders are balanced
From the viewpoint of -balanced sequences, round-robin leader selection is not as good as random — so we focus on random choice!
Good news: reasonabe to expect randomly chosen leaders to be balanced.
Bad news: stuck with non-zero (but hopefully astronomically small) failure probability.
⇒ forced to settle for probabilistic guarantees (i.e., have probabilistic finality). [Aside: this an extremely powerful idea in cryptography and computer science: things that are completely impossible become very much possible if we are okay with having 0.9999999 probability of them happening. And this is completely okay for reality, because the chance that an asteroid hits the planet is also non-trivial…]
Notation: = fraction of nodes that are Byzantine. Assume .
Note: for any window of consecutive leaders, expect fraction to be honest (on average)
Intuition: bigger window length ⇒ bigger gap between expected number of honest vs Byzantine nodes ⇒ bigger gap to absorb variation around expectation ⇒ less likely to see ≥50% Byzantine nodes.
Math: this probability is decreasing exponentially with !
Pr[a given length- window is ≥50% Byzantine] ≤ (c = some constant) [This is due to Chernoff bound, see Problem 3 in HW, or this link…]
⇒ Pr[any window of length ≥ is ≥50% Byzantine] ≤ Pr[…] (by a union bound)
⇒ Pr[any window of length ≥ is ≥50% Byzantine] ≤ (where T is the length of a sequence; because window starts and ends somewhere)
⇒ get failure probability ≤ as long as
⇒ get failure probability ≤ as long as where constant depends only on
Since and grow very slowly, we can take that satisfies the above inequality for all conceivable …
(0) Balanced leaders property ⇒ (1) Common prefix property
Rmk. We wrote “any possible in-tree” due to tie-breaking and Byzantine nodes.
Proof: assume the theorem is not true. Then there is an in-tree G that looks as follows:
Trace back to the first B which is honest (possible due to A1). We are doing this since B* could have been produce by Byzantine node (who delayed its production, for example).
Suppose r is the round of B’s creation and announcement.
Now, the claim is that the bad unbalanced window (with ≥50% Byzantine leaders) is the window between r and the moment when the in-tree G is known to everyone.
(def: height of a block = the length of the chain from the block to the genesis block.)
Claims: (that finish the proof)
- ≥2k+2 leaders after r until G
- ≤ h-h honest leaders after r until G
- ≥ h-h Byzantine leaders after r until G
All the blocks pointing to B are created after r, since they needed to know about B ⇒ 2k+2 purple blocks are created after r ⇒ 2k+2 leaders appeared, by the assumption (A4’): qed.
Proposition: the heights of honest blocks strictly increase over time; follows obv from (A5). The statement b. now follows immediately from the proposition.
Well, half of the purple blocks need to be produced by Byzantine nodes + red blocks need to be produced by Byzantine nodes ⇒ we get precisely at least h-h Byzantine nodes.
(0) Balanced leaders property ⇒ (2) finality
Rmk. We can write as opposed to “ known to a node ” because an in-tree known to one honest node is immediately propagated to the others, via assumption (A5).
Proof: suppose the opposite: a block b belongs to but not for some .
⇒ b belongs to every longest chain of (we can even say that it is >k blocks deep there), but does not belong to any longest chains of (if it is >k blocks deep, then it is in which is not true; if it is ≤k blocks deep, then this contradicts with b being in )
Consider now the sequence of in-trees , each time one block is added (if Byz node adds multiple blocks, we view it as multiple in-trees)
There needs to be the first (in the sequence ,) in-tree when the block b is excluded from the longest chain:
(The content of Theorem 1 above was precisely the fact that such structures as above cannot arise…)
⇒ has two longest chains that disagree on last ≥k+1 blocks ⇒ is not well-defined, contradicting Theorem 1.
(0) Balanced leaders property ⇒ (3) liveness
Proof: define an epoch as 2k consecutive rounds (+ their leaders).
By 2k-balancedness, ≥k+1 honest leaders ⇒ ≤k-1 Byzantine leaders in each epoch.
Note 1. Every honest leader adds 1 to the length of longest chain.
Note 2. Each Byzantine leader contributes ≤1 block to any given chain, by assumptions (A4).
⇒ after T epochs, longest chain contains ≥(k+1)T blocks, of which ≤(k-1)T are contributed by Byzantine nodes
⇒ ≥ 2T blocks on longest chain are produced by honest nodes (⇒ honest blocks are added to the longest chain infinitely often)
⇒ ≥ 2T-k of these blocks have been finalized
⇒ honest blocks are finalized inifinitely often
⇒ a tx* known to all honest nodes will eventually be finalized! [This is true since after each honest block is added the tx* will be in the longest chain (either in the past or in the new block)]
Chain Quality: the fraction of honest blocks
Possible stronger liveness guarantees:
(i) assume tx known to only one honest node; but then Theorem 3 becomes false!
(ii) quantitative bounds on time-till-finalization (research on this exists)
(iii) lower bound fraction on honestly produced bllocks on the longest chain
Revised analysis of randomly chosen leader, property (0)
As long as , we have
Pr[any window of length ≥w has ≤() fraction of Byzantine leaders] ≤
⇒ leader sequence is ()-balanced with probability ≥1-
(Thm 3 is a special case of above, where is very close (and slightly less than) to 1/2, then tends to 0)
Proof: is very similar to Thm 3:
… ⇒ after T epochs, longest chain contains ≥2k(1 )T blocks, of which ≤2k()T are contributed by Byzantine nodes
⇒ ≥ = fraction of blocks is honestly produced.
What about partial synchrony?
We had synchronous model with ∆=0, but as wee said, this can be extended to ∆≠0 provided the average block time is >>∆, which guarantees that honest nodes don’t produce accidental forks too often.
If we want to impose partial synchrony and consider what happens to longest-chain consensus in an attack, some of the theorems above do not work:
Note: finality (Theorem 2) of longest-chain consensus breaks down in partial synchrony, even if all nodes are honest (f=0)! This is because the adversarial message delivery can model a network partition, where the sat of nodes A does not talk to the set of nodes C, and therefore they operate independently:
⇒ once the length of two forks exceeds k, we lose the common prefix property, and therefore it doesn’t even make sense to talk about finality. In particular, if one fork is shorter than the other (let’s say the one below, created by nodes C), when the network partition ends finality for nodes C truly breaks, since they have to throw out their whole chain.
Note: longest-chain consensus still makes progress with network partition!
In a nutshell, under attack we have lost consistency (finality, to be precise: the blocks can roll back, arbitrarily large number of them), but did not lose liveness (see below). In contrast to this, BFT-type protocol (e.g. Tendermint) in case of such a network partition, will just stall.
⇒ “longest-chain favors liveness over safety” (fail via reorgs) “BFT-type protocols favor safet over liveness” (fail by stalling)
BFT-type protocols wouldn’t quite work right away, since they are based on voting, and nodes wouldn’t know the number of nodes in advance, and therefore they wouldn’t know the threshold.
This problem can be circumvented by “wrapping” the BFT-type protocol via some sort of authentification of nodes (see ETH 2.0 design, for example, where validators do know about each other).
Longest-chain consensus, in contrast, extends to permissionless setting amazingly well: (Nakamoto probably invented longest-chain consensus presiccely to be able to extend it to permissionless setting.)
(1) “Genesis block” hard-coded by an honest party.
(2) In each round r=1,2,3,…
(2a) One node is randomly selected as a leader.
(2b) Leader creates and broadcasts one new round-r block, specifying a round-k predecessor block, where k<r and the predecessor is the tip of current longest chain. [PoW setting: longest according to total difficulty]
Key points for transition:
- Balanced leadder sequence ⇒ consistency + liveness, and here we did not really care about being permissioned / permissionless.
- Proof-of-work (see Lecture3) allows to substitute one-node-one-vote by one-CPU-one-vote, and allows us to have f<n/2 in the premissionless setting, provided the majority of computational power is honest!
- [the only subtlety]
⇒ we only need to take care of Theorem (0), where we proved that the leader sequence is w-balanced for some w>0 provided f<n/2.
More precisely, as long as the majority of hashrate is honest, we get that in each round r Pr[round-r leader is Byzantine]≤ for some <1/2, and therefore the leader sequence will be w-balanced with high probability (parameter w increases as tends to 1/2).
The probability of being chosen as a leader now depends on the difficulty number of the previous block, and therefore, in order to preserve assumption
“(A3) Nodes cannot manipulate their probability of being chosen as a leader”
we need “longest chain” to mean longest with respect to total difficulty.