Lecture 5. The SMR and BB consensus problems and the Dolev-Strong protocol.

‣

Resources:

Primary:

l2.pdf499.1KB

time spent in class: 3 hours https://decentralizedthoughts.github.io/2019-10-15-consensus-for-state-machine-replication/

https://decentralizedthoughts.github.io/2019-12-22-dolev-strong/

http://www.cs.umd.edu/~jkatz/gradcrypto2/NOTES/lecture26.pdf (Section 3)

http://elaineshi.com/docs/blockchain-book.pdf (Chapters 3 & 6)

‣

diagrams:

flowchart LR
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

subgraph Nodes
direction LR
N1("<b>Node 1.</b> History:\ntxA, txB\n\n")
N2("<b>Node 2.</b> History:\ntxA, txB, txC ...\n\n")
N3("<b>Node 3.</b> History:\ntxA\n\n")
end
Nodes:::blue
Cl1("Client 1"):::green-->|tx| Nodes
Cl2("Client 2"):::green -->|tx| Nodes
Cl3("Client 3"):::green -->|tx| Nodes
flowchart LR
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

subgraph Nodes
direction LR
N1("<b>Node 1.</b> History:\ntxA, txB\n\n")
N2("<b>Node 2.</b> History:\ntxA, txB, txC ...\n\n")
N3("<b>Node 3.</b> History:\ntxA\n\n")
end
Nodes:::blue
Cl1("Client 1"):::green-->|tx| Nodes
Cl2("Client 2"):::green -->|tx| Nodes
Cl3("Client 3"):::green -->|tx| Nodes

‣

HW:

Problem 1. (15 points) In Lecture 2 we studied the Byzantine broadcast (BB) problem, where a single node (the sender) has a private input and the goal is to broadcast that input to everyone else. For this problem, you can assume that the private input is either 0 or 1.

A closely related problem is Byzantine agreement (BA). Here, each node i ∈ {1, 2, . . . , n} has its own private bit $b_i$ ∈ {0, 1} (not only a sender!). Up to f of the n nodes may be Byzantine, meaning they can behave in arbitrary ways. We are interested in deterministic BA protocols that satisfy the following two properties:

Agreement: the protocol always terminates with all honest nodes outputting the same bit.
Validity: if all honest nodes have the same private input b ∈ {0, 1}, then the protocol terminates with all such nodes outputting b.

We’ll work in the same synchronous model used in Lectures 2 and 3.

(a) (5 points) Give one deterministic BA protocol that satisfies agreement (no matter what f is), and another that satisfies validity (no matter what f is). Prove that your protocols satisfy the property in question.

(b) (10 points) Suppose f < n/2. Prove that there exists a deterministic BB protocol satisfying validity and agreement (as defined for the BB problem) if and only if there exists a deterministic BA protocol satisfying validity and agreement (as defined for the BA problem).

Problem 2. (30 points) In this problem, we continue to study the Byzantine broadcast problem. (You can continue to assume that the sender’s private input is either a 0 or a 1.) You can assume n ≥ 3.

Consider the following protocol (to be executed by all honest nodes): t = 0: The sender sends its private bit (along with its signature) to all other nodes. The sender then outputs its own private bit and terminates. t = 1: Every non-sender node i echoes what it heard from the sender to all the other non-sender nodes (with i’s signature added). t = 2: Every non-sender node collects all the votes it received (up to n − 1 votes, with at most one from the sender in round 0 and at most one from each non-sender node from round 1) and chooses its output by majority vote. (If there’s an equal number of votes for both 0 and 1, output 0.) If a node submits multiple votes for different values, all votes by that node are ignored.

For each of parts (a)–(f), provide either a convincing proof or an explicit counterexample. (a) (5 points) Suppose f = 1 (i.e., at most one node is Byzantine, and all the rest are honest). Does this protocol satisfy agreement? (b) (5 points) Suppose f = 1. Does this protocol satisfy validity? (c) (5 points) Suppose f = 2. Does this protocol satisfy agreement? (d) (5 points) Suppose f = 2. Does this protocol satisfy validity? (e) (5 points) Recall the Dolev-Strong protocol from Lecture 2. Suppose we stop that protocol one time step earlier (with nodes committing to an output at time f rather than time f + 1), without changing anything else. Does the protocol continue to satisfy agreement? (f) (5 points) Does the protocol in (e) continue to satisfy validity?

Having covered digital signatures, we now understand why we can trust that each tx in a blockchain comes from the particular public address — simply because it was signed by a private key associated to that public address. (Remember that public address ~= public key)

We now start addressing another core component of blockchains: the consensus mechanism. So far we have covered the basics of PoW and PoS without much details. We now wish to build a certain framework around consensus mechanisms, so that we can reason about them mathematically. We start with more basic consensus protocols, in the permissioned setting, meaningn that the set of nodes is known in advance. [PoW and PoS are quite advanced consensus protocols in the permissionless setting, meaning anyone can become a node/miner/validator]

Permanent assumptions from now on:

Internet exists, and serves as a comunication layer for nodes.
Cryptography exists:

Hash functions
Digital signatures

The SMR problem

State Machine Replication in 80’s is concerned with a problem of keeping several data centers in sync, which itself is motivated by the need for backups. […]

It is a really interesting fact that the problem of decentralizations in blockchains from 2008 is extremely similar to the SMR problem.

SMR problem [“data centers trying to stay in sync”]

Clients submit txs to nodes (data centers)
Each node maintains local history (append only data structure)

Goal: a protocol, which is an event-driven code to be run by nodes, that satisfies:

Consistency (or safety): all nodes agree on their histories [for each pair of nodes history of one is a prefix of the other; in other words, lagging is okay, but different ordering of txs is not]
Liveness: every valid submitted tx is eventually added to all nodes’ history.

Plan: introduce strong assumptions, solve the SMR problem, and then relax assumptions one-by-one and see whether the SMR problem is still solvable.

Assumptions

Permissioned setting

Each node knows the set of all nodes: {1,2,…,n}

It makes sense for SMR, but not for blockchains. Nonetheless, it is highly useful to study and extrapolate from here.

Public Key Infranstructure (PKI)

All nodes have public-private key pairs
Each node knows public keys of all other nodes (this is an example of a trusted setup assumption)

We will not be relaxing this condition that much. Note that Bitcoin and initial Ethereum don’t have this assumption.

Synchronous setting

Existence of a shared global clock 1___2___3___4___5___6___7___→
Bounded message delays (if sent at time t, message arrives by t+1; critical assumption that we will want to relax)

Rmk. This assumption obviously fails IRL for network outages and DoS attacks. Later we will see that relaxing synchronous assumption (asynchronous setting) breaks either liveness or safety — thus, when evaluating blockchains, one of the key questions is what happens in case of prolonged network outage. Eventually, we will work in a so-called partially synchronous setting.

All honest nodes

All nodes run the intended protocol (no bugs, no downtime, no malicious behavior)

Ridiculous assumption even for SMR, will start relaxing very soon.

Solving SMR via simple the “rotating leaders” protocol

If nodes don’t communicate, then the protocol fails to solve the SMR problem if client don’t send txs to all nodes ⇒ nodes need to communcate.

Coordination via rotating leaders

Node $k$ is a leader at times k, n+k, 2n+k, …
All nodes know this, since we are in permissioned setting
Leader node at time t does broadcasting:

Collects all not-yet-included txs, and orders them arbitrarily
Sends the ordered list of txs to every other node

Since we are in synchronous setting, all nodes have the new list by time t+1, and they append them.

Proposition. This protocol satisfies conssitency and liveness, under all 4 assumptions in the previous section.

Proof: […]

Introducing Faulty/Byzantine nodes

Honest node = never deviates from the protocol (intentionally or not intentionally)
Falty node = not honest node

Liveness and consistency properties have to be tweaked: every valid submitted tx is eventually added to all honest nodes’ history, and histories have to agree only for honest nodes.

Types of faults:

Crash faults (hardware errors in SMR)

Breaks the rotating leaders protocol if crash happens in the middle of leader’s broadcast, in which case consistency fails. Otherwise, protocol is fine, if liveness goal is tweaked properly.

Omission faults (network outage in SMR)

Seriously breaks the rotating leaders protocol, since leader can send txs only to some nodes.

Byzantine faults = arbitrary/malicious deviations! (software error in SMR)

Obviously breaks the rotating leaders protocol.

❗

We now relax [Assumption 4], by allowing f Byzantine node (0<f<n)

The Byzantine Broadcast Problem

Rotating leaders protocol works if f=0, but fails if f≥1

Idea: keep rotating leaders, but add a cross-checking subroutine

Byzantine Broadcast problem. - One node is a sender, others are non-senders - Sender has a private input

v^* \in V

(for us this is an ordered list of txs) - The goal is, as before, a certain protocol. Desired Properties of the Byzantine Broadcast protocol: 1. Termination. Every honest node

i

eventually halts with some output

v_i \in V

, node’s best guess for what

v^*

is. 2. Agreement. All honest nodes halt with the same output. (~ safety property) 3. Validity. If the sender is an honest node, then the common output of the honest nodes is the private input

v^*

of the sender. (~ liveness property)

Rmk 1. (1&2) or (1&3) are easily achieved. As before, (1&2&3) is what’s difficult (both safety and liveness!)

Rmk 2. In BB, there is one output ⇒ BB is a single-shot consensus problem, in contrast to SMR, which is a multi-shot consensus proble,.

SMR reduces to Byzantine Broadcast

Assumptions: synchronos and permissioned setting (PKI also, but not important here)

Given: a protocol $\pi$ for the Byzantine Broadcast problem, which with ≤f Byzantine nodes satisfies Termination+Validity+Agreement in at most T time steps.

Reduction: at each time step 0, T, 2T, … :

Define a leader using round-robin ordering (0→ node 1, T→ node 2, 2T→ node 3, …)
Leader assembles not-yet-included txs into an ordered list $L^*$
Invokes subroutine $\pi$ with leader = sender and $v^*=L^*$
When $\pi$ terminates, every node $i$ appends output $L_i=v_i$ to its local history

Theorem. SMR protocol above satisfies consistency (restricted to honest nodes) and liveness (restricted to honest nodes).

Proof: […]

Solving Byzantine Broadcast when f=1

If f=1, the cross-checking subroutine $\pi$ can be as follows:

Sender sends $v^*$ to all non-senders
Every non-sender $i$ echoes the message $m_i$ to all non-senders
Each non-senders picks most frequently referenced message in step 2.

Theorem. If f=1 and n≥4, then the simple cross-checking protocol above solves the BB, i.e. satisfies Termination+Aggreement+Validity.

Proof: […]

Bad example when f=2, n≥4, n is even

As a result, the left (n-2)/2 honest nodes each receive $n/2$ votes for “0” and $n/2-1$ votes for “1”, so they choose “0” as their $v_i$ . Similarly the right (n-2)/2 honest nodes each receive $n/2$ votes for “1” and $n/2-1$ votes for “0”, so they choose “1” as their $v_i$ .

This collusion shows well how easy it is to break consistency
One can imagine a ton of other ways Byzantine nodes can collude
More rounds of cross-checking do help! See below.

The Dolev-Strong Protocol ‘83

[solution to BB for any f, under PKI+synchronous+permissioned assumptions]

Assumptions: synchronous and permissioned setting, PKI.

Definition. Node $i$ is convinced of value $v$ at time $t$ if it receives a message that: - references the value $v$ ; - signed first by the sender - signed also by ≥t-1 other distinct nodes $\neq i$ (in other words, it needs t-1 attestations; signatures are nested) Once the node is convinced of $v$ , it can no longer be “unconvinced” of $v$ , but it can be convinced of other values.

Example: if node7 at t=3 receives ((( $v$ ,sender’s sign), node5 sign), node2 sign), it is convinced of value $v$ at time 3.

Dolev-Strong Protocol. (to be run by honest nodes to solve BB) t=0: sender sends ( $v^*$ , sign) to all nodes. t=1, 2, 3, …, f+1: (f is known upfront, and can be any) if node $i$ is convinced of value $v$ at this time step by a message $m$ , it adds $i$ ’s signature and echoes ( $m,sign_i$ ) to all other nodes. Final output: if node is convinced (in any of the previous steps) of exactly one value $v$ , then it outputs $v$ . Otherwise, if convinced of 0 or ≥2 values, outputs $\perp$ (some default value)

Remark. In SMR context, this protocol is useful only if f < n/2, because otherwise at any moment the conflict cannot be resolved by a majority vote, and Byzantine nodes simply take over.

Theorem. (Dolev-Strong ‘83) Under current assumptions, this protocol for BB satisfies validity & agreement.

Proof of validity and agreement in Dolev-Strong

We start with validity: - at $t=0$ , honest sender sends $(v^*,sign)$ $v^*,sign)$ to all nodes - at $t=1$ , all honest nodes are convinced of $v^*$ , so they send $((v^*,sign), sign)$ to all nodes - no other (than $v^*$ ) convincing messages can be receieved at t=2, … , t=f+1 since digital signatures are not forgable (and you can also assume that a timestampt t=0 is included in the message of the sender, so that previous signed messages of a sender cannot be used)

We now proceed to proving agreement. We care about Byzantine sender case, and we will prove that all honest nodes are convinced by the same set of messages (this guarantees the same output).

Lemma. If an honest node $i$ gets convinced of a value $v$ , then all honest nodes are convinced by $v$ by the end of the protocol.

Proof: case 1: honest node $i$ gets convinced of $v$ by message $m$ at time t<f+1. In this case node $i$ will send a message $(m,sign_i)$ to all other honest nodes, and that message will convince them of $v$ .

case 2: honest node $i$ gets convinced of $v$ by message $m$ at time t=f+1, the end of the protocol. But this means that at least one honest node $j$ node signed $v$ at some time t=k, since f+1 signatures were received by node $i$ , and there is ≤f Byzantine nodes, and therefore one of the signatures must come from an honest node. But then that node $j$ convinced other honest nodes of $v$ at time t=k+1.

qed.

Summary

Assumptions: (1) Permissioned setting (2) PKI (3) Synchronous setting (4) ≤f Byzantine nodes

Goals for the SMR protocol: - consistency (honest nodes agree) - liveness (valid txs are added to honest nodes’ histories)

Goals for the BB subroutine protocol: - termination (honest nodes halt) - agreement (honest nodes output the same message) - validity (if the sender is honest, honest nodes output the correct value v*)

Dolev-Strong protocol, solving BB ⇒ SMR