‣

Resources:

time spent in class: 2.5 hours

Princeton book: Chapters 1.1-1.4

Video on how secure is SHA-256

Hashing functions in python and bash

In-depth description of SHA-2

Dan Boneh’s intro to cryptography course

Merkling in Ethereum

UBasel course: 1

‣

Diagram codes

flowchart RL
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

subgraph b1 ["Block N"]
direction TB
ph("hash(-)")
sr("nonce")
tr("txs  root = hash(- -) ")
rr("timestampt")
end
subgraph b2 ["Block N+1"]
direction TB
ph2("hash(-)")
tr2("nonce")
rr2("txs_receipts root")
sr2("timestampt")
end
subgraph b3 ["Block N+2"]
direction TB
ph3("hash(-)")
tr3("nonce")
rr3("txs_receipts root")
sr3("timestampt")
end
b1:::blue
b2:::blue
b3:::blue
ph2-->b1
ph3-->b2

h2a("hash(- -)"):::orange
h2b("hash(- -)"):::orange
h3a("hash"):::orange
h3b("hash"):::orange
h3c("hash"):::orange
h3d("hash"):::orange
tr-->h2a
tr-->h2b
h2a-->h3a-->tx1:::white
h2a-->h3b-->tx2:::white
h2b-->h3c-->tx3:::white
h2b-->h3d-->tx4:::white

flowchart RL
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

subgraph b1 ["Block N"]
direction TB
ph("hash(-)")
sr("nonce")
tr("txs  root = hash(- -) ")
rr("timestampt")
end
subgraph b2 ["Block N+1"]
direction TB
ph2("hash(-)")
tr2("nonce")
rr2("txs_receipts root")
sr2("timestampt")
end
subgraph b3 ["Block N+2"]
direction TB
ph3("hash(-)")
tr3("nonce")
rr3("txs_receipts root")
sr3("timestampt")
end
b1:::blue
b2:::blue
b3:::blue
ph2-->b1
ph3-->b2

h2a("hash(- -)"):::orange
h2b("hash(- -)"):::orange
h3a("hash"):::orange
h3b("hash"):::orange
h3c("hash"):::orange
h3d("hash"):::orange
tr-->h2a
tr-->h2b
h2a-->h3a-->tx1:::white
h2a-->h3b-->tx2:::white
h2b-->h3c-->tx3:::white
h2b-->h3d-->tx4:::white

flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

subgraph b1 ["Block N"]
direction TB
ph("hash(-)")
sr("state root = hash(- -) ")
tr("txs  root = hash(- -) ")
rr("txs_receipts root = hash(- -) ")
end
subgraph b2 ["Block N+1"]
direction TB
ph2("hash(-)")
tr2("txsroot")
rr2("txs_receipts root")
sr2("state  root")
end
subgraph b3 ["Block N+2"]
direction TB
ph3("hash(-)")
tr3("txs root")
rr3("txs_receipts root")
sr3("state root ")
end
b1:::blue
b2:::blue
b3:::blue
ph2-->b1
ph3-->b2

h4("..."):::orange
h5("..."):::orange
tr-->h4
tr-->h5
h6("..."):::orange
h7("..."):::orange
rr-->h6
rr-->h7
h2a("hash(- -)"):::orange
h2b("hash(- -)"):::orange
h3a("hash"):::orange
h3b("hash"):::orange
h3c("hash"):::orange
h3d("hash"):::orange
sr-->h2a
sr-->h2b
subgraph acc1 ["acc1"]
direction TB
nonce
balance
ch("code hash(-)")
shr("storage root = hash(- -) ")
end
ch-->code
acc1:::white
shr-->h8("..."):::orange
shr-->h9("..."):::orange
h2a-->h3a-->acc1
h2a-->h3b-->acc2:::white
h2b-->h3c-->acc3:::white
h2b-->h3d-->acc4:::white

flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

subgraph b1 ["Block N"]
direction TB
ph("hash(-)")
sr("state root = hash(- -) ")
tr("txs  root = hash(- -) ")
rr("txs_receipts root = hash(- -) ")
end
subgraph b2 ["Block N+1"]
direction TB
ph2("hash(-)")
tr2("txsroot")
rr2("txs_receipts root")
sr2("state  root")
end
subgraph b3 ["Block N+2"]
direction TB
ph3("hash(-)")
tr3("txs root")
rr3("txs_receipts root")
sr3("state root ")
end
b1:::blue
b2:::blue
b3:::blue
ph2-->b1
ph3-->b2

h4("..."):::orange
h5("..."):::orange
tr-->h4
tr-->h5
h6("..."):::orange
h7("..."):::orange
rr-->h6
rr-->h7
h2a("hash(- -)"):::orange
h2b("hash(- -)"):::orange
h3a("hash"):::orange
h3b("hash"):::orange
h3c("hash"):::orange
h3d("hash"):::orange
sr-->h2a
sr-->h2b
subgraph acc1 ["acc1"]
direction TB
nonce
balance
ch("code hash(-)")
shr("storage root = hash(- -) ")
end
ch-->code
acc1:::white
shr-->h8("..."):::orange
shr-->h9("..."):::orange
h2a-->h3a-->acc1
h2a-->h3b-->acc2:::white
h2b-->h3c-->acc3:::white
h2b-->h3d-->acc4:::white


flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

h1("root hash(-||-)")
h2a("hash(-||-)")
h2b("hash(-||-)")
h3a("hash(-)")
h3b("hash(-)")
h3c("hash(-)")
h3d("hash(-)")
h1-->h2a
h1-->h2b
h2a-->h3a-->data1:::white
h2a-->h3b-->data2:::white
h2b-->h3c-->data3:::white
h2b-->h3d-->data4:::white

flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

h1("root hash(-||-)")
h2a("hash(-||-)")
h2b("hash(-||-)")
h3a("hash(-)")
h3b("hash(-)")
h3c("hash(-)")
h3d("hash(-)")
h1-->h2a
h1-->h2b
h2a-->h3a-->data1:::white
h2a-->h3b-->data2:::white
h2b-->h3c-->data3:::white
h2b-->h3d-->data4:::white


flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

h1("root hash(-||-)")
h2a("hash(-||-)"):::green
h2b("hash(-||-)")
h3a("hash(-)")
h3b("hash(-)")
h3c("hash(-)")
h3d("hash(-)")
h1-->h2a
h1-->h2b
h2a-->h3a-->data1:::white
h2a-->h3b-->data2:::white
h2b-->h3c-->data3:::orange
h2b-->h3d:::green-->data4:::white


flowchart LR
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

h1("hash(-)")--> Data:::white


flowchart LR
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

IV("IV\n(256 bit)\n\n"):::green --> h1 --> h2 ---> h3 --> Hash("Hash = output\n(256 bit)\n\n"):::blue
s1("substring_1\n(512 bit)\n\n") -->h1(h):::orange
s2("substring_2\n(512 bit)\n\n") -->h2(h):::orange
s3("substring_N\n(512 bit)\n\n") -->h3(h):::orange
Input -->s1
Input --->s2
Input ----->s3


flowchart LR
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

X -->ED("E,D"):::green-->X2("X")
K-->ED


flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

BC("(1) block cipher") -->|"(3) Devies-Meyer construction"| CF("(2) compression function")-->|"(4) Merkle-Damgrad paradigm"| H("cryptographic hash function")

flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

BC("(1) block cipher") -->|"(3) Devies-Meyer construction"| CF("(2) compression function")-->|"(4) Merkle-Damgrad paradigm"| H("cryptographic hash function")

flowchart TD
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

A("<b>Alice</b>")
At("<b>Attacker</b> \n(7) doesn't obtain passwords \ndue to pre-image resistance\n\n"):::red
W("<b>Website</b> \n (2) store H(r||x); r is the 'salt' known only to Website \n (4) compare H(r||x)=H(r||x') => x=x' => Alice is logged in  \n\n"):::blue
A-->|"(1) sets up password x"|W
A-->|"(3) logs in by typing x' into pswd field"|W
At-->|"(5) hacks the website"|W
W-->|"(6) leaks database of hashes of type H(r||x)"|At

flowchart LR
classDef default fill:#F8ECFB, stroke:#000, stroke-width:2px, text-align:left
classDef green fill:#C5F1AB,stroke:#000,stroke-width:2px
classDef red fill:#FFA69D,stroke:#000,stroke-width:2px
classDef blue fill:#DAF4FF,stroke:#000,stroke-width:2px
classDef white fill:#ffffff,stroke:#000,stroke-width:2px
classDef orange fill:#FDDCBE,stroke:#000,stroke-width:2px

A("Alice\n(1) compute H(file_x1)=y\n (4) H(file_x2)=y => file_x1=file_x2 \n\n")
D("Dropbox"):::blue
A-->|"(2)store  file_x1"|D
D-->|"(3)download  file_x2"|A

‣

HW:

Why storing plain hashes(passwords) (without salt) by a website is not acceptable?
Find the largest number N, such that your laptop can find in under 10 minutes a nonce r such that sha256(your_first_name || r) starts with N zeroes.
Read an in-depth article about sha256 hash function.

Overview of the remaining course topics: Cryptography, different consensus protocols, details on Bitcoin/Ethereum, presentations on DeFi primitives, DAOs, NFTs, Scaling, MEV, zk-tech.

Hash functions

Hash function is a function that

has input = any string (in reality there is a large upper bound, $2^{64}$ bits is common);
has ouput = some fixed size number (often 256-bit);
is efficiently computable, which means computation for n-bit string takes O(n) time.

You should think of a hash function as a map $H:\{\text{strings with less than }2^{64}\text{ bits}\}\to \{0,1\}^{256}$

Example: computing the remainder: H(x)= x (mod 97).

Applications: checksums to detect typos.

‣

1. International Bank Account Number example:

Screenshot taken from here

‣

2. Ethereum public address example: (capitalization doesn’t matter in the address, but helps to have a checksum!)

Screenshot taken from here

3. Checksums are also used in credit cards, vin numbers (US and Canada)

Such simple hash functions are helpful to detect accidents, but not to prevent attacks. For example, these two IBANs are very similar but have the same checksum, and so an attacker may try to sub one with the other in order to steal money:

CH5604835012345678009 vs CH5604835012345687709

The above is an example of a collision. To prevent attacks we need:

Cryptographic(ally secure) hash function is a hash function satisfying:

Collision resistance
Pre-image resistance
Puzzle-friendliness (intuitively ”uniform randomness”)

Let’s take a close look at these properties. (Later on we will also cover the construction of hash functions.)

Cryptographic hash functions: properties and applications.

‣

1. Collision resistance

Definition: a function H:X→Y is called collision-resistant if it is infeasible to find x and x’ such that H(x)=H(x’).

Collisions exist. Hash functions do have collisions, since the domain $X=\{\text{strings with less than }2^{64}\text{ bits}\}$ is larger than the target $Y=\{0,1\}^{256}$ . In fact, trying $|Y|+1=2^{256}+1$ values as an input will for sure produce a collision. Moreover, if we pick random inputs and compute the hash values, we’ll find a collision with high probability long before examining $2^{256}+1$ inputs. In fact, if we randomly choose just $2^{130}+1$ inputs, it turns out there is a 99.8% chance that at least two of them are going to collide. This phenomenon in probability is known as as the birthday paradox [need only 23 people for >50% of the same birthday].

Collisions are hard to find via enumeration. The catch, is of course, this way of finding a collision is extremely resource-intensive. If a computer computes 10000 hashes per second, we would need $10^{27}$ years to compute $2^{128}$ hashes, an average number of hashes needed to find a collision. Thus we can conclude that finding a collision in a life-span of a human (something someone might try to do) is much less probable than an asteroid hitting the earth, and therefore we do not need to worry about it — for practical applications, we can assume hash functions to be collision resistant.

Collisions may be easy to find by some other, clever method. Here is an example where a different, cleverer method to find a collision works. Hash function $h(x)=x ~(mod ~ 2^{256})$ is not collision-resistant, since $3$ and $3+2^{256}$ give an obvious collision. [Remember that a string = sequence of bits = binary number] This example showcases that some arbitrary function can very well be not collision-resistant. Turns out, there are no functions that are proven to be collision-resistant.

Cryptographic hash functions are empirically collision resistant. The cryptographic hash functions that we rely on in practice are just functions for which people have tried really, really hard to find collisions and haven’t yet succeeded. In some cases, such as the old MD5 hash function, collisions were eventually found after years of work, leading to the function being deprecated and phased out of practical use.

Application: recognition of a file that you have seen before.

‣

2. Pre-image resistance

A function H is pre-image resistant if given y=H(x) it is infeasible to guess what x is.

Problem: what if we have y=H(”heads”) and also know that x can attain only two values, ”heads” or “tails”. Then a brute-force search algorithm will quickly find the pre-image: try H(”tails)” and try H(”heads”), and pick the one which gives y.

Solution: we can add to the hash something very random, a secret string r that can have many many values. Then, knowing y=h(r || ”heads”), one will not be able to guess what x was.

Definition: a hash-function h is pre-image resistant (or hiding) if: when a secret value r is chosen from a close to uniform distribution on a large set of values, then given y = H(r ‖ x) it is infeasible to guess x, even if x is chosen from a small set. [Note 1: r is not given to the finder] [Note 2: “close to uniform distribution on a large set of values” is used here as a good substitution for the more technical “distribution with high min-entropy”]

Application 1: secure password storage on a website server.

Question: why storing plain hashes(passwords) (without salt) by a website is not acceptable?

‣

Application 2: commitment to a secret message now, with a possibility of revealing the message later. A commitment scheme consists of two algorithms:

com := commit(msg, nonce) = H(nonce || msg) The commit function takes a message and secret random value, called a nonce, as input and returns a commitment.
verify(com, msg, nonce) The verify function takes a commitment, nonce, and message as input. It returns true if com = commit(msg, nonce) and false otherwise.

We require that the following two security properties hold:

Hiding: Given com, it is infeasible to find msg [This follows from the pre-image resistance of H, and ensures that the message is hidden]
Binding: It is infeasible to find two pairs (msg, nonce) and (msg’, nonce’) such that msg ≠ msg’ and commit(msg, nonce) = commit(msg’, nonce’). [This follows from the collision resistance of H, and ensures that you cannot change your mind later.]

Rmk. Every time you commit to a value, it is important that you choose a new random value nonce. In cryptography, the term nonce is used to refer to a value that can only be used once.

‣

3: Puzzle-friendliness (intuitively ”uniform randomness”)

Definition: A hash function h is said to be puzzle-friendly if for every possible n‐bit output value y, if k is chosen from a close to uniform distribution on a large set of values, then it is infeasible to find x such that h(k ‖ x) = y in time significantly less than $2^n$ . [Note: k is given to the finder; this is the difference compared to pre-image resistance]

Intuitively, what this means is that if someone wants to target the hash function to come out to some particular output value y, that if there’s part of the input that is chosen in a suitably randomized way, it’s very difficult to find another value that hits exactly that target.

Application: search puzzle. It consists of:

a hash function, H,
a value, id (which we call the “puzzle‐ID”), chosen from a close to uniform distribution on a large set of values
and a target set Y

A solution to this puzzle is a value, x, such that H(id ‖ x) ∈ Y.

How this fits into proof-of-work in Bitcoin. There H = sha256, ID = new block, x = nonce that miners try to find, $Y=\{y\in \{0,1\}^{256}~|~y<N_d\}$ , where the number $N_d$ is the difficulty number automatically adjusted by the Bitcoin nodes’ software once in two weeks so that the average time between blocks is ten minutes. The fact that ID comes from a close to uniform distribution is insured by the fact that it cointains the hash of the previous block.

The size of Y determines how hard the puzzle is. If Y is the set of all n‐bit strings the puzzle is trivial, whereas if Y has only 1 element the puzzle is maximally hard. The fact that the puzzle-ID is randomly chosen from a large value-set ensures that there are no shortcuts. On the contrary, if a particular value of the ID were likely, then someone could cheat, say by pre‐computing a solution to the puzzle with that ID.

If H is puzzle‐friendly, this implies that there’s no solving strategy for the search-puzzle which is much better than just trying random values of x. And therefore, if we want to pose a puzzle that’s difficult to solve, we can do it this way as long as we can generate puzzle‐IDs in a suitably random way. This is crucial for the proof-of-work idea in mining.

Cryptographic hash functions: construction

There are several components that we need to introduce:

‣

(1) Block ciphers (cryptography’s workhorse construction)

Block cipher consists of two maps $E,D: K\times X \to X$ satisfying $E(k,-)^{-1}=D(k,-)$ . Each E(k,-) is essentially a permutation on X. E stands for enccryption, D stands for decryption.

Secure block cipher is a block cipher where a random permutation from $Perm(X)$ is probabilistically indistinguishable from a random permutation from $S_K=\{E(k,-)~|~k\in K\} \subset Perm(X)$ . In other words, the subset $S_K$ is scattered uniformly inside $Perm(X)$ , as illustrated below.

White: $S_K=\{E(k,-)~|~k\in K\} \subset Perm(X)$

Black: $Perm(X)$

How are secure block ciphers constructed? Secure block ciphers are built using iterations of a round function R:

Screenshot taken from Dan Boneh’s Cryptography I course in coursera.

Key point: if the round function R is chosen appropriately (obv needs to be non-linear) and if n is large enough, this proccess arrives at a secure block cipher. In real life this is done more or less experimentally, two standard algorithms:

3DES: X={0,1}^64, K={0,1}^168, 48 iterations
AES: X={0,1}^128, K={0,1}^128 (or 192 or 256), 10 iterations

‣

(2) Compression functions

Compression function is the name for a fixed-length-input hash function, eg $h:\{0,1\}^{768}\to\{0,1\}^{256}$ . They should satsify the same properties: collision and pre-image resistance, and uniform randomness.

‣

(3) Davies-Meyer construction

Given a block cipher ( $E,D$ ), one can construct a compression function as follows:

h(x || k)= E(x,k) \oplus x

where $\oplus$ is bit-wise summation (mod 1) [0 $\oplus$ 1=1, 1 $\oplus$ 0=1, 0 $\oplus$ 0=0, 1 $\oplus$ 1=0] (aka “xor” functions, aka exclusive “or”)

Thm: if a block cipher is secure, then the Davies-Meyer compression function is as collision resistant as possible.

‣

(4) Merkle-Damgrad paradigm

Enables transition [compression function] → [arbitrary-length-input hash function], via the scheme illustrated below (the specific numbers of bits are from SHA-256).

The input is subdivided into substrings, 512 bit each; the last substring is padded at the end with the total length of the initial input, so that the total length of the resulting input is a multiple of 512 bits. IV = Initialization Vector = some number that is chosen a priori.

Thm: if compression function h is collision resistant, then so is the resulting hash function.

SHA-256 example (hash function used in Bitcoin)

Block cipher is chosen: SHACAL-2 (x (256 bit), k (512 bit) ) → x’ (256 bit)
Compression function is constructed using the Davies-Meyer method: h( H (256 bit) , m (512 bit) ) = SHACAL-2 ( H, m ) $\oplus$ H
SHA-256 hash function is constructed using the Merkle-Damgrad paradigm.

Hash pointers and Merkle trees

Hash pointer is a data structure that consists of: (1) a pointer to where data is stored; (2) hash of that data.

Whereas a regular pointer gives you a way to retrieve the information, a hash pointer also gives you a way to verify that the information hasn’t changed.

Block chain data structure is a sequence of blocks with hash pointers to previous blocks.

The key property of a block chain data structure is that it is temper evident, which means that by storing the head of a blockchain, any alteration of previous data will result in a change of our hash.

Merkle tree is another data structure that uses hash pointers. Each hash is obtained by concatenating the two lower hashes and taking their hash.

It is also temper evident, but has the following advantages over the block chain:

Proof of membership. Given n pieces of data (above n=4), only about log(n) hashes are needed in order to prove that some particular piece of data is stored in a Merkle tree.

If a “light client” knows the root hash and wants to be convinced that data3 is stored in the merkle tree, a “full-node” (the one that stores the merkle tree) only needs to send the 2 green hashes to “light-client”. Note that $2=\log_2 4$ .

Proof of non-membership. If pieces of data are sorted (lexicographically, say), then proof of non-membership requires O(log(n)) items to be shown. (just show that two consecutive items are stored, and the item in question would need to be between them)

Super-important blockchain applications of Merkle trees. (explained here)

In Bitcoin, full nodes store all txs inside Merkle trees (takes hundreds of GBs), while light clients (= wallets) store only the hash roots, and can prove membership of transactions in a fast manner by asking full nodes for merkle proofs.

In Ethereum, not only txs are stored in a Merkle tree, but also the Ehereum state is stored in a Merkle tree. There are also Merkle trees for txs receipts and for storages of smart contracts (see the diagraь below).

Various Ethereum nodes:

Archival nodes: in each block they store all txs + all Ethereum state in Merkle trees.
Full nodes (typical node): in each block they store all txs in Merkle trees and store only the Merkle root of Ethereum state; can query archival nodes for the state.
Light clients (wallets): in each block they store Merkle root of txs and Merkle root of Ethereum state; can query archival nodes for states (usually this is done by a central provider), and full nodes for txs.

Lecture 3. Cryptographic hash functions.