Blockchain and The Third Web

“Tell us where you hid the USB drive
with the bitcoin on”

Last time I talked about the origins of the architecture of the modern web and how it is what it is because people like Tim Berners-Lee and Roy Fielding radically decomposed the problem space into its stable, long-living, constituent parts. These parts are the constraining building blocks of REST and the web.

The design for the web came off the back of some visionary work by Vannevar Bush in the 1940s and Ted Nelson in the 1960s, 1970s and 1980s, which illuminated a path to follow but proved to be less practically useful on how to achieve it.

The takeaway was this: the architecture of the modern web has been successful because it has moved forward in very small increments, each building on the last, edging towards a grand vision, but where each step has used what Vint Cerf called “micro-standards” to constrain ambition.

Fast forward to now and, in the last few years, two schools of thought have emerged suggesting different technical ways forward for the web.

The Webs

Web3

In the red corner, we have Web3. Web3 is a new, and quite different, decentralised web, based on blockchain-like technologies. We don’t have Web3 yet but, according to Crunchbase, there are tens of thousands of start-ups trying to build it. Web3 is most definitely popular with a new wave of technologists.

Web 3.0

Then, confusingly, in the blue corner we have Web 3.0. Web 3.0 is an extension to the web we have now with richer semantic meaning to the data it holds, which itself may also be decentralised. Web 3.0 is being led by web traditionalists like Tim Berners-Lee. It feels strange referring to web pioneers as traditionalists, but here we are.

Web3 vs Web 3.0

Often when there’s an apparent tech showdown we talk in terms of winners and losers when in reality it’s actually a far more boring competition for market share. But Web3 and Web 3.0 propose fundamentally different directions.

Tim Berners-Lee is not a fan of web3 and moreover would prefer we do not confuse the two.

Futuregazing

I predict that Web3 will die. I predict that in perhaps 5-10 years from now we will look back on Web3 as the Enterprise Service Bus of its day: an interesting idea that lost its way and exploded into layers of arbitrary complexity that had nothing to do with the problem it was trying to solve.

To back up my prediction I’m going to show (interactively) how blockchains work from first principles. Not to knock blockchain technology but to show how cool it is. Because here’s the thing - Web3, cryptocurrencies, DAOs and Smart Contracts are all super-interesting and cool ideas.

All these ideas hang on the premise of blockchain technology and I love blockchain technology.

It feels to me inevitable that one day there will be global, digitally-based, currencies that are not controlled by governments. It is the way of things and it can not be stopped. It also feels inevitable that the future will be decentralised. However inevitable does not mean fast. Doc Searls wrote The Intention Economy in 2012 predicting that ‘centralised’ applications, like CRM systems maintained by large businesses, would be replaced with a decentralised model where our personal data (probably stored on our phones) would be shared with companies rather than mastered by them.

It is the way of things, but the way of things can be slow

Just as we saw last time, the question here is not what the future is, but what technology underpins it. What works? What is the simplest thing that works? What we can adjust to? What we can augment the current experience with?

Much as I admire Ted Nelson’s work, it has to be said that he charted an unconstrained path that required impossible technology to make it work. He wanted all the benefits of a centralised system in a distributed network. Distributed systems lead to asynchronous communication which leads to issues of consistency and concurrency. A problem purportedly solved by blockchain.

Blockchain

So let’s build a blockchain together and see what we discover along the way. You won’t need anything more than a basic technical understanding to follow along.

Hashes

In fact, the only concept to wrap your head around is the hash. A hash, from the French word hacher which means to chop (which is itself derived from the old French word hache meaning axe), is just a function that takes some data and chops it up in very special ways.

The special ways are:

1. The same data input should always give the same output, i.e. they are deterministic
2. The chopping-up process is fast enough to be used in real-time
3. The chopped-up data output can not be correlated with the input - that is to say, if you chopped up the string "aaaa" and then the string "bbbb" you couldn’t work out from the output what the chopping-up process is doing. There can be no way to predict what the output will be for any input. i.e. even though it’s deterministic, it feels random
4. No two inputs can have the same output
5. Regardless of the input length, the chopped-up output is always the same length. So the length of the output for "aaaa" would be the same as the output for "aaaaaaaaaaa"

The output is often referred to as the ‘digest’ because it’s not an encoded or encrypted version of the input, but rather, as per the dictionary definition of digest, a compilation or summary of material or information and by convention the input text is often called the ‘message.’ To keep it simple here, I’ll call the output the hash and the input message the input.

Here’s a hash function in action.

Note that this is all interactive (oh yes) so you can edit the plain text input (message) and see the hash (digest) changing with each keystroke. The hash text will turn red if it is different from its original value, highlighting that the input has been altered.

As you change the text you can validate that this hashing activity complies with all the rules above. I know it does because it’s using SHA256 which is one of many hash functions.

Hashes are cool and they have many uses. If I create some text (like a will, or a contract) and also generate a hash from it, that hash acts as a succinct shorthand for that text, i.e. I can send my text to anyone in the world and ask them to generate a hash of it using the same function and if my hash and their hash match, then we both know the message hasn’t been changed (because of rule 1 and rule 4 above).

If the text were some personal information unique to me, like my password, then that hash acts as a digital signature that can represent me.

For example, if I run the password "il0v3blockch4in" through the SHA256 hash function (and you can try this above if you don’t believe me) I get:

25fb88108a848cb3c02c28a3b96cf915291360c502ece64c10d1bc8173a40ef0

and rule 3 says I can share that hash anywhere and no one can reverse engineer my password. And when I need to verify myself later, I can prove I know the password by entering it into the hash function again, which will generate the same hash.

(I should add here that this is not quite how actual password management systems work because it’s trivially easy to pre-generate huge lists of hashes for strings, as you can see for yourself by googling 5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8 which is the hash for the string "password").

And one last interesting feature is if you delete all text in the input box, you’ll see it gives a hash of:

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

which is the hash of the empty string because of rule 5.

So that’s the hash. Simple, powerful, often misunderstood and has uses beyond generating digests for text.

And all blockchain is, really, is clever things with hashes, which we will come onto in a moment. First though, a little history lesson.

The History Mystery

Blockchain first appeared in 2008. On 18 August 2008, the domain bitcoin.org was registered and shortly after, on 31 October, the famous paper “Bitcoin: A Peer-to-Peer Electronic Cash System” (link in the notes) was released. Some software, based on the paper, was released on 9 January 2009 and the first block in the chain, the Genesis Block, had been ‘mined’ (we’ll cover all this in a bit) on 3 January 2009.

These things we know for sure because that first block had embedded within it some text from The Times newspaper: The Times 03/Jan/2009 Chancellor on brink of second bailout for banks

The Times, 3rd January 2009

And that’s about all we do know. The person behind all this was “Satoshi Nakamoto.” No one knows who he/she/they were because Satoshi Nakamoto doesn’t exist. There are lots of theories, including one from Ted Nelson, but I think an important fact here is Bitcoin and Blockchain did not iterate out of a standards body or open-source project with lots of committers.

Blockchain is more like Xanadu than it is like the web. It’s a vision and a solution. The web was a 50-year shared (and debated) vision that eventually took existing technologies and built on them in increments.

That doesn’t by default make it wrong or bad. It just means it arrived fairly well-formed. Satoshi Nakamoto may have been one person that died or a group of people who, for whatever reason, would rather not declare themselves. We may never know. We know that there were some emails and discussions from 2008-2010, a domain registration and a paper, some code and the first chain in the block.

Satoshi Nakamoto, Cryptography Mailing List, November 2008

Back to those hashes…

Chains of Hashes

You can run any text through a hash function, including hashes created from previous use of a hash function, a hashed-hash if you will.

Let’s say we have some text like hello world which we call text-1. If we run it through the hash function we get hash-1 which happens to be

b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9

Now, we take another piece of text lorem ipsum called text-2 and we run that AND hash-1

b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9 lorem ipsum

through the hash function to get hash-2

87445811e30e7d4233b2ab73d8b3b61ffe35678be5db4cadc76b1746b04db63d

hash-2 is the hash of the second plain text (text-2) and hash-1 from the first text (text-1).

This new “super-hash” would be different if EITHER text-1 OR text-2 OR both changed.

If text-1 changes then hash-1 and hash-2 change. If text-2 changes then hash-2 changes, but not hash-1 (because it’s the hash of only text-1). So we can tell both when some text has been changed AND be directed to which text input changed.

Imagine if we did this for text-1, text-2, … text-n - we’d have a chain of hashes where the last hash in the chain would be different if ANY message were tampered with and a pointer to the change location.

Something like this:

You can see here that changing any text input changes all hashes that follow. But not any before. If we could save those hashes somewhere private, then later a quick re-hashing exercise would tell us a) if any data has been tampered with and b) where it took place (the first hash in the chain that’s different).

This is cool, but it has the obvious flaw that you can only add new data to the end of the chain. There’s no scope for parallelism.

Trees of Chains of Hashes

Because you can hash a piece of text or a hash (which is just text), you can hash the hash of one chain of text messages with the hash of another chain of messages thereby creating a tree of messages and hashes. This is called a Merkle Tree, after Ralph Merkle who invented it in 1979. Merkle Trees allow us to create a chain of input data messages and hashes and link them up with other chains, eventually ending up with a “top hash” like this:

Merkle Tree

Let’s see that in action:

See that if you edit any of the bottom input fields, you can see a cascade of red hash updates all the way to the top hash.

This is our first look at one of the essential building blocks of a blockchain. Give or take, this is what a block looks like. To link blocks together, the top hash in one block becomes an input to the top hash in the next block. This means a change to one field in one block would be detectable within the block itself (as above) AND by every block thereafter.

A chain of blocks from Nakamoto’s paper

However, as lovely as all this is, we still have a problem caused by rule 1 above (hash functions can be used in real-time) because hash functions are fast and that means, in theory, if no one was looking, it would be possible to edit some data and then really really quickly redo all subsequent hashes in the block and in the rest of the chain.

We’re missing one important element to secure this chain. We need some kind of mechanism to “seal up” our block so that it’s very hard to recreate that top hash. That way, when it’s added to the chain of other blocks, we can focus on new blocks and not worry that historic data can be meddled with.

Enter the miner.

Mining

Mining is a strange term, but it’s not too far off a description of what is happening.

Look at that top hash again. It represents a signature for all data in the block. It feels almost random. But it’s not random. It’s a by-product of the data at the bottom. Change the data, change the hash.

What if we inserted a special system text message that was full of meaningless data, used only for that block, to manipulate the hash? Why would we do that? Well, here’s an actual block hash from block 103882 of the bitcoin blockchain:

0000000000000000c3d2d2b5364ffbaf67dc7e627757384bc2c5a098a2130c9b

And here’s the hash from block 103881 just before it:

0000000000000001b6fa7566996f97faed4adc88b9c3b0d5eb74356bb183ec84

Notice anything? They begin with a series of zeroes.

Block 103881 has 15 leading zeroes in the block hash and block 103882 has 16. And yet, remember rule 3 - that you cannot predict the output hash from the input.

So how can we get a hash that begins with several zeroes?

By trying every combination of random text in our system message until we find (mine) one. And that can take a long time. The more zeroes we are aiming for, the longer it will take. On the flip side, it can also be proven in an instant that you did all that work by simply re-hashing the block.

From Satoshi’s paper:

the hash begins with a number of zero bits. The average work required is exponential in the number of zero bits required and can be verified by executing a single hash.

OK, so we take a series of input messages (transactions) and hash them all, then throw in a system message that we can change at will, and then generate a top hash. That top hash is extremely unlikely to begin with the correct series of zeroes initially but we can fiddle with the system message until it does.

The one-time-use made-to-measure system message is called the “nonce,” from the expression for the nonce which means “for the once.”

And this is all worth it because once that block is finished if anyone did try to mess with those transactions, they’d have to do all that re-hashing and create a new hard-to-find one-time nonce that fits the pattern. Once you have a few blocks joined in a chain that effort would take years. And that’s without recognising that the actual blockchain is distributed among lots of servers, so an attacker would have to re-mine the whole chain and somehow get all those servers to trust the new data.

What’s being described here is a Proof of Work algorithm.

Proof of work is any system where one party (the miner) has to expend a lot of effort to achieve something (finish off a block in the chain) and where other parties (in the case of blockchain, the whole world) can easily verify that the effort has been expended. The concept wasn’t new in blockchain, it goes back to a paper from 1992 called Pricing via Processing by Cynthia Dwork and Moni Naor. That paper largely focuses on combating junk email, but the problem is the same. If you only accept emails that contain some token of the sender having completed a computationally intensive task (and you can verify this automatically and easily) then it hugely disincentivises spammers from sending bulk emails.

Let’s do some mining.

Below we have:

• a series of input data items
• a nonce value (which is initially an empty string but remember that even empty strings have a hash value)
• a top hash that’s derived from a hash of the hashes of the data items and a hash of the nonce value
• a target hash pattern that the leading characters in the top hash must match

It’s not quite a tree, but it’ll do for mining. If we change the nonce value, we change the top hash (you can try that by entering text into the nonce box). The Hash Target box gives us the pattern we are looking for in the leading characters of the Top Hash. I’ve limited the length of the target for the top hash to three hex characters because much longer than that and you’ll be there all day listening to your CPU go bananas.

You could try entering random nonce values until the Top Hash matches the Hash Target, or you can click ‘mine’ and brute-force a match.

In this example, you can set Hash Target to whatever you want from 000 to FFF. The blockchain spec uses leading zeroes for the simple reason that makes it trivial to say the top hash value must be less than x and test it (more zeroes, lower value). Because of this, it’s possible to hit on a hash that has more leading zeroes than the target and still be valid. The lowest value on record had 23 leading zeroes.

The target adjusts every two weeks or so to cater for the fact that the speed at which it’s possible to generate hashes keeps increasing. The present rate is just under 300m terahashes a second.

Hash Rate Over Time

And, of course, all chains must start somewhere.

Genesis

The first ever block is the one we know was mined in 2009 because it contains the quote from The Times. It had a target hash beginning with 10 zeros which became

000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f

The Blockchain Genesis Block

The blockchain, as of writing, is about 460Gb in size and contains slightly over three-quarters of a million blocks, which is a lot of hashing activity.

Longest chain wins

Once a miner generates a top hash that matches the specification, the block is added to the chain and all other miners can verify it and accept the new chain. This is proof of work in action and it has two benefits: the first is that anyone can verify that the work was done in an instant (e.g. I know that if you didn’t change any of the values above, the mined ‘block’ would have a top hash of 000d213d560b383bc454a0499e8216c2cae665bf90e62defc361b91773cd202b which would have taken 4307 attempts to find) and it brings consensus to participating servers in the network, something known as the Byzantine Generals’ Problem, which is something I have written about in the past.

The proof-of-work chain is a solution to the Byzantine Generals’ Problem

It’s both a way of knowing each block is sound and a way of gaining consensus across servers that the chain is the chain. Anyone wanting to trick servers into accepting a different chain would have to replicate the whole thing (computationally impossible) and get a majority of the active participants to accept it (not feasible in an open environment).

I don’t propose to go into detail on the way participants agree on everything because it’s all fairly standard distributed systems stuff and/or blockchain conventions from here on out. A good read though is Nakamoto’s Longest-Chain Wins Protocol by Kartik Nayak.

Bitcoin

I seem to have reached this point without really talking about bitcoin. To be honest, I find the bitcoin aspect of blockchain fairly arbitrary. When a hash is mined and a new blockchain block is minted, the miner is rewarded with bitcoin. That’s awesome if we agree bitcoin has intrinsic value. But you could build your own blockchain and create your own currency. Many have. Recent surveys suggest there are somewhere around 10,000 active cryptocurrencies in the world. Whether they are worth anything falls into the same psychological question as asking whether a paper dollar is worth anything. It is if enough people believe it is. And it may be today, but not tomorrow.

Is This a Pound?

That’s a ‘one pound’ note, issued by the British Army in 1962 for use in canteens. At the time it was worth one pound of canteen food, or one actual UK pound if you could exchange it. Outside army canteens it was worth nothing. Today you can buy them for a few UK pounds.

Summary

I began this by saying I’d show that blockchain will die and web3 won’t happen. Now we have a grounding in the basics of blockchain, let’s get to that.

No doubt Satoshi Nakamoto’s paper was food for thought. Blockchain takes some older ideas and presents them in a novel way. As the basis for a cryptocurrency, Proof Of Work makes a lot of sense. Money has always involved ‘work.’ That’s why cash paper notes are intricately detailed and difficult to copy. The effort required to create forged notes is discouraging to counterfeiters. A future digital currency will have to have some asymmetric property that makes it hard to fake but easy to verify.

Will one or more of the current 10,000+ cryptocurrencies eventually replace fiat currency? Who knows. As we spend less and less paper money it certainly feels like digital-only finances aren’t too far away. But right now security is a bit of a question mark for crypto. Investing in cryptocurrencies has been compared to 17th-century Tulip Mania many times. I think we need to let some bubbles burst and see exactly who’s benefiting from the hype before anyone can make pronouncements on the crypto future.

I think the more pressing issue is blockchain mania. Blockchain has been cursed with a cool name and too many investors thinking it’s magic. And that has led to it being repurposed into areas it has no business in. Either because it’s not a good problem-solution fit, or because existing solutions already do the job well enough. One caveat to this is if your start-up idea is a bit lame then you might well get more funding if it’s a lame idea plus blockchain.

Blockchain is a data store that partly solves some distributed systems consensus problems but at a cost. And that’s fine. All solutions start somewhere. What usually happens next is people iterate towards a better solution or agree there isn’t one. This is what happened with the web. People came up with great ideas (Xanadu) and systemic forces judged them. Eventually, Tim Berners-Lee iterated a solution that worked as a micro-standard on top of existing technology. Didn’t make Ted Nelson wrong, it just proved that physics always wins.

And the physics of blockchain is wrong.

It’s slow. By design, it takes about 10 minutes to mine a block. That’s not awful for minting currency, but it limits blockchain’s applicability beyond that. Take away the mining and you have an append-only log database. And there are plenty of good ones already on the market.

It’s open. The whole point of the blockchain distributed ledger is it works in the open. Most use cases require degrees of privacy. If you want a privately controlled ledger that works for known parties then you are describing a database.

Distributed systems laws still apply. People have wrestled with problems caused by distributed data for decades. There are no easy answers because time and space conspire to make concurrent common knowledge a relative concept. It’s always about trade-offs. That’s what CAP Theorem is, thinking about what to trade for what at any one moment. Blockchain hasn’t solved those problems. It’s avoided overt voting for consensus but that’s about it.

In summary, it’s an interesting solution to a different problem than the one the web is actually solving.

In 2017 Kai Stinchcombe wrote Ten years in, nobody has come up with a use for blockchain and followed it up in 2018 with Blockchain is not only crappy technology but a bad vision for the future. In the second piece, he describes blockchain as:

a very long sequence of small files — each one containing a hash of the previous file, some new data, and the answer to a difficult math problem

which is sustained by a simple proposal to participants:

divide up some money every hour among anyone willing to certify and store those files for us on their computers

But the problem we are trying to solve is this:

What if everyone keeps their records in a tamper-proof repository not owned by anyone

Which to me is a description of a vision for the web. i.e. the next set of micro-standards.

What we are witnessing with blockchain is Solution Creep: the technology version of Mission Creep.

You see this all the time in scaling tech teams. Once upon a time, someone thought it was a good idea to create a bespoke framework to run the company’s platform on. Their reasons sounded plausible at the time: we are different, we need to scale, there are no frameworks suitable for us. Yet as the months (or sometimes weeks) pass, that proves not to be true. Rather than face facts, more solutions based on the original premise are bolted on. Even as alternatives become trivially easier to implement, it becomes de facto policy to solve the problem in front of you rather than restate the original intent and sense check against that.

It’s gone so far that we now have Dfinity building an internet compute platform based on blockchain. And it has some hugely smart people working on it. But this is the thing. They always are smart. Smart people love solving hard problems. To suggest they might instead like to fight over web micro-standards would be taking all their toys away. And I don’t blame them. The most fun I have ever had at work is when solving huge technology challenges. It’s just that it’s all starting to fall apart now. Eventually, these technologies have to hit genuine mainstream adoption or they die. And they take the dreams of hopeful investors with them.

Ethereum moved away from Proof Of Work in 2022 to Proof of Stake. Proof of Work has become fairly toxic in a world waking up to climate change because all those competing miners use a lot of energy. Estimates over the years of electricity cost have varied from about 10k USD to over 20k USD to mine a single bitcoin. To be fair, that’s also partly what governs the value of a coin (you wouldn’t spend more on electricity than you’d get back). So alternative consensus mechanisms are a hot topic - which is one of many hints that perhaps we are building on sand. There are several variations of Proof of Stake but all (obviously) build on the premise of a very long sequence of small files joined by hashes.

Smart Contracts being a thing seems like lunacy to me. The very idea that a contract for anything could be reliably reduced to compute tasks in a world that is so incredibly nuanced, changing and complex is laughable. This is why one of the main issues with smart contracts is the Oracle Problem - they don’t know about the real world so we (solving the problem in front of us) connect them to centralised “oracles” to fix that.

This brings us to Web3.

I remember the early days of the web. It was lovely. Even Twitter was lovely. Early adopters of the web wrote blogs and shared memes and were generally nice to each other. There were no ads and no clickbait and if you read something online the chances are it was true. Finding things was hard until Google came along but other than that it was lovely. What went wrong is the click was monetised. Once there was value in ‘engagement’ it was a slippery slope towards Sodom and Gomorrah. People lied, trolls trolled, companies chased metrics and advertisers accepted that quantity trumped quality. Google joined in and now it’s a sea of shite. We’re not just enticed into false flag arguments about pointless bullshit, our political views are being manipulated.

In 2014 Gavin Wood, one of the founders of Ethereum, coined the term Web3 as the antidote. Web3 is hard to pin down, but the promise is it will use blockchain technology, decentralised control and tokens (micro currencies) to rip the power away from ‘big tech’ companies like Facebook and usher in a new dawn of loveliness.

But like so much of the web today, it is itself a false argument. As Stephen Diehl wrote:

Web3 is a deeply polarizing topic for technologists because it’s designed to be that way. It’s a rhetorical trick to set up a false dichotomy between the legacy internet world of popup ads and Zuckerbergs — which legitimately does suck — and a fantasy world built on technologically incoherent pipe dreams and phoney crypto-populism

It is true that large parts of the web suck. Facebook is full of time-wasting shit and lies. Twitter regularly plumbs new depths of immorality. But web technology hasn’t mysteriously become centralised. HTTP does what it’s always done. The problem is us. We go to Google and follow the links that are closest to our mouse. We don’t think for ourselves. We get enraged and engaged because the content we read is designed that way. Adding some fucking blockchain isn’t going to save us from being lazy morons. Facebook is just a business that makes money from ads. And they are really good at it. They’ll keep doing that until we stop clicking on them as part of our Dumb Contract.

It’s a classic manipulation technique to tell people that some group of individuals is “the elite” and that they are (via often unclear means) controlling us in evil ways. It could be Trump’s media elite or Boris Johnson’s European elite. The pattern is to point at a problem (usually one caused by the narrator) and ascribe it to the elite and then offer a better future if only you click, vote, or buy cryptocurrency like a good soldier.

Web3 is blockchain where it serves no purpose and adds no value. Web 3.0 isn’t pretending it has the answers to social media covering us all in goo, but it will make information on the web incrementally more usable. The rest is up to us.

Notes

• The painting is Prometheus Being Chained by Vulcan (1623) by Dirck van Baburen. It depicts Prometheus being chained by the god Vulcan for stealing fire from Zeus and giving it to humans. Prometheus was in for a shock too because once chained, an eagle was going to appear every day and peck at his immortal liver.

• You can find Satoshi Nakamoto’s original bitcoin paper here

• The Economist ran a nice summary of Blockchain called “The great chain of being sure about things” in October 2015

• I’ve done a couple of talks based on this, both of which were hugely inspired by Anders Brownworth’s awesome Blockchain Demo. It’s totally worth watching his video of the demo.

• The code used in this article is on GitHub.

• If you remain convinced blockchain is superior to at least one pre-existing technology solution then there’s a good point by point analysis on Reddit worth reading