It first came to our attention as the plausible-sounding but obscure data structure that made Bitcoin unhackable. Then some people lost a lot of money in attacks on Bitcoin exchanges. But before the cyberspace version of yellow crime-scene tape could come down, blockchain was back. Exonerated for the bitcoin fiascos, blockchains were now being researched as the next big thing in banking—the potential foundation for a new, revolutionary generation of secure asset-trading systems, handling everything from crude oil to credit derivatives. Maybe.
And before we could enquire about that little caveat, blockchains morphed again. Now they were the best friend of industry—managing supply chains, regulating software updates, perhaps even offering a secure programming environment for the Internet of Things (IoT).
But just a moment. What are blockchains? How could they be so widely useful? And do you need to care? Therein lies a tale.
Let’s start our story with money. Setting aside the small fraction of transactions that use physical cash, money is pretty much what the banks say it is. You open an account, and the bank records all the transactions on that account and maintains a reasonably current value of the balance. Every transaction involves the bank, and you have to trust them, although you can occasionally check their work.
But let’s not think like bankers—let’s think like libertarian citizens of cyberspace: egalitarian, suspicious of authority, firm believers in the wisdom of crowds. We don’t really need the bank. If we all just keep one big, public list of all transactions, anyone can look through the list at any time and see if a proposed transaction is valid—so nobody can cheat.
Of course there are some issues. There is no privacy in this public ledger. And if even one clever person finds a way to falsify a record, they could cause havoc.
In the Fall of 2008 a person or group using the pseudonym Satoshi Nakamoto published a paper describing how a pair of mathematical processes doable on personal computers—secure hashes and public-key cryptography—could work together to make a public ledger both anonymous and secure. This rather confounding stack of clever ideas gave birth to Bitcoin.
The paper proposed a protocol under which a participant who wished to transfer money to someone else would broadcast a transaction to all the other participants (Figure 1). This post would identify in hashed form the previous transaction in which the poster received the funds. It would give the amounts to be paid. It would provide the poster’s public key and a hashed version of the payee’s public key. And it would include a hash of the whole transaction cryptographically signed with the poster’s private key.
This may all seem needlessly obscure, but it satisfies two vital requirements. It allows any participant to verify the transaction—by proving that the poster is in fact the person who signed the transaction (or at least, that it was someone who had access to the correct private key) by checking the signature. Anyone can also prove that the poster was a payee on the cited previous transaction, by hashing the poster’s public key and comparing that to the hashed key in the earlier transaction. And no one can trace the transaction back to the actual identity of the poster or the payee.
So this public ledger becomes a linked list of anonymous transactions—verifiable, but not traceable to their owners. But this linked list of transactions is not yet a blockchain. As the paper noted, there is still a problem.
It would be virtually impossible to forge a transaction undetected, since everyone in the network is verifying each transaction as it is posted. But an evil player could cheat by spending the same funds twice—posting two legal-looking transactions spending the same coins. The protocol we have described provides no way to verify that a particular transaction is the first to spend the funds from a given source.
In a centralized system like a bank, transactions get time-stamped and—generally—processed on a first-come, first-served basis. But Bitcoin, being distributed, needs a way for all the participants to agree on the chronological order of transactions without a central authority. That is where blockchains come in.
As transactions are broadcast to the network, each official participant gathers them up, verifies them, and packs them into a block (Figure 2). When the block is full, one participant puts the finishing touches on it, linking it to the most recent previous valid block, and broadcasts it. Now everyone has an immutable linked list of blocks that unambiguously establishes the sequence of transactions. If you want to verify that a transaction is using a bitcoin that the owner hasn’t already spent, you only have to walk back through the linked list to the prior transactions on that coin.
There are still some issues, though. What if someone knows that they are going to make the next block, so they falsify a transaction, and then put it into their block? The paper’s answer is to have a random lottery for the right to make each block, so you can never know in advance that you will get to do the next block, and you are highly unlikely to get to make two blocks in a row. If you fake a block, the next participant to win the lottery will try to verify your block, find a problem, and instead of linking their block to yours, they will ignore yours. Now there are two versions of the blockchain—yours and the next guy’s. And the next lottery winner will link to the next guy’s new block, again leaving yours out. A little math in the paper shows that with very high probability, at any given time the longest version of the blockchain—the one verified by the most people—is the correct one. So even a concerted attack by a number of participants can’t get a forged block into the chain, unless they are a majority of the users.
The facet of Bitcoin that gets the most public attention, ironically, is not the linked list of hashed and signed transactions, or the blockchain, but the way it conducts the lottery. The seminal paper’s authors borrowed an idea from previous work on cryptographic currencies to come up with a provably random way of awarding the winner. When it’s time to publish a new block, everyone sets to work on a mathematical problem that can only be solved by trial and error. Specifically, when a new block is posted, each participant validates it. If it is valid, each participant begins searching for a random string—called a nonce—that, concatenated to a hash of the previous block and hashed again, will produce a string with a given number of leading zeros. Hard to find, but easy to verify. The first participant to find a working nonce wins, and gets to form the new block. They will pack in the collected transactions, the hash of the previous block—which links the new block into the chain—and the nonce, so everyone can verify their work.
In the Bitcoin protocol, each lottery winner receives a new Bitcoin for their trouble. They insert into the block a transaction that simply says the new coin is theirs. This means that even if you never intend to cheat, winning the lottery has real monetary value. And lust for money led quickly to an arms race in which participants—who came to be called miners—moved from their home PCs to rooms full of servers to FPGA accelerators to ASIC-based supercomputers in their efforts to be first to find the next working nonce.
Bitcoin seized the imaginations of many people—some seduced by what appeared to be the prospector’s dream of free money, some attracted by the libertarian prospect of making banks redundant, and some lured by the anonymity, which allowed them to conduct illegal businesses with impunity. But something in that point about eliminating banks caught the attention of others. For all its ups and downs, Bitcoin’s blockchain was succeeding at maintaining a decentralized public ledger that no one had been able to corrupt. (The widely-publicized attacks went after not the blockchain but the exchanges—the independent services that maintained private keys and balances for users.) It turned out that there are many potential applications for such a data structure.
Many of these interested parties were, ironically, banks—not because they wanted to eliminate checking and savings accounts, but because of a seeming detail in the way Bitcoin defined transactions. We said that each transaction contained all the information necessary for anyone else to verify it—public keys, hashes, and a signed hash. We didn’t mention that these items are included, along with commands, in the form of a stack-oriented, Forth-style script embedded in the transaction. Participants who want to verify the transaction can just execute the script.
But this script isn’t limited to just verifying a simple transfer of funds. It can be an arbitrarily complex program, using any data that would be available at the time it was executed. You could write a script, for instance, to make the transaction valid only when a set of complicated conditions is satisfied. So you could write scripts that would make the transaction behave like a futures contract, becoming valid only when a stock reached a particular price. Or you could implement much more complex derivative contracts, allowing participants to hedge interest rates, insure international shipments, or nearly anything else two parties might agree upon. The key word here is contract: a blockchain can be a secure, distributed way of trading and administering financial contracts. This could be hugely attractive to a bank that spends a fortune on back-office costs, and whose customers care a great deal about speed and integrity.
But that is quite a different environment from Bitcoin, in at least two respects. First, banks often want trading to happen in a relatively small, closed pool, very unlike the anyone-can-play Bitcoin ethos. The bank may want some special powers in this pool, such as the ability to give priority to certain participants or to alter the blockchain—things that would be anathema to true believers.
Second, in the financial world latency matters. As Shigeru Sasaki, CEO of Fujitsu Laboratories, pointed out in a recent Fujitsu seminar on blockchain technology, the Bitcoin world gets by happily on a rate of about ten transactions per second. In fact the overseers of Bitcoin adjust the number of leading zeros they require in the block hash to limit the rate at which miners find new Bitcoins. So a transaction might not be considered fully valid until it is in a block several layers down in the chain, which might take minutes. But financial traders think in terms of microsecond latencies. They are willing to pay big money to reduce the wire length between their servers and the servers supporting a financial exchange. Such demands preclude something like Bitcoin’s polynomial-solving proof-of-effort approach.
Such issues have spurred interest in building other kinds of blockchain environments. There are controlled-access blockchains, blockchains with central administration, and platforms for developing your own blockchain protocol. Intel, for one, has created an open-source development platform called Sawtooth Lake, currently available in beta form on github. It supports a quite different, and potentially much faster, lottery mechanism. Instead of a cryptographic proof of effort, which, Intel authors point out, requires considerable energy as well as time, Sawtooth Lake uses a proof of elapsed time. This is a piece of code running in secure mode on an Intel® CPU. It generates a random time delay within a prescribed range, as well as a token allowing all other participants to verify that the time delay routine actually ran in the protected mode.
The effect is the same as for Bitcoin’s intense computation—each participant receives a random delay—but without the computation. This cuts the energy consumption and eliminates the advantage of using hardware acceleration to make the computations faster. This, in turn, allows the whole process of creating blocks to go much more quickly, no longer restrained by Bitcoin’s need to keep making the proof-of-effort problem more difficult to manage the rate of coin discovery. And while the latencies are not deterministic, they are at least bounded—an acceptable situation for many kinds of financial trading.
Making the lottery process faster leaves cryptographic computing as the critical path in transaction latency. There are hashes to compute, a private-key signature to compute, and strings to be checked with public keys. A protocol designer can eliminate some of these steps, but only by compromising either the security of the blockchain or the anonymity of the users, or by transferring some responsibilities to a central authority. Alternatively, to further reduce transaction latency hardware crypto acceleration becomes important. In a data-center setting this is not a major issue. But as we will see, not all interest in blockchains will be inside data centers.
Let’s forget about Bitcoin now and examine what, in abstract form, a blockchain is. First, it is an immutable public ledger that can record the nature, sequence, and validity of transactions. Second, it is distributed, and can be implemented so that attack on, or failure of, many nodes cannot disrupt the network. Not even Byzantine attacks can disrupt it. Third, it can keep secure the identities of its participants. Now, what other uses might there be for such a data structure?
The answer depends on how broadly you imagine the word “transaction.” If you think of a transfer of funds, you get Bitcoin. If you think of a contract between two investors, you get a financial trading floor. But what if you think of a transaction as transfer of any sort of tangible or intangible property from one domain to another? In that case, blockchains could be useful for asset tracking—in international trade or supply-chain management, for example. Companies such as Skuchain are exploring these possibilities.
In the applications we’ve discussed so far, the participants have been people or organizations. But what if the participants were things—elements on the IoT? IoT things certainly engage in transactions: providing sensor data, performing physical actions, updating code; even, in mesh networks, forwarding traffic for each other. Companies like Filament, combining wireless network and cryptography expertise, are looking into just such systems.
Finally, a bit of speculation. A network such as the one a blockchain creates is capable not only of securely recording transactions, but of securely storing scripts that can conditionally cause agents to act upon data. In other words, a blockchain can define a secure, distributed computing system.
Embedding code in a blockchain may seem an utterly obtuse way to program an IoT network. But it could have one important advantage. As recent events have illustrated, even the smallest programmable devices on the IoT can be co-opted, either to participate in distributed denial-of-service attacks or to compromise their own systems. A blockchain protocol—especially one using a light-weight lottery scheme—defines the few specific crypto functions a participant must be able to perform. This turns what is rapidly becoming an open-ended countermeasures war for IoT security into a clear and limited set of requirements for each node—requirements that can be kept within the range of an MCU or passed on to a local hardware accelerator if necessary. A blockchain-based programming protocol may be the most effective way of securing some IoT networks.
We have seen an idea start out as a scheme to create an alternative currency. We watched it generalize into a way to implement a secure, immutable, distributed data structure. And we have seen how flexible that concept can become if you use an open-minded definition of transactions—a definition that includes executable scripts within the transactions themselves. From financial services to supply-chain management to securing the IoT, we have not seen the end of blockchains yet.
For Further Reading
See an example of an SHA hashing core on an FPGA.
Read a case study of a blockchain logging solar-energy production data.