Lightning’s Balancing Act: Challenges Face Bitcoin’s Scalability Savior

Jameson Lopp is a software engineer at BitGo, creator of Statoshi.info and founder of Bitcoinsig.com. He enjoys building web services and is intrigued by problems of scale.

In this feature, Lopp examines the Lightning Network, a proposed solution for scaling the bitcoin network and enabling low-cost microtransactions, analyzing the potentially daunting challenges the initiative has ahead.

The bitcoin community has been discussing the concept of the Lightning Network for a year now. It is often cited in scalability debates as a solution to bitcoin's limited transaction throughput capabilities.

However, it's a complicated concept and several parts of the implementation are still theoretical or in development.

This article will attempt to provide more insight into how the Lightning Network will operate and the challenges it will face. The following assumes you have a basic understanding of Lightning Network. If you’re new to this concept then you can check out this explanation in layman’s terms or read the technical details here.

Since the Lightning protocol is in flux, some of my conclusions may already be invalid or may be invalidated in the near future. A variety of models and simulations will be presented throughout this article, but there is no guarantee that any of them will hold true – no amount of theorizing can substitute for real-world data once the network is operational. There are currently four different projects (Lightning, Blockstream, Eclair and Thunder Network) working on code for compatible implementations, but we don’t expect a minimum viable protocol until summer at the earliest.

Some people like to call Lightning Network a caching layer, but it may be more accurate to think of it as a settlement deferral system. If you realize that a healthy economy has value constantly flowing back and forth amongst the participants (nodes) in the economy, you can see that many of the value transfers will actually cancel out and the net change in value distribution around the economic network will be much smaller than the total value transacted.

In order for Lightning Network to succeed, it will need to leverage this economic property – but it won’t be simple.

As a software engineer for a bitcoin wallet, I'm fascinated by theories and proposals to improve the system. Once it’s clear that a proposal is on the path to becoming reality, implementation details become much more important.

After spending countless hours soaking up all the information available about Lightning Network, I’m happy to report that its theoretical capabilities are quite convincing.

However, scalability without usability will prevent us from transforming the theoretical into the concrete.

Macro complexities

While the Lightning Network white paper is quite clear about how Lightning payment channels will operate, there are still many questions regarding how the network itself will operate as a whole.

One of the biggest unknown questions is what the graph of the Lightning Network will end up looking like.

To be more specific: What will the proportion of highly connected nodes to sparsely connected nodes be? A network with mostly well-connected nodes will be more dense and allow traversal of the graph with fewer hops, which will be more efficient.

A network with only a handful of "hub" nodes will be more spread apart, requiring users to either choose longer, less efficient routes through the network or to choose to go through the few well connected nodes. If users are incentivized to use a few well-connected nodes for payment routing, this poses privacy issues and creates more substantial targets for attackers to disrupt the network.

Here is a simplistic example of potential graphs that the Lightning Network nodes may end up forming, courtesy of Paige at MaidSafe.

I suspect that the network shape will be most like the second graph, with some noticeably well-connected nodes.

This is based upon the analysis of monetary flow within the bitcoin network in 2012 that you can read about in the paper "A Fistful of Bitcoins: Characterizing Payments Among Men with No Names."

While bitcoin’s economic network is clearly much larger and more complex today than it was in 2012, the relationships between participants are probably still similar.

Will the incentives of the network result in centralization around a relatively small number of highly connected nodes?

We may be able to predict some of the Lightning Network’s dynamics by examining Bose-Einstein Condensation in Complex Networks.

The gist is this: In networks where the nodes are competing for being the most well connected because it gives them an advantage over other nodes, phenomena such as “first-mover-advantage,” “fit-get-rich,” and “winner-takes-all” result in network centralization as a minority of nodes eventually capture a significant portion of the network’s links.

Anthony Towns ran the above simulation of a simple Lightning Network that shows how value could end up being concentrated at routing nodes over longer periods of time.

It seems that there is an incentive, at least for routing nodes, to be as well-connected to other nodes as possible so that you can route the most traffic and thus extract the most fees. On the other hand, there is a disincentive to run a popular routing node because it would require a large amount of bitcoins to be both locked up in terms of liquidity and yet exposed in terms of private key security.

It remains to be seen where the equilibrium will settle between these opposing dynamics.

It seems likely that Lightning will exhibit a power law distribution of node connectivity that would give it properties of a scale-free network.

This may give us some insight into network's robustness to failure. In a scale-free network the most well-connected nodes are connected to slightly less well-connected nodes, and these nodes are then connected to other nodes with an even less connectivity, and so on.

This hierarchy enables a degree of fault tolerance.

If failures occur at random and the vast majority of nodes are those with low connectivity, the likelihood that a highly connected node would be affected is negligible. Even if a highly connected node fails, the network will generally not fracture due to the remaining highly connected nodes.

On the other hand, if many highly connected nodes fail simultaneously, the network can fracture into isolated graphs. Thus, the Lightning Network can have some centralization around highly connected nodes while still being tolerant of partial network collapse

Another reason why the shape of the network is important is that it could be possible for the network to become imbalanced.

As mentioned earlier, Lightning Network takes advantage of the fact that the net value transfer around an economic network is miniscule in comparison to the total volume. Imagine what happens if a great deal of value flows to a poorly-connected area of the network (an “edge” on the network graph) – it would have nowhere to go.

This could occur if a service experiences a surge in popularity, such if it releases a new killer feature or announces a firesale. If a flood of value was sent to them, then it could unbalance many of the channels in the nearby vicinity, forcing them to either raise fees significantly or to close.

If the shape of the network is reasonably decentralized, then the rest of the network would remain unaffected by mass channel closures.

However, if it looked more like a hub and spoke shape, the network could be severely disrupted. Mass channel closures during periods of low on-chain contention probably won't be a big deal because nodes can simply re-establish a closed payment channel with the same on-chain transaction that closes it.

This situation would be more problematic during periods of high contention for blocks, which we’ll explore later.

Q: How does path discovery work on the Lightning Network?



A: We don't know. It's a hard problem.#ScalingBitcoin







— Jameson Lopp (@lopp) December 7, 2015

The dynamics of the Lightning Network will be further complicated by the fact that value transfers will not always take the shortest path between two nodes.

This is a result of the Onion routing being implemented due to privacy concerns. The specifics of how a node will find a path to route a payment through the network are not yet finalized, thus it’s hard to say what effect routing will have upon the network dynamics.

It sounds like the initial plan is to implement node discovery via IRC channels – the same way that the early versions of bitcoin performed node discovery. We may also see "beacon nodes" implemented which will facilitate path finding, and long term we may see paths stored in a distributed hash table.

If we're thinking adversarially, we may ask, 'Could a wealthy attacker open high-capacity channels on opposite ends of the network and route huge payments through the network, imbalancing many channels and forcing them to close while paying negligible fees to conduct the attack?'

This is not particularly different than a similar attack that can be conducted on-chain with a wealthy attacker filling up blocks, though it could be cheaper to conduct on the Lightning Network.

It remains to be seen how robust the Lightning Network will be against denial of service attacks; it will depend upon how much liquidity is available from other nodes in the vicinity of a "value flood" and how intelligent the nodes are about adapting to such a flood and routing value around the network to rebalance channels.

Lightning developer Tadge Dryja noted that fees will increase asymptotically as channels become more unbalanced, raising the cost of routing through an imbalanced area of the network, causing it to become prohibitively expensive for someone to sustain a denial of service attack.

If a substantial portion of nodes fall off the network, how does the network heal?

As stated earlier, this would mainly be problematic if it is difficult to re-establish channels with on-chain transactions. The question of network health is tricky because the various teams now developing Lightning Network implementations are doing so with privacy in mind.

As a result, it probably won't be possible to systematically map the structure of the entire network. If this is the case, then it will also be difficult to develop tools that allow us to determine the network health in terms of the graph connectivity and channel balances. Thus we may only know that the network is being disrupted if the number of hops to route to a specific node suddenly increases or it becomes impossible to find a route to that node.

The ability for the Lightning Network to recover from partial network collapse (due to channel exhaustion) will depend upon the size of the network (the number of open channels) in relation to the number of on-chain transactions that the bitcoin network is able to accommodate.

The larger the Lightning Network grows in terms of number of channels, the less resilient it will be to network disruption due to events that force mass channel closures because of the limited capacity for the bitcoin network to support reestablishing the channels.

In a situation where many nodes are forced to reestablish channels but block contention is high, the channels will be essentially useless (unidirectional) until an on-chain transaction can be confirmed in order to re-establish the channel with a balanced value.

It may be advisable for network participants with a lot of liquidity to operate nodes in different parts of the network graph so that if such a situation arises, they can route money from the unidirectional imbalanced channels to their other node in order to rebalance some channels and mitigate the disruption.

If cascading channel imbalances coincide with high contention for block space, we will be forced to deal with the problem of lock times for revocation windows.

It has been theorized that in the event of full blocks, we could add logic such that a "full block" won't count toward the locktime, but at time of writing there is no foolproof way to designate which blocks are full. Blockstream co-founder and bitcoin developer Greg Maxwell has suggested that miners could set a flag, but this means it’s up to miners to agree how to designate blocks as full.

It seems unlikely that such a flag would be a consensus rule, thus its reliability would be limited.

Guarding against these high-level network issues will require meticulous engineering, but there is no reason to believe that they aren't solvable.

Micro complexities

It's clear from the white paper that a great deal of game theory goes into making individual payment channels and routing of payments across channels robust against counterparty theft. However, there will be even more complexity that will need to be handled at the individual node level in order to maximize long-term efficiency of the channels it is maintaining.

Users may be able to earn a return on their bitcoin by running a well-connected, reliable node and routing many payments through it.

It will likely be a brutally competitive fee market as barriers to entry are minimal and there is not much to distinguish one node from another other than the available liquidity and the fee charged. Think of it like bitcoin mining but even more competitive because you won't need custom hardware or lots of power, just an online computer and some bitcoin.

Nodes will want to keep the value in channels balanced in order to prevent them from becoming unidirectional, requiring closure of the channel onto the blockchain – a relatively costly operation.

The optimal fee schedule on a local and global scale is one where transfers that unbalance channels (lead to a more lopsided channel with one party having more than half the channel allocation) are charged a higher fee than those that help balance a channel.

In fact, if you are running a routing node and one or more of your channels has become unbalanced to the point that you may need to close it soon, it could make economic sense for you to offer a negative transaction fee to route a transaction that could rebalance the channel, thus preventing you from having to pay a much costlier on-chain transaction fee to close the channel.

A lot of literature and presentations about Lightning Network tout that sequence number-based channels can remain open indefinitely.

This is true within the context of a single channel, but will probably not be true in the broader context of the entire network. This is because routing fees will degrade channels over time.

Imagine that you operate a routing node with multiple channels that have a total capacity of 10 BTC and you initially funded them with 5 BTC of the total value. If you collect a 0.01% fee on each transaction, you'll gain 5 BTC after routing 50,000 BTC and you won’t be able to collect any more fees for routing payments. You could still route payments for free or negative fees, but you’d likely want to close the channels and re-establish them in order to continue collecting fees.

There are also questions of liquidity given that Lightning users will need to determine (and preferably overestimate) the amount of value they want to dedicate to the network.

This will likely be dependent upon a number of variables such as the use case for each channel, whether or not the node will be routing payments, and the length of time the user is willing to keep a channel open.

Anthony Towns

predicts we’ll see something along the order of:

• $1-$10 for Internet of Things devices sending micropayments
• $10-$100 for routers/cloud computers
• $1,000-$5,000 for merchants doing reasonable volume
• $1,000-$100,000 for nodes operating as revenue generating investments.

How many payment channels will the average user keep open?

This is hard to answer because ideally, it will be handled under the hood by wallet software. To my knowledge, the logic behind how wallets will decide with whom to establish channels has not been explored. For ease of use, this would preferably be automatic, but then wallets would also need a way to find Lightning nodes that have bitcoins available with which to establish a new channel.

It's unlikely that we will see many high value channels with capacity greater than several thousand dollars – Lightning users who have a lot of value dedicated to the network will spread that value across many channels and possibly even multiple nodes in order to facilitate rebalancing.

While payment channels are bidirectional, they need not necessarily be funded in a balanced fashion if the channel is not being operated by a routing node. Merchants and payment processors may be incentivized to establish many channels that are initially funded by other parties – these channels will only really be used for receiving funds.

On the other hand, consumers will mostly establish channels that are only funded by themselves and are primarily used for sending funds. Payment processors such as BitPay may end up opening many channels to receive payments and a few to send payments to other entities such as exchanges, but they may opt to not route payments due to AML/KYC concerns.

Any of the above usage of the network would be suboptimal; it sounds like Lightning developers want wallet software to default to opening multiple channels and default to routing payments if possible, since it would be best for the network’s health. Tadge Dryja has noted that Lightning's implementation does not automatically match funding for an incoming channel by default. Without two-party cooperation, imbalanced channels are automatically created when a payment is sent.

There will also be interesting dynamics at play when an entity with multiple open channels is about to have one closed by the counterparty.

When a channel is about to close, you will need to decide if you want to:

• Try to keep as much of your value still on the Lightning Network
• Take as much as possible off
• Keep the other channels as-is.

If a counterparty is closing a channel and you still have a fair amount of value in it that you want to retain, you may send it through them back to yourself on a different channel.

On the other hand, if you want to close a channel and move as much value out of Lightning Network as possible, then you may send funds from another channel back to yourself onto the channel you’re closing.

There will be an incentive for operators of Lightning Network nodes to also run bitcoin nodes so that they can monitor the blockchain for revocation and sweep transactions. Otherwise, anyone who is participating in a Lightning Network channel is taking the risk that they will be defrauded by the channel’s counterparty.

Of course, we might also see people using third-party services to manage Lightning Network payment channels on their behalf, which would include running a full node to monitor the blockchain.

You can give the sweep transaction to third-party companies and as long as they notice malicious activity by your counterparty and sweep on your behalf, you’re safe. You could further incentivize the third party to monitor the blockchain honestly by adding an output to the transaction that pays a fee to the third party.

Will it be possible for third parties to manage payment channels in a non-custodial fashion? This is especially relevant for my employer, BitGo, because we consciously avoid custodianship.

It’s clear that it will be simple for third parties to trustlessly monitor the blockchain for commitment transactions on behalf of a Lightning Network user. It would be preferable if we can use multi-sig to facilitate updating the state of a Lightning channel, though latency requirements may pose an additional challenge.

On the bright side, according to Lightning developer Joseph Poon, it is theoretically possible to have a third party act as an escrow agent for a Lightning channel.

How will users operate payment channels if they’re not online 24/7? You could have a channel that you create for occasional payments and only hop online when you want to buy something. The trickier part is that if you want to receive a payment, you also have to be online.

This means that a Lightning node that is usually offline should not establish a channel with a node that isn’t constantly online. Will the inability to receive payments while offline result in users choosing to allow custodians to operate channels on their behalf? This remains to be seen, though the ability to run 2-of-3 multisig may make it a safe option.

As covered in a previous post, the cost of running a bitcoin node greatly affects how many users end up deciding to run one. There will be many complexities to running a Lightning node and developers should strive to abstract away as many of them as possible in order to maximize the ease of use and thus the time investment involved in operating a node.

New network, new dynamics

With a new complex system that has its own dynamics will come many unknowns, and people tend to fear the unknown. Some have argued that a successful Lightning Network will weaken the bitcoin network because it will "steal" fees from miners.

This stance seems short-sighted, however, because if Lightning Network allows a user to reduce their on-chain transactions by a factor of X, then it makes economic sense for a user to be willing to pay any fee that is less than AVERAGE_BITCOIN_FEE * X - AVERAGE_LIGHTNING_FEE * X.

Also, Lightning Network fees will “trickle down” into bitcoin fees. As mentioned earlier, fees collected by Lightning nodes that route payments will degrade the amount of value available to be routed through the channel until it is exhausted and must be closed.

The node will have to pay a bitcoin fee to do so, even if they use the on-chain transaction to re-establish a channel with the profits from fees reinvested back into the Lightning Network. Then if the node decides to spend the profits on the Lightning Network, this will generate Lightning fees on other nodes that will eventually trickle down into bitcoin fees when those channels are closed.

Finally, a successful Lightning Network will enable completely new use cases that are not possible on-chain, bringing more economic activity (and mining fees) to bitcoin as users open and close channels.

An interesting result of Lightning’s reliance on on-chain transactions is that a wildly popular Lightning Network could create more demand for on-chain transactions than it offloads, drastically increasing the on-chain fees that users are willing to pay.

Currently, bitcoin users can protect themselves from exchange rate volatility by buying right before sending a payment and a merchant or payment processor can sell shortly after receiving it.

In the Lightning Network, users and payment processors will need to hold bitcoins for weeks or months. We'll likely see exchanges offer Lightning deposits in order to facilitate near-instant conversion. Another potential solution would be atomic cross-chain swaps to a fiat-based crypto asset.

Earlier we noted the liquidity problem of needing to lock up bitcoins in a channel for an indeterminate period of time. What if you need to make an on-chain payment, but all of your coins are locked in the Lightning Network and you don’t want to close your channels? We’ll likely see people offering off-ramp services in order to solve this problem.

You could send a Lightning payment to service X and they’ll make an on-chain bitcoin transaction on your behalf.

We may also see "liquidity provider services" arise, probably also run by exchanges. If, for example, you have a Lightning channel open that has become unbalanced, you could send money out-of-band (such as via ACH transfer) to a liquidity provider and they’d route value to you over the Lightning Network to rebalance the channel.

This could save people the hassle of making as many on-chain transactions to close and reopen their channels, though it does involve trusting a third party to send the funds over Lightning Network. In fact, these liquidity providers may be crucial to sustaining balanced channels over long periods of time, according to further simulations conducted by Anthony Towns.

In each of the following simulations we have four people (A/B/C/D) who each buy a coffee each day from a random barista (E/F/G/H) via Lightning. In the first simulation, routing nodes are underfunded, A/B/C/D have no income received via Lightning, and E/F/G/H have no expenses they can pay for via Lightning.

As a result, network activity grinds to a halt when the routing nodes become unbalanced.

In the second simulation, routing nodes are sufficiently funded, A/B/C/D have no income received via Lightning, and E/F/G/H have no expenses they can pay for via Lightning.

As a result, network activity grinds to a halt when A/B/C/D go broke and E/F/G/H's wallets are full.

In the third simulation, routing nodes are sufficiently funded, A/B/C/D have no income received via Lightning, and E/F/G/H have no expenses they can pay for via Lightning.

However, an exchange (node X) is added, and A/B/C/D sell fiat for more lightning funds when they come close to being broke, while E/F/G/H sell lightning funds for fiat when their channels become unbalanced in their direction.

Having a fiat exchange seems the most sustainable way to enable channel rebalancing by allowing income to enter and exit the Lightning Network.

Theoretically, a complicated closed system (the baristas buy coffee beans from someone, who buys farming equipment from someone else, who buys hardware supplies from one of the people who drink coffee) could also work, but it won’t be feasible while the network is being bootstrapped.

It’s clear to me that an efficiently operating Lightning Network has huge potential, but there are many challenges that need to be surmounted in order to ensure that it is also robust and easy to use.

This is no different from bitcoin or even the Internet in their infancy.

Just as with those other network technologies, we’re more likely to see adoption if the complexities are hidden from users under layers of software. As Lightning matures, I expect to see it enable new classes of economic interaction and usher in the dawn of yet another economic revolution.

Disclosure: In an effort to improve the technical analysis, Lightning Network developers provided feedback on this article.

Electrocution image via Shutterstock