Points of centralization in Ethereum – Part II
Sep 28, 2022 - 9 min read
The missing weak spots
In a civil case filed Monday, 19th September 2022 against a crypto influencer, the SEC claimed that the U.S. government has jurisdiction over all Ethereum transactions since ETH contributions “were validated by a network of nodes on the Ethereum blockchain, which are clustered more densely in the United States than in any other country. As a result, those transactions took place in the United States.” As of writing 43.37% of the nodes are located in the U.S. and thus evidently lack jurisdictional diversity. Given that the top three staking services (Lido, Coinbase, Kraken) amount for >50% of staking ratio, there is a clear case to push heavily for more decentralization.
Governments are good at cutting off the heads of a centrally controlled networks like Napster, but pure P2P networks like Gnutella and Tor seem to be holding their own.
Satoshi Nakamoto, 2008
As we outlined in Part I, staking services, hosted nodes, stablecoins and a wide range of other less severe points of centralization such as oracles, hosted source code or rollups represent points of centralization in the Ethereum tech stack. These single points of failure might open the door for censorship. To complete the list of weak spots, we now introduce missing important gateways of centralization. Among them are MEV relays, block builders frontends, and infrastructure services such as RPC endpoint providers Infura or Alchemy.
As PoW has come to an end on Ethereum, PoS validators now earn transaction fees from the execution layer (EL) in addition to rewards from the consensus layer (CL) that stem from protocol issued inflation rewards. EL rewards can be referred to as a combination of tips and rewards generated through Maximal Extractable Value (MEV, previously Miner Extractable Value). Technically, validators receive EL rewards as additional rewards for prioritizing and/or reordering transactions. MEV happens primarily on decentralized exchanges (arbitrage and sandwich attacks), on lending protocols (liquidations) and on the NFT market, see Illustration 1. According to Flashbots that currently provides the most adopted relay, the total extracted MEV since January 2020 is at $675,5m while $1,4m was wasted on failed MEV transaction fees.
Illustration 1: Profit by MEV type and corresponding transaction count
MEV refers to not just relying on CL rewards from the protocol and user transactions but finding opportunities to generate rewards from changing the transaction order to build blocks with the highest possible economic value. With MEV, validators are able to outsource parts of the block production pipeline to increase their staking rewards. In PoS, the solution to toxic MEV is labelled outsourced Proposer Builder Separation (PBS), as it allows validators to equally gain MEV rewards without facing permissionless mempools and off-chain deals. Since the Merge, this block production pipeline, see Illustration 2, consists of searchers and their own private order flow, that find MEV opportunities and bundle up transactions, builders that aggregate transactions and craft the most economically sound blocks, and proposers also known as validators, that receive the blocks (execution payload) via relays. As the proposer only sees the block header, the relay's role is to estimate the MEV-related value of each block and to verify the validity of the block body so that the validators don't have to depend on the builders directly. The relay therefore acts as a trusted escrow between proposers and builders.
Illustration 2: Post-Merge block production pipeline
When it comes to MEV relays, there are currently 7 active relays aiming to find the most economically sound blocks, see Table 1. Relays leverage a middleware called MEV-boost that handles communication, the profit-switching logic, and a fallback mechanism in case of system failure. MEV-boost is a post-Merge PBS implementation built by Flashbots for validators aiming to maximize their rewards by selling blockspace to block builders. Proposers using MEV-boost were able to boost their staking reward by 135% post-Merge as an average MEV-boost block yielded 0.2109 ETH compared to 0.0898 ETH for vanilla blocks.
Table 1: Active relays that found at least one block post-Merge
As Table 1 indicates, Flashbots’ relays are a clear outlier with 7’617 found blocks so far. Interestingly, BloXroute offers various approaches with regard to maximum profit (neutral), ethical and regulatory compliant whereas the maximum profit approach attracts most proposers. Flashbots relays are flagged as regulatory compliant while e.g. Manifold is not only neutral aiming for maximum profit but also enables access to MEV rewards via their platform token. Despite having only built 23 blocks yet, their average block value is outstanding at 1.44 ETH, up 6.5x compared to the second most valuable blocks producing relay (data could be subject to errors). Manifold realizes this via their private order flow routed through its own RPC called SecureRPC, acting as an aggregator on top of Flashbots. This might snowball as builders with the highest valuable blocks get included more consistently and that might attract even more private order flow, likely leading to builder centralization.
Despite MEV relays lowering gas prices for users by moving gas auctions off-chain, the current interim solution of outsourced block production poses risk of builder centralization and trusted relays. By running middleware such as MEV-boost however, the centralizing force of MEV gets isolated to builders where it’s easier to address (see ”Preventing weak censorship in the block production pipeline” below). The current design reinforces builder centralization as a result of profitability and opens the door for censorship within the block production pipeline. Aside from censoring transactions, builders might also unpack bundles and steal MEV from searchers. According to a model of pmcgoohan on ethresearcher, builder centralization might even boost validator centralization. Validator centralization is a possible feedback loop induced by higher staking APRs attracting more validators. Bigger staking pools likely have more opportunities to extract MEV and might furthermore be able to even extract more out of these opportunities via proprietary optimizations. While the current trust model is an improvement compared to the PoW era, trusted relays could still withhold execution payloads from validators, or unpack and steal MEV from builders and hence pose a risk to both builders and validators.
RPC endpoint providers and frontends
The Tornado cash sanctions also exposed the weak spots within infrastructure providers and frontends. Infura and Alchemy, centralized API and node infrastructure providers, started blocking API access for Tornado Cash. Infura is the largest RPC (Remote Procedure Call) endpoint provider in the Ethereum ecosystem and service provider for MetaMask, the most popular wallet for Ethereum. To access the blockchain, users can either run a node on their own, or connect to a public node via the node's RPC endpoint. With many decentralized Applications (dApps) and users relying on Infura as their node provider, Infura became a central point of failure for the ecosystem. As these services are fully centralized, governments and other third parties can track and censor operations.
Frontends were taken down as well rendering Tornado Cash inaccessible except for direct smart contract interactions. Moreover, Curve Finance’s frontend got exploited in August, showcasing how frontends represent single points of failure too.
Preventing censorship across the spectrum
Crypto and DeFi are heading towards an inflection point as regulation and sanctions are threatening censorship-resistance. It is now on the industry and users to signal whether decentralization or convenience is more important, an uphill battle if fighting for decentralization. However, as we outline below, there is light at the end of the tunnel. We shed light on solutions to maintain censorship-resistance beginning with weak censorship in the block production pipeline, see Illustration 2, followed by solutions regarding weak censorship happening above the protocol level and finally, outlining solutions if Ethereum faces strong censorship.
Preventing weak censorship in the block production pipeline
As learned in Part I, censorship ranges from weak to strong censorship. When it comes to weak censorship, the block production pipeline offers a multitude of attack vectors. For instance, Bitcoin has an advantage in terms of weak censorship as it comes with simple Unspent Transaction Output (UTXO) transactions w/o MEV to be extracted. The whole pipeline therefore collapses down to self-building.
Let’s start at the genesis of block production, the mempool, a pool of pending transactions waiting to get included into blocks. A highly elegant way to mitigate centralizing forces here is a concept called encrypted mempool. Whereas censorship-resistance lists and MEV smoothing, see next paragraphs, will improve risks in the block auction process, encrypted mempools improve the transaction auction aspect by encrypting transactions before broadcasting. Thus, applying censorship is almost impossible as it prevents censors from knowing the content during block production. It furthermore removes toxic MEV and frontrunning. Encrypted mempools might be even able to completely remove builder centralization risks from the picture as the vast majority of transactions can be encrypted essentially making it white noise. On the proposer side, sophistication stays lower as well since proposers simply keep proposing the highest paying block. Despite encrypted mempools being highly promising, it’s currently limited to being concept only yet might be implemented as future mechanism.
Moving further in the pipeline, we can rule out searchers in Ethereum’s block production as they pose no risk of censorship. Builders next in the pipeline pose a threat to censorship though that might be even reinforced by builder centralization. In the case of censoring builders however, non-censoring proposers are still able to build their own blocks yet miss out on MEV rewards. A hybrid solution to mitigate that opportunity cost is self-building but still proposing blocks of builders where the value delta overcomes a certain threshold. According to Ethereum researcher Justin Drake, only 10% of proposers building on their own is enough to maintain a decent user experience by mitigating delayed transaction inclusion. Builder centralization can not only be addressed by encouraging competition among builders but also by various tools such as censorship-resistance or transaction inclusion lists. It’s mechanism to fight censoring builders w/o incurring a financial loss and a measure against a massively centralized block building market putting more power back into the hands of proposers (validators). Proposers are thus able to force builders to include certain transactions in blocks with empty blockspace that are likely being censored. It’s a constraint. The proposer is basically constraining the builders to comply or their blocks won’t get validated.
The next piece in the pipeline are centralized and trusted relays, the mediators between block proposers and block builders. First and foremost, these relays are an interim solution and will get wiped out with enshrined PBS in a network upgrade down the road. Enshrined PBS provides the same separation of powers, allows for easier builder decentralization, and removes the need for proposers to trust anyone. However, according to estimates, the implementations will take at least 18 months post-Merge to implement. Despite one non-censoring relay being already enough for non-censoring proposers to migrate, relay diversity is an important factor. Therefore, the dominating relay Flashbots, flagged as compliant, just recently open sourced their codebase to enable anyone to spin up a server and boost adoption of non-censoring relays. Yet, achieving a diverse basket of relays is challenging as it is complex to run these servers while server operators must be trusted. Overall, regulatory compliant and censoring relays play into our hands as they will likely face social backlash from the community and be less competitive as they lose MEV opportunities. As we point out below, proposers also have the option to opt out completely from relays, searchers and builders and build their own blocks or leverage tools such as censorship-resistance lists. As an intermediary measure before enshrined PBS, there is also research about removing trusted relays using threshold encryption schemes. Third-party block builders could furthermore be enabled to directly sign and propose blocks to validators in a private manner such that centralized relays are neither able to censor blocks nor steal the MEV in blocks. Overall, non-censoring relays, as indicated by Table 1, will likely yield the highest block rewards and thus encourage validator nodes to choose censorship-resistant relays.
As we learned, censoring searchers, relays and builders are either a non-issue or preventable, but what about censoring proposers? When it comes to proposers, a technique called MEV smoothing developed in MEV research reduces the role of proposing a block in a slot to one single bid. With MEV smoothing, proposers are forced to either accept the top paying bid from a builder or do not get to include the block and lose all the rewards associated with the block. The enforcing mechanism lies in the attestations. Attestors in the committee are only attesting a block if they observe it to be the most economically sound block. By that, they are essentially forcing the auction to be fair and prevent proposers from censoring as they would face massive opportunity cost. Moreover, it is likely that the most economically sound blocks will stem from neutral relays, as implied by Table 1.
To sum up, technology is available to render weak censorship within the block production pipeline ineffective. Even in the case of high builder centralization, proposers will be able to force through transactions.
Preventing weak censorship above the protocol level
Above the protocol level, fighting weak censorship can be accomplished by addressing issues regarding RPC endpoints and frontends. For instance, there is a clear roadmap to decentralize infrastructure via in-browser light clients instead of going through RPC endpoints like Infura or Alchemy. In PoS, light clients that figure out the tip of the chain via sync committees are way easier to build and it’s reasonable to expect that MetaMask will move to light clients post-Merge therefore enabling users to route around and avoid endpoint censorship. Moreover, Infura itself just recently announced plans to launch a decentralized, open source protocol to connect dApps to Ethereum.
Regarding censoring frontends, it is possible to work around with other frontends using IPFS, ENS or frontends hosted in other jurisdictions. For example, there are dozens of different ways to access Uniswap, via centralized frontends like Zapper, Zerion or the IPFS version of Uniswap. ENS is an alternative domain solution yet does not have perfect integration as of writing. Over time however, more browsers will likely have better ENS integration and leverage IPFS and solutions like Filecoin for static content. For dynamic content, there are solutions available like the Graph, a web3 indexing in a decentralized fashion. Maybe a DAO could even come up to curate and maintain a decentralized set of frontends.
Preventing strong censorship
Now that we solved weak censorship on and above the protocol level, let’s dig into strong censorship, that fully censors transactions if successful, and how to prevent it. Strong censorship is a special type of 51% attack compromising the liveness of the chain. You achieve this kind of majority by either buying 51% of the stake or by coercion. In order to fight strong censorship, there are two measures available: preventive measures and recovery measures.
Preventive measures aim to raise the bar achieving a position (51% of stake) where strong censorship is even possible. They primarily include improving jurisdictional and operator diversity by e.g. activist staking. As the name implies, activist staking aims to improve diversity by actively restaking, deploying fresh stake or unstaking. If it is jurisdictionally not possible to stake, liquid staking derivates might offer a reasonable solution to keep earning rewards. Arbitrage will likely bring competitive yields for liquid staking derivatives to direct staking after withdrawals are enabled with the Shanghai upgrade. Activist staking is akin to the ecosystem successfully pushing for client diversity within the last years. Notably, preventive measures mostly happen on the social layer since every individual network participant can improve critical metrics as they vote by stake allocation and their respective actions in order to represent Ethereum’s values of maintaining credible neutrality. It is therefore mandatory to prevent choke points such as centralized staking. For instance, when summing up the amount of staked ETH across Lido, Coinbase, Kraken, and Staked alone, 56.57% of staked ETH currently resides in service providers directly or indirectly under the jurisdiction of the U.S. government. Thus, not only plurality in staking pools is of great importance but also coercion resistance. For instance, Lido and Rocketpool come with higher coercion resistance since they are DAOs, therefore network states and in a way their own jurisdictions. Despite complexity, solo staking is another elegant way to increase censorship resistance at a risk minimized interest rate. Things might tilt to solo staking in the future, as many improvements are in the pipeline to remove barriers:
- Enshrined PBS removes requirement for validators to trust relays
- MEV smoothing removes variance of MEV
- Single slot finality make deposits and withdrawals much faster
- Statelessness, DA sampling and zk EVMs will significantly reduce hardware cost
- Technology that will reduces the size of necessary staking size
- Privacy preserving deposits and staking
- Anti-slashing hardware that tracks previously signed messages to avoid accidental slashing
If we face a successful strong censorship attack, recovery is the second and last resort solution to fight back. PoW is better in that regard because it is possible to easily repoint hashrate after an attack. However, if there is a strong attacker with inexhaustible resources such as a nation state, Bitcoin might face problems even if hashrate is repointed. In PoS, once an attacker has 51% of the stake under control, they can even censor deposits and withdrawals themselves rendering it impossible to deploy fresh stake or restake. If an attack is successful, how do we recover then in PoS? Say Coinbase, Kraken and Lido (note that Lido is a DAO and consists of 28 independent operators) that currently make up slightly more than 50% of the staked ETH decide to censor. At a high level, a minority fork away to a fresh version of Ethereum by censoring the censoring entity offers relief. The minority is, in that case, opting out of the censoring majority and in the process redeclares sovereignty and credible neutrality. Coordinating a minority hard fork is challenging though as it highly involves the social layer to enshrine recovery measures. The goal therefore is to mitigate emerging chaos by streamlining everything on the protocol level as good as possible.
A promising approach is to prepare code that automatically creates a fork when detecting censorship with enough lead time. Such a fork would come with a reasonable time horizon to enable the unwinding of centralized DeFi positions such as USDC or USDT. In a first instance, that fork would let censoring entities leave via the exit queues only suffering penalties from inactivity leaks. With another attack, slashing could ramp up until 100% of the attacking stake would get slashed.
For other, harder-to-detect attacks (notably, a 51% coalition censoring everyone else), the community can coordinate on a minority user-activated soft fork (UASF) in which the attacker's funds are once again largely destroyed (in Ethereum, this is done via the "inactivity leak mechanism"). No explicit "hard fork to delete coins" is required; with the exception of the requirement to coordinate on the UASF to select a minority block, everything else is automated and simply following the execution of the protocol rules.
Vitalik Buterin, 2020
Yet, a minority hard fork that aims to social slash represents a potential trap as it might enable the censorship that it intends to combat, might risk funds of innocent users, and violate their property rights. Interestingly, the optionality of a recovery fork is a preventative measure in itself as the sheer fact that the infrastructure to defend such an attack exists might prevent these attacks, similar to mechanisms in the optimistic rollup design.
Apart from the protocol level, all the other weak spots we identified offer solutions to improve decentralization and censorship-resistance too. As such, rollups will eventually be decentralized. A viable approach to overcome the sequencer problem for example is by implementing (delegated) PoS-like mechanisms. Moreover, fraud proofs for Arbitrum and Optimism will be enabled, leveraging the full security of Ethereum. Oracles like Chainlink aim for decentralization. There will likely be greater emphasis on reducing reliance upon custodial stablecoins in favor of more decentralized stablecoins or reserve assets that are un-censorable and/or seizure-resistant. Non-custodial projects such as Liquity with fully decentralized front-ends, ETH as sole collateral and immutable smart contracts w/o governance might offer a viable permissionless and censorship resistant solution. Moreover, non-custodial stablecoins will likely aim to be less reliant on centralized collateral for stability and, similar to MakerDAO, will strive to unwind exposure.
Conclusion and outlook
Cryptocurrency has the potential to take the issuance and control of money away from the state. As history has proven that a vested interest to control this endeavor will likely emerge, it is essential to take preventive measures and have quick-witted answers to looming censorship attacks even if some do not yet realize the benefits of decentralization until tail risk appears. Despite 432’089 validators running the network post-Merge, Ethereum faces a multitude of attack vectors and single points of failure threatening centralization and censorship across its tech stack. As we learned in this two-part series, however, there are either solutions in place, in development or possible if needed.
Censorship-resistance is mandatory. If Ethereum wants to be a self-sovereign public good with secure blockspace and equal access for everybody, it must have immunity from nation states. Ensuring censorship-resistance will to a significant extend be up to the social shield being the end users and the Ethereum community actively choosing and supporting permissionless applications and services built atop Ethereum. Improving operator as well as jurisdictional diversity to maintain Ethereum’s credible neutrality and anti-fragility will thus heavily rely on a conscious social layer.
Disclosure: at time of writing, the author holds ETH and FOLD.