Evaluating Smart Contract Security

Nov 9, 2020

“Code is law”. This (controversial) mantra illustrates one particular property of a trustless system based purely on code: There is no direct recourse against exploits of bugs in smart contracts. As such, smart contract risk is a major factor to assess when using blockchains with such capabilities.

During the recent hype around decentralized finance (DeFi), hundreds of millions of dollars flowed into smart contracts, often even ones without an audit by reputable firms. Part of the DeFi yields can be argued to come from the fact that smart contract risk always remains – any piece of code can contain bugs. Thus, the question becomes: How does one judge how risky a certain smart contract is?

Smart Contract Audits Traditionally, the first factor to look out for is an in-depth, high quality audit (or multiple audits) which tries to find possible exploits. On a technical level, these can simply be bugs in the code that would lock funds in the contract or leave them at risk of being stolen. However, the economic dimension also needs to be considered: Are there any design choices that allow a malicious actor to exploit the system for profit?

Additional consideration should be given to the level of decentralization (do so-called admin keys that grant full power exist?) and the potential impact of governance decisions. Governance of protocols is often in the hands of token holders (e.g. in a decentralized autonomous organization, a DAO). In this case, security measures such as a time delay for the implementation of governance decision can give protocol users enough time to react.

Overall, however, the assessment of smart contract risk remains highly complex and only a small minority of crypto users will be able to properly judge it for themselves. Thus, a way to gauge the security of a smart contract more simply could complement the fundamental approach via audits.

The Game-Theoretical Approach From a game-theoretical perspective, the security of a smart contract is a function of its time in existence as well as the total value locked/secured by the contract. This can be illustrated with an intuitive example: Imagine a situation where one must place $10M into one of two smart contracts. Each of the two smart contracts already hold $100M, but the first smart contract did so for one year, whereas the second contract is only one week old. The logical choice, given no other information, would be to place it into the first smart contract. The example can also be inverted to account for value locked, where both exist for one year, but one has $100M locked and the other one only $10M. Mathematically speaking, the smart contract security is the integral over time of the (time-dependent) total value locked function.

Illustration 1: Smart contract security increases both with time passing, as well as with a higher amount of money secured by the smart contract. It can be interpreted as the entire area below the total value locked curve.
Source: Bitcoin Suisse Research.

Smart contracts that secure large amounts of money are essentially a perpetual bug bounty program, where the maximum bounty is the value locked in the smart contract. Flash loans – which enable anyone to borrow unlimited amounts, although only if the loan is returned within the same block/transaction – have made exploiting bugs more capital efficient. The bug bounty program is hence open to anyone in the entire world, irrespective of available capital.

The DeFi boom in the recent months has boosted these bounties to multiples of their previous levels. While Maker accounted for the majority of value locked in 2018/2019, since the start of liquidity mining in Compound, all major protocols have increased their total value locked (TVL) to >$500M.

Illustration 2: The total value locked (TVL) – and thus the “bug bounty” for finding catastrophic exploits – has skyrocketed in the last months. In the years 2018 and 2019, Maker has been the undisputed king of DeFi by TVL.
Source: defipulse.com, Bitcoin Suisse Research.

Based on this data and the considerations above, it is possible to generate a “ranking” for smart contract security of DeFi protocols by summation of the value locked over time. The top 15 protocols by TVL at the time of writing were considered, but the methodology is readily scalable to hundreds of contracts.

Illustration 3: In the ranking of smart contract security by the methodology outlined above, Maker, Compound, and Synthetix take the top spots. The score of Maker is normalized to 100.
Source: defipulse.com, Bitcoin Suisse Research. The formula for calculating the Security Score is: 100*Sum of daily TVL(project)/Sum of daily TVL(Maker).

In this model for security, the top 5 spots are occupied by DeFi heavyweights Maker, Compound, Synthetix, Aave and Uniswap.

Shortcomings of the Model While this model is appealing due to its simplicity of both design and security score calculation, the simplification comes with multiple tradeoffs and should hence only be viewed as a rough estimate of how trustworthy a smart contract is.

To list a few of the tradeoffs or approximations that the model makes:

  • Smart contract upgrades and upgradeability are not considered (such as, for example, Maker’s switch to multi-collateral DAI from being solely ETH-backed)
  • It does not consider past exploits and subsequent bug fixes, which should reset the score to zero
  • Composability of DeFi is not considered – for example, Yearn (#7) uses a multitude of other protocols on the list in the background and should thus never score higher than any of the protocols it uses

There are other refinements that could be made, such as evaluating whether time in existence should be weighted more strongly (“Are $100M locked for 1 week as valuable for security as $10M locked for 10 weeks?”) or if there are diminishing marginal benefits to TVL once it crosses a critical threshold, arguing that a $100M “bug bounty” attracts as many capable eyes as $1B does.

Additionally, the model may also be extendable to smart contract wallets, as well as smart contracts that do not “lock” value inside directly, but handle large volumes of transfer (such as relayer contracts).

Conclusion The model of evaluating smart contract security from a game-theoretical perspective and attempting to gauge it through a combination of time in existence plus value locked is simple in its design and highly scalable, since all necessary data is readily available on the blockchain. Thus, it may serve as a quick, first estimate of a smart contract’s security. However, a thorough assessment of smart contract risk remains a complex problem, and a more fundamental analysis of the risk should always complement the model.

Bitcoin Suisse