Risk Scores
This section provides a model for developing a risk score for a project based on the answer to many questions. The score is a combination of the network consensus, architectural, implementation, and operational risks. A protocol that scores well in all categories will have a high overall score. However, a protocol that scores well in three of the four categories, but scores very poorly in one, will have a low overall score. This reflects how risks materialize in Web3 projects: small single point issues can cause great projects to suffer catastrophic failures.
How the risk score questions below are answered is subjective. Whereas one person may interpret that a project has complied with a certain consideration, and hence should answer, "yes" to a question, someone else may feel that the project partially complies, and hence the answer should be "no".
A project will receive different scores depending on the information available to the person reviewing the project. As such, it might be expected that a project team that has all the information about a project available to them may be able to score a project accurately. However, a user or external reviewer may not have all of the information available, and may have to make some assumptions. The internal project team may not have adequate perspective or experience to review their own project, hence, even a project team may not be able to provide a completely accurate score.
A project is likely to receive different scores over time. Based on this risk framework, the project team may take steps to improve their score. Issues that have been undetected could surface and reduce the project's score. Additionally, a project team might issue an upgrade of their protocol. This could invalidate the projects previous risk score.
Determining the trustworthiness of one crosschain protocol relative to other protocols is an extremely complex task. This risk score model below is just a very preliminary step. Please provide feedback to help us improve this scoring system.
Risk Scoring Overview
The overall risk score is calculated as the maximum of the risk scores of each of the categories:
Overall Risk Score = Maximum of (Network Consensus Risk, Architectural Risk, Implementation Risk, and Operational Risk)
All of the risk scores range from 0 to 100, where 0 is a perfect score. That is, a low risk project will have a low risk score. The rationale for using the worst risk score (maximum value) across the categories is that a weakness in any part of a crosschain project render the entire project vulnerable to attack.
Network Consensus Risk Scoring
The Network Consensus Risk Score assesses the risk posed by the Network Consensus properties of the blockchains being bridged.
The equation for the Network Consensus Risk Score is:
Network Consensus Risk Score = (N001 + N002 + N003 + N004) x 10 / 17
| Question ID | Question |
|---|---|
| N001 | Finality: Does the crosschain protocol wait for transactions to be final on the source chain prior to acting upon them?
|
| N002 | Network Delays and Liveness Reliability: Has the source chain been offline (excluding planned network upgrades)?
|
| N003 | Network Delays and Liveness Congestion: Has the source chain been congested such that transactions can not be submitted and included in the blockchain?
|
| N004 | Network Safety Violations: Has the source chain had any safety violations?
|
Rationale for scoring:
- The ideal configuration is a protocol that only uses information based on transaction that are final on the source chain, and the source chain has never has any liveness or safety issues.
Architectural Risk Scoring
No aspects of Architectural Risk have been considered when determining the Architectural Risk Score.
The equation for the Architecture Risk Score is:
Architectural Risk Score = 0
Implementation Risk Scoring
Implementation Risk Scoring is based on the Implementation Risk section.
The equation for the Implementation Risk Score is:
Implementation Risk Score = Maximum of (Reducing Implementation Risk Score, Uncovering Extant Risk Score, and Responding to Materialized Risk Score)
Reducing Implementation Risk
Reducing Implementation Risk Score =
Mixing of Control and Data Plane Risk Score +
Role Based Access Control Risk Score +
Upgradable Risk Score +
Secret Storage Risk Score +
Product Development Maturity Risk Score +
Well Known Platform Risk Score +
Well Known Smart Contract Programming Language Risk Score
Mixing of Control and Data Plane Risk Score
Mixing of Control and Data Plane is described in the Mixing of Control and Data Plane section. If the Control and Data Plane are mixed, then the Mixing of Control and Data Plane Risk Score is 20, or 0 otherwise.
The Control and Data Plane are considered mixed if the answer to any of the following questions is yes:
- Can Control Plane functions be called from Data Plane functions without some additional authentication checks?
- If Control Plane messages are sent across the crosschain communications channel, are Control Plane messages signed by the same keys as Data Plane messages?
Role Based Access Control Risk Score
Role Based Access Control is described in the Role Based Access Control section.
The Role Based Access Control Risk Score depends on the permissioned control / ownership of the bridge contracts. If there are multiple contracts, and they are configured differently, then score the highest score.
- There is no permissioned ownership feature: Score 0.
- There a single EOA as the owner: Score 10.
- Role Based Access Control has been implemented: Score 0.
- There is no Role Based Access Control, but a threshold number of signers need to approve administrative actions. If number of signers is five or more score 3, otherwise score 5. Additionally, if the number of signers that need to sign is more than 50%, score 0, otherwise score 5.
Upgradable Risk Score
Upgradable Risk is decribed in the Upgradability section.
The risks associated with being able to upgrade contracts appear to be similar to the risks associated with not being able to upgrade a contract. As such, the Upgradable Risk Score is always 0.
Secret Storage Risk Score
Secret Storage is described in the Secret Storage section. The Secret Storage Risk Score is defined by the equation below.
Secret Storage Risk Score = I401
| Question ID | Question |
|---|---|
| I401 | Are any private keys, symmetric keys, passwords, or passcodes used in the project stored on disk in a plain text file? If yes, score 20, if no score 0. |
Product Development Maturity Risk Score
Product Development Maturity is described in the Product Development Maturity section. The Product Development Maturity Risk Score is defined by the equation below.
Product Development Maturity Risk Score = I501
| Question ID | Question |
|---|---|
| I501 | Does the organisation building the project have a documented SDLC approach? If yes, score 0, if no score 20. |
Well Known Platform Risk Score
The risks associated with not using Well Known Platforms is contained in the Well Known Platform section.
Scoring of this category has proven to be controvertial. At present, the risk score for this category is 0.
Well Known Smart Contract Programming Language Risk Score
The risks associated with not using a Well Known Smart Contract Programming Language is contained in the Well Known Smart Contract Programming Language section.
Scoring of this category has proven to be controvertial. At present, the risk score for this category is 0.
Uncovering Extant Risk Score
Uncovering Extant Risk Score =
Formal Verification Risk Score +
Test Risk Score +
Code Auditing Risk Score +
Open Source Code Risk Score +
Verified Code on Block Explorer Risk Score +
Documentation Risk Score +
Bug Bounty Risk Score
Formal Verification Risk Score
The risks of not formally verifying code is contained in the Formal Verification section.
The Formal Verification Risk Score is 5 if the code has not been formally verified, and 0 if it has been formally verified.
Test Risk Score
The risks associated with not thoroughly testing an application are described in the Testing section. The Test Risk Score is defined by the equation below.
Test Risk Score = I901 + I902 + I903 + I904 + I905 + I906
| Question ID | Question |
|---|---|
| I901 | Is there a test plan documenting which tests scenarios should be tested? If yes, score 0, if no score 10. |
| I902 | Are contracts unit tested? If yes, score 0, if no score 10. |
| I903 | Are their system tests that check the operation of the entire system? If yes, score 0, if no score 10. |
| I904 | Is their a continuous integration system that executes unit and system tests automatically on commit to a source code respository? If yes, score 0, if no score 10. |
| I905 | Are any of the tests executed by the continuous integration system unreliable (that is, some times unexpectedly fail)? If yes, score 5, if no score 0. |
| I906 | Can a product release occur if any test is failing? If yes, score 10, if no score 0. |
Code Auditing Risk Score
Code auditing is described in detail in the Code Auditing section. The equation for the Code Auditing Risk Score is:
Code Audit Risk Score = I1011 + I1012 + I1014 + I1015 + I1013
| Question ID | Question |
|---|---|
| I1011 | How many times has the on-chain components of the system been audited? If none, score 50, and score 0 for questions I1012, I1013, and I1014. If once score 10, if twice or more score 0. |
| I1012 | Has any version of the on-chain components of the system been audited by more than one audit organisation? If yes, score 0. If no, score 5. |
| I1013 | For the most recent audit, were all critical issues addressed? If yes, score 0. If no, score 10. |
| I1014 | Has the deployed version of the on-chain components of the system been audited? If yes, score 0. If no, score 20. |
| I1015 | Has any version of the off-chain components of the system been audited? If yes, score 0. If no, score 20. If the system contains no off-chain components, then score 0. |
Rationale for scoring:
- Unaddressed critical issues could indicate vulnerabilites.
- Having an audit, addressing issues, and then deploying the revised code without re-auditing can result in vulnerabilities.
- The quality of audit firms is not factored in the score as this is very subjective.
Open Source Code Risk Score
The risks of not using open source code for a project are discussed in the Open Source Code section. The equation for the Open Source Code Risk Score is:
Open Source Code Risk Score = I1101 + I1102
| Question ID | Question |
|---|---|
| I1101 | Is the source code of the project in a public code respository? If yes, score 0, otherwise score 10. |
| I1102 | Is it possible to develop code and deploy the project from a private repository? That is, can vulnerability fixes be resolved without publishing on the public code repsotory? If yes, score 0, otherwise score 20. |
Verified Code on Block Explorer Risk Score
The risks associated with not publishing code to blockchain explorers is covered in the Verified Code on Block Explorer section. The equation for the Verified Code on Blockchain Explorer Risk Score is:
Verified Code on Block Explorer Risk Score = I1201
| Question ID | Question |
|---|---|
| I1201 | Do all deployed contracts have verified code uploaded to block explorers? If yes, score 0, otherwise score 50. |
Documentation Risk Score
The risks associated with not documenting a project well are discussed in the Documentation section. The equation for the Documentation Risk Score is:
Documentation Risk Score = I1301 + I1302 + I1303 + I1304 + I1305 + I1306
| Question ID | Question |
|---|---|
| I1301 | Does the project have architectural documentation? If yes, score 0, otherwise score 5. |
| I1302 | Does the project have a documented threat model? If yes, score 0, otherwise score 5. |
| I1303 | Does the project have sequence diagrams for all major data flows? If yes, score 0, otherwise score 5. |
| I1304 | Are there code comments in smart contract code? If yes, score 0, otherwise score 20. |
| I1305 | Are there code comments in the off-chain code? If yes, score 0, otherwise score 5. |
| I1306 | Are there code comments in the test code? If yes, score 0, otherwise score 5. |
Bug Bounty Risk Score
Bug Bounties are described in detail in the Bug Bounty section. The equation for the Bug Bounty Risk Score is:
Bug Bounty Risk Score = I1401
| Question ID | Question |
|---|---|
| I1401 | Is there a bug bounty for the project? How big is the bug bounty?
|
Responding to Materialized Risk Score
Responding to Materialized Risk Score =
Ability to Pause Risk Score +
Ability to Ban Addresses Risk Score
Ability to Pause Risk Score
The ability to pause a project is described in the Ability to Pause Project section. The equation for the Ability to Pause Risk Score is:
Ability to Pause Score = I1501
| Question ID | Question |
|---|---|
| I1501 | Can all data plane code paths be paused? If yes score 0, otherwise score 20. |
Ability to Ban Addresses Risk Score
The ability to ban specific addresses is described in the Ability to Ban Addresses section. This capability is in contrast to the censorship resistance property that is of utmost importance to many protocols. So as to not downrank protocols that strive to provide censorship resistance over protocols that provide the ability to ban addresses, the ability to ban addresses is not scored. As such, the Ability to Ban Addresses Risk Score is always 0.
Operational Risk Scoring
The following aspects of Operational Risk are currently not taken into account when assessing the Operational Risk Score: Decentralization of Operations, Diversity of Codebase, and Off-chain Security.
The equation for the Operational Risk Score is:
Operational Risk Score = (Operational Security Score + Operational Ability to Pause Score + Vulnerability Response Planning Score) x 100 / 29
Operational Security Score
Operational Security is defined by the ability to meet the properties defined in the Operational Security section. The equation for the Operational Security Score is:
Operational Security Score = O001 + O002
The Operational Security Score ranges from 0 to 2.
| Question ID | Question |
|---|---|
| O001 | Has sensitive information as defined in the Operational Security section been identified? Has this information been documented? If yes, score 0. If no, score 1. |
| O002 | Is a complete Operational Security process, as described in the Operational Security section implemented? If yes, score 0. If no, score 1. |
Operational Ability to Pause Score
The Operational Ability to Pause is defined by the ability to meet the properties defined in the Operational Ability to Pause section. The equation for the Operational Ability to Pause Score is:
Operational Ability to Pause Score = O101 + O103 + O104 + O105
The Operational Ability to Pause Score ranges from 0 to 27.
| Question ID | Question |
|---|---|
| O101 | Type of pausing access control:
|
| O102 | Multisig Wallets: Is the threshold of the multisig wallet greater than one? Are all signers independent? If either answers is no, the score 5, otherwise score 0. |
| O103 | Automated Pausing: Does the project operate services that monitor the health of the protocol?
|
| O104 | Geographic separation of pausers: Are people who can pause the project spread across time zones?
|
| O105 | Shared keys: Are any accounts / private keys used for pausing the protocol shared with multiple people. If yes, score 10. If no, score 0. |
Rationale for scoring:
- The ideal configuration is Role Based Access Control, with active monitoring and a geographically dispersed support team. For decentralized projects, in addition to the automated monitoring system, pausing could be controlled a multisig wallet behind one or more of the accounts of the Role Based Access Control.
- Shared keys indicates poor security management. This indicates that there may be other poor practices in the system that are not known.
Vulnerability Response Planning Score
Vulnerability Response Planning is defined by the ability to meet the properties defined in the Vulnerability Response Planning section. The equation for the Vulnerability Response Planning Score is:
Vulnerability Response Planning Score = O501 + O502 + O503 + O504
The Vulnerability Response Planning Score ranges from 0 to 10.
| Question ID | Question |
|---|---|
| O501 | Is there a defined Vulnerability Response Virtual Team? If yes, score 0, otherwise score 1. |
| O502 | Is there a vulnerability reporting mechanism that allows issues to be reported without making them public? If yes, score 0, otherwise score 3. |
| O503 | Is there a defined triage process for vulnerabilities? If yes, score 0, otherwise score 1. |
| O504 | Can vulnerabilities fixes be deployed without needing to put code into a public repository? If yes, score 0, otherwise score 4. |
| O505 | Are Root Cause Analyses published? If yes, score 0, otherwise score 1. |
Rationale for scoring:
- It is important that vulnerabilities can be reported, and then vulnerability fixes worked on, and then deployed without having to make them public.
Created: October 13, 2023