Trust Score 2.0: CoinGecko Updates Trust Score to Improve Exchange Transparency

In May 2019, we introduced Trust Score on CoinGecko to combat the alarming trend of exchanges reporting fake trading volume. Unlike in regulated financial markets where trading volume is a good representation of liquidity, in the cryptocurrency industry, trading volume is no longer a good indicator of liquidity. This is because many exchanges are known to fabricate their trading volume to gain visibility among users.

With so much noise in the industry data, the search for a reliable exchange can be a daunting task without the right tools in place. As such, here at CoinGecko we have made it our mission to provide traders and investors with insights that will help them choose exchanges with the best liquidity.

Our first implementation of Trust Score revolves around Liquidity measurements with the following analyses:

• Normalized Trading Volume by Web Traffic – Trading volume for each exchange is normalized using SimilarWeb traffic estimates
• Order Book Analysis – The bid/ask spread and ±2% order book depth is computed to measure liquidity for any given trading pair.

Trust Score has turned out to be very effective at highlighting the real liquidity for each exchange and has made it possible, for the first time, to easily filter out exchanges and trading pairs with suspicious volumes. 

Trust Score 2.0

When we first developed Trust Score, we wanted to focus on liquidity metrics to provide more insights into exchanges’ real liquidity. As we continue to iterate and improve on our methodologies, we have found other measures that define a good exchange from a user’s perspective. We have bucketed them into the following categories: 

1. Liquidity
2. API Technical Coverage
3. Scale of Operations
4. Cryptocurrency Reserves 
5. Regulatory Compliance

We have upgraded Trust Score to version 2.0 to incorporate all the above categories as there is no way to easily synthesize all the above into something that can be easily understood at one glance. Exchanges are now graded on a scale of 1 to 10, with 10 being the best exchange and 1 being the worst exchange.

Trust Score 2.0 with the new exchange grades in action 

Our new grading system takes into account multiple metrics in the liquidity, API technical coverage, and scale of operations categories to make for a more robust ranking that is less prone to manipulation. With this update, CoinGecko’s exchange rankings will be much more independent of an exchange’s reported volume.

The CoinGecko Trust Score 2.0 grading breakdown is as follows:

Note: Cryptocurrency Reserves and Regulatory Compliance categories are not included in the overall Trust Score 2.0 calculations for now but are considered candidates for inclusion in future Trust Score algorithm updates. Information about these two categories is displayed on CoinGecko.

A) Liquidity (50%)

1. Normalized-Reported Volume Ratio (NRR)
   Using the Normalized volume that was previously calculated in our Trust Score algorithm, we now calculate a ratio to determine the degree of difference between normalized and reported volume.
   
   A high NRR indicates that the reported volume is likely true, while a low NRR indicates that an exchange has a high likelihood of reporting questionable volume, all things being equal. The maximum value of this ratio is 1.
2. Average Bid-Ask Spread
   In each exchange, we look into the bid/ask spread and take the average bid/ask spread across all trading pairs that have been traded within the last 1 hour. A liquid exchange will have a very low average bid/ask spread percentage as it indicates that all markets on the exchange are relatively liquid.
3. Active Trading Pair Ratio (ATR)
   Active Trading Pair Ratio (ATR) refers to the ratio of actively traded pairs in the last hour against actively traded pairs in the last 24 hours. This measures trading activity for each exchange, which ties back to the overall liquidity on an exchange as a liquid exchange will likely have a very high ATR.
   
   
   Trading pairs without any active trades in the last 24 hours are excluded as it is assumed that such trading pairs are longer active.
4. Trading Pair Trust Score
   At present, we are measuring the Trust Score for each trading pair in an exchange. This metric looks at the various liquidity metrics for assigning a Green, Yellow, or Red Trust Score.To grade the exchange, we will now measure the percentage of trading pairs in Green, Yellow, and Red for each exchange. A liquid exchange is expected to have a high percentage of Green trading pairs, while an illiquid exchange will have a higher percentage of Yellow and Red trading pairs.

B) Exchange Scale of Operations (30%)

1. Normalized Volume Percentile
   Exchanges are sorted from the highest to lowest normalized volume. Then points are given based on an exchange’s normalized trading volume percentile relative to all other exchanges.
2. Normalized Depth Percentile
   In addition to normalized trading volume, the sum of each exchange’s ±2% order book depth in USD is used as a means of determining an exchange’s scale. The order book depth is normalized using the NRR and points are given based on an exchange’s depth percentile relative to all other exchanges.

C) API Technical Coverage (20%)

API Grading 
A well-run exchange should have the technical competencies necessary to operate a well-run API. Exchanges that are transparent will make all the trading data available with an acceptable rate limit while exchanges that have suspicious trading volumes may want to hide some information and not make all trading data available via API.

Exchanges’ APIs are graded on the following availability:

1. Tickers Data – The snapshot data – such as last done price and 24 hours rolling volume – should be available for all pairs traded on an exchange.
2. Historical Trades Data – An exchange should be able to return historical trade data for any given pair.
3. Order Book Data – Order book data is necessary to determine bid/ask spread and order book depth.
4. Candlestick/OHLC – Having OHLC (Open/High/Low/Close) data will allow for further examination of an exchange’s historical data and trade conditions.
5. Web Socket API – WebSocket provides a stream of live data such as live trades executed and live order book data for traders and researchers to query. This is an upgrade of the REST API snapshot data.
6. API Trading – This is vital for users who wish to trade via API.
7. Public Documentation – This is for referencing and usage of said API.

D) Estimated Cryptocurrency Reserves (0%)

One of the biggest benefits of operating on the blockchain is the open ledger. From a data analyst perspective, we can conduct on-chain analysis and look at exchanges’ cryptocurrency reserves through their known addresses. 

We have partnered with Bitfury’s Crystal Blockchain to conduct on-chain analysis of exchanges’ Bitcoin, Ethereum and Bitcoin Cash reserves. 

An exchange that reports a large amount of trading volume would be expected to have a correspondingly larger reserve. The reverse would be true for a smaller exchange. If an exchange does a lot of volume yet has low reserves, this could be perceived as a sign that something is not right.

As the estimates are still a continuous work in progress, we feel that this metric is not ready for inclusion in the overall Trust Score 2.0 grading algorithm. We are working hard to ensure data accuracy for exchanges’ estimated reserves and are looking to expand the exchange coverage as well. More blockchains will be supported in the near future through our partnerships with other on-chain analytics providers.

E) Exchange Regulatory Compliance (0%)

Aside from figures that can be derived from exchanges, it is important to look into an exchange’s operating conditions to determine its viability. We have partnered with Coinfirm’s AMLT Network to analyze the regulatory compliance of exchanges. This includes the following risk categories:

1. License & Authorization
2. Sanctions
3. Senior Public Figure
4. Jurisdiction Risks
5. KYC Procedures
6. Negative News
7. AML

These are all factors that can directly affect the operations of an exchange. Thus, they should also be considered when one is looking for an exchange on which to trade. 

Similar to (D) Estimated Cryptocurrency Reserves, this is a work in progress that requires further refinement before it can be included in future Trust Score updates.

[Update @ February 2020] Improved Trust Score for Exchange Ranking

As the leading source of transparent and actionable data, we’re always working on improving our datasets to make them more robust. We’ve recently upgraded the accuracy of Trust Score for Exchange Ranking by incorporating several other SimilarWeb web traffic quality metrics to more effectively capture the value of visitors of a website. 

Our mission, as always, is to present actionable data and help our users make informed decisions as we continuously improve Trust Score!

Closing Remarks

In the unregulated space that is the crypto industry, trading volume is quickly becoming an obsolete method of evaluating exchanges. While much data is available with which to evaluate exchanges, it is often difficult to fit these data together into actionable pieces of information. 

With the introduction of Trust Score 2.0, the market can now leverage CoinGecko’s extensive dataset for a “360 degree overview” of both cryptocurrencies and exchanges. We believe that the best way for the market to grow is to be transparent and to empower cryptocurrency traders with data that helps them make better informed decisions.

We are constantly looking for ways to improve our exchange ranking methodology. If you have any suggestions, please get in touch with us on Twitter, Telegram, or Discord or drop us a ticket at support.coingecko.com