The first of a series on The Evolution of Data Science in Web3, exploring the convergence between data science and on-chain data: With a million transactions a day executing on the Ethereum blockchain, and millions more on Layer 2s, an enormous amount of data has accumulated. This parallels the exponential growth of data from social media and online commerce during the first two decades of the 21st century and offers enormous opportunities for Web3 protocols.

From Heavenly Data Hoarding to Spectral’s Decentralized Machine Learning

Hven, a tiny isle located midway between Sweden and Denmark was once home to the world’s most valuable data hoard: data recording the transit of stars and planets that the 16th Century astronomer Tycho Brahe observed peering at the night sky. He jealously guarded this hoard, releasing tiny dribbles of information during his lifetime. He left the island near the end of his life, and moved to Prague, where, after Brahe’s death, Johann Kepler finally got unfettered access to decades’ worth of data following the planet Mars allowing him to notice eccentricities and postulate new rules for planetary motion, which helped set the foundation for Galileo’s evidence for a heliocentric solar system, and, according to Steve Brunson, a professor at the University of Washington, inspired Newton’s Principia Mathematica. Imagine what could have been realized had Brahe given everyone else access during his lifetime.

Data science is all about finding weird wobbles and patterns, and most of it takes enormous amounts of carefully organized information to be useful. For most of the thirteen years that public blockchains have been adding to their ledgers, useful insight has been confoundingly difficult to generate, despite the appearance of complete transparency; the sheer complexity of blockchain data, and the technical expense of maneuvering through the different permutations of smart contracts, differing blockchains and across millions of pseudo-anonymous wallets has made analysis extremely difficult and prevented much of the analysis that powered web 2.0 leviathans like Alphabet and Meta.  

At Spectral we believe we’re at the dawn of a new era. Part of a group of analysts, data scientists, platforms, and decentralized applications who are using brute force, artificial intelligence, and advances in data science to build a proverbial bridge to Hven and unlock new insights and applications that will help deliver on the promises of decentralization and data science that have yet to be realized. Web3 is about more than cryptocurrency and owning digital assets—it’s also about stewardship, learning how to use decentralized tooling to make Artificial Intelligence as accessible and equitable as possible.

Blockchains Are A Difficult Data Set

One of the challenges we faced in creating an on-chain creditworthiness assessment score was finding a way to quickly and efficiently process blockchain data. By its nature, blockchain data is hard to process. Grinding through five