At Dune we’re building the world’s first community data platform. A big part of that is Spellbook, our community data pipeline.
Spellbook allows web3 analysts all across the industry to work together on data assets.
This saves countless hours of intensive work, and eliminates the huge duplication of effort that would come with each analyst or team working individually.
Because some core data resources are commonly needed by power users to do their best work, Spellbook is key for empowering and enabling them. Working in public creates traction for solving the biggest underlying data challenges.
But we're ngl, running a collaborative data pipeline has its ups and downs.
Deploying and maintaining pipelines on a growing code base of community contributors can be difficult. We can't always keep up with the influx of contributions. Onboarding new contributors is not streamlined and power users need better spaces to discuss upcoming work.
But ultimately, the collaborative approach is the right one for building on web3 data, and it’s one that we intend to continue iterating on and improving.
There are a few aspects that make collaboration especially appealing in the web3 space.
In Web2, an analyst might only specialize in a single type of data like marketing or sales. Web3 data analysts on the other hand investigate DEXes, NFTs, DAOs, and many other sectors - all of which require specialized knowledge.
Domain switching is common for our power users. To make things trickier still, even schemas in the same domain vary substantially from competing projects!
That means access to analysts working across sectors - and the shared data assets they create - are vital to the success of all.
Without them, analysts would be overwhelmed preparing and consolidating data assets before they could even begin their real work.
E-commerce company analysts don't wake up and suddenly need to care about ride-sharing data.
In our world though, projects and sectors can blow up very quickly.
For example, it wasn't possible for us to know that Blur trades were going to suddenly become crucially important all of a sudden.
Because we can't anticipate what projects or sectors are going to gain traction, we need agile tools that let people proactively model what they care about as the action unfolds.
We can’t rely on stale quarterly data roadmaps.
This isn't unique to web3, but the scale of our data amplifies this. When developers don't consider data properly, they can easily introduce issues that break downstream pipelines.
Handling that within a startup is hard, handling that with data from thousands of startups is harder.
That's why collaborating with other analytics experts instead of going it alone is so beneficial.
We know that we can’t scale our community data pipeline while acting as gatekeepers reviewing and merging every pull request. We also know that our contributors need better tools for coordinating work.
Over the next few weeks, we’re thinking hard about these problems and finalizing our roadmap for spellbook.
There are several directions we believe will be worth following:
Are you excited to learn more, join the discussion, or contribute to Spellbook?
We would love to hear from you! Please head over to #spellbook on our Discord and drop us a line, our team will be happy to guide you to the right resources to get you started.
Getting set up to cast your first Spell isn’t hard - provided you know the basics of DBT & SQL. You can follow this guide to get started.
Also, writing code isn’t the only way to contribute.
Head over to #discussions to see what problems the community is working on, chime in with your knowledge, or propose your own.
We are shifting discussions from side channels (a well known issue in Web3) to the main repository. You should start to see more activity in this section over the coming days.
Thank you all for your contributions. The data must flow.
Solidus Labs demystifies the nuances of A-A wash trading in the DeFi ecosystem.
Building a startup is a walk on the wild side. Survival can’t be taken for granted.
The revolution will not be reported quarterly.
Celebrating Dune wizards. Dashboards, stories, alpha dropped in your inbox weekly.