We talked to Samuel Brack, CTO and co-founder of DIA Data, in December on our Twitter/X Space . This follow-up blog post captures some key points about zk for oracles and DIA’s market-leading push to innovate this market.
In the Web3 landscape, oracles play a pivotal role in enabling blockchains to interact with external data sources. These interactions can be anything from price feeds, weather data, and sports event results to random-number-generation oracles. There are diverse applications across many industries that expand opportunities for decentralized app (DApp) developers. One of the goals of oracle providers is to continuously innovate to minimize security risks from points of centralization.
If we compare how oracles work in decentralized finance (DeFi) to the way traditional finance (TradFi) operates, we see a big difference. While DeFi has come a long way in matching some of the products we see in TradFi, the oracles that are used to bring real-world information into these DeFi systems are not as advanced or versatile. This makes it harder for DeFi to achieve its full potential in terms of the scope and innovation in product design and risk management.
For example, when DeFi systems want to check how much collateral (security) is enough for a trade, they often look at the current prices of assets. But these prices may not give the full picture. They need to show how well the market can handle a large sell-off of assets. It’s like judging the water’s depth just by looking at the surface.
A better oracle would not only give the current prices but also tell us how easy or difficult it might be to sell a lot of assets at that price based on factors such as liquidity. With this insight, it’s like knowing how deep the water is before you jump in.
Another problem is that sometimes these prices can be manipulated by techniques such as flash loans. A flash loan is a type of uncollateralized loan borrowed and repaid within a single transaction block. To handle this, the oracle could also give a warning if there’s any suspicious activity happening around these prices.
These are the types of problems that DIA Data, a participant in the o1Labs Partner Program, is tackling by using only data from its actual sources to come up with price information. Additionally, DIA utilizes metadata like liquidity or volume to determine how accurate the source data is and to determine if a reliable data feed can be constructed. In general, all processing is done in a transparent way so that any observer can obtain information on how data feeds are constructed and maintained.
Bringing Together zk and Oracles
At o1Labs, we often cite DIA’s use case when introducing people to zero knowledge on an application layer, in contrast to the go-to use case, which is typically about privacy. We believe there is a huge space for innovation using zk’s properties of verifiable off-chain computation that would bring the benefits of lowered gas costs, and especially the costs of data-intensive on-chain computation.
So, why don’t we know how deep the water is before jumping in? Oracle price data involves calculations (even spot prices that are typically the result of multiple price data points from multiple exchanges, from which outliers are removed and an average calculated), and on-chain calculations are gas-expensive, and this cost has inhibited innovation.
The approach that DIA is exploring with o1js involves performing these calculations off-chain in zk circuits with verifiable computation. This makes the oracle more cost-effective while ensuring the same level of trustlessness that on-chain calculations deliver. In theory, an off-chain calculation can be performed anywhere and the correctness of the result is proven by also sending a proof to a recipient or to an agent that can perform verification of the proof for the whole market. DIA developed and tested this POC circuit with data points that each consist of price and trade volume. The data points were normalized for outliers and then turned into a volume-weighted average price.
The POC was built by Samuel himself, and after he was up to speed on o1js, its tooling, and our provided samples, he estimates that the current main build took approximately one day. He especially liked the extensive documentation and support resources when building with o1js, as well as the possibility of potentially integrating existing code. While he shared feedback on challenges such as the learning curve of understanding zero knowledge proofs and recursion and how to best translate their potential within a product’s architecture, he noted that the toolchain and developer experience of recursion in o1js made it easy.
At o1Labs, we believe that off-chain verifiable computation has the potential to address the current limitations and potential high costs associated with oracles. This innovation is poised to revolutionize the industry by enabling a wider range of oracles. However, certain design considerations are still in progress. For example, while a single calculator for a price feed results in the lowest cost, it should be recognized that introducing some redundancy and decentralization could enhance factors such as liveness, robustness, and censorship resistance. Or that the reliability of the output depends on the trustworthiness of the input data and the market’s confidence in the integrity of the calculation. Zero knowledge proofs can verify that the calculation adhered to specifications, but it’s crucial for the market to trust that the code’s specifications are correct. This trust can be established through methods such as code publication for market review, auditing, or a combination of both. Our work with DIA has brought light to these design requirements, and we’re excited to continue exploring the future of oracles and zk.
Thanks to Samuel and DIA Data for their input to this post. Stay tuned for more news on DIA’s work on next-generation oracles!