Please describe your proposed solution.
Wolfram Blockchain Labs’ (WBL) mission is to advance the science of blockchains, making blockchain data computational and accessible. The task of collecting high-resolution financial data from a wide range of centralized crypto exchanges (CEXs) and decentralized crypto exchanges (DEXs) AND making it useful to developers is a complex undertaking.
Wolfram Exchange Data Infrastructure: Video Presentation
Real-time data aggregation services are critical components of financial applications. Providing an open-source component should advance the quantity of financial applications in the Cardano ecosystem by making it easier to start development.
Wolfram Blockchain Labs (WBL) is taking two simultaneous tactics to provide this data—collecting vertically through on-chain decentralized exchanges (DEXs) and horizontally across chains and centralized exchanges (CEXs).
Our open-source software will allow developers to deploy their own data collectors to store pair pricing data from the exchange sources with customizable intervals so they can build applications on top. Potential applications might range from visual dashboards to on-chain price feeds.
Horizontal data collection from CEXs
Through the CCXT library, we get quick access to market data for storage. CCXT is an open-source data and trading library that supports over 100 exchanges.
We have set up one collector per exchange that pulls the following trading pair data:
- Order book
- Trade
- Ticker
- Candlestick
Once the response is received, this data is stored in a database, making it easily retrievable by app developers. In our PoC (available in the GitHub repo attached to this application), we have integrated five exchanges and five trading pairs.
We will expand this coverage to 10+ exchanges and 100+ trading pairs. Also, we will connect the tokens and coins to ITSA.global ITINs for linking additional data.
Vertical data collection from DEXs
We will build a solution that incorporates the top native token pairs listed in Cardano’s decentralized exchanges (DEXs) to empower developers to generate new applications on top. We will also connect the relevant Cardano tokens (up to a maximum of 50) to ITSA ITINs for linking additional data.
Additionally, we plan to incorporate pricing information obtained from the leading three Ethereum DEXs, including Uniswap.
Connecting to ITSA's "International Token Identification Number" establishes an industry standard for token identification which could be helpful for developers across blockchains, applications and interoperability protocols.
Development
To ensure the smooth and consistent deployment of our services, we have implemented a CI/CD pipeline that constructs Docker images of the updated services, allowing them to be retrieved by other developers on the target machine or cloud service.
Service
As a benefit to the Cardano community, we will provide an API with access to the data we collect. This API will operate continuously for a duration of two years, ensuring sustained availability of the service, provided the volatility of ADA is low. To maintain a reasonable and secure usage environment, appropriate rate limits will be implemented to prevent misuse that could potentially impact the overall performance and reliability of the system. By offering this open API, we aim to empower developers and stakeholders within the Cardano ecosystem with convenient and reliable access to our collected data, fostering innovation and enabling the development of diverse applications and services.
General Improvements
We will also add several general improvements to ensure that the project is more usable as a component for financial applications, such as queuing to improve data collection, historical data compression and system status monitoring. Additionally, WBL recognizes the significance of the broader cryptocurrency exchange ecosystem for pricing data. For example, significant volume and economic activity occurs on decentralized exchanges like Uniswap. With this in mind, the Wolfram Cardano Exchange Data Infrastructure project aims to incorporate DEXs across the spectrum. These DEX services represent significant data sources for obtaining a more comprehensive understanding of the broader cryptocurrency ecosystem.
Summary of Project Objectives:
- Make significant architectural improvements to enhance the stability of data collection and enable simultaneous searching across multiple exchanges, among other features
- Generate an archive with historical pricing data for relevant pairs
- Enable data streaming of aggregate pricing data from Centralized Exchanges through an API
- Incorporate Cardano DEX pairs into our current data collectors
- Incorporate top three Ethereum DEXs into our current DEX data collectors for additional pricing data
- Provide an open-source package that enables other developers to generate a pricing database according to their financial application needs
Overall Benefits:
- Open-source pricing database
- API (REST and WebSocket) of open, high, low, close, pricing, and coin/token volume data
- Ability to adapt/fork the database
- Expansion to the top 100 trading pairs with connections to ITSA.global ITINs for additional information
- Expansion to incorporate Cardano DEXs and price data for the top 100 Cardano tokens
- Expansion to incorporate top 3 Ethereum DEXs
- Reduced friction for third-party developers to build dashboards, trading strategies, smart contracts, etc.
- Database compression algorithm to save space and reduce cost overtime.
- Tool to enable system monitoring of the servers collecting the data.
Our team has experience integrating various blockchain ecosystems into our technology stack and is excited to continue incorporating Cardano exchange data into the foundation of our services.
How does your proposed solution address the challenge and what benefits will this bring to the Cardano ecosystem?
The <u>Development and Infrastructure</u> challenge asks, "what research, tools or software can improve the developer ecosystem or infrastructure to make it easier to build and scale on the Cardano blockchain?"
Our proposed solution addresses this challenge through the following sub-category:
- Data oracles, indexers & explorers
- Tools and software that integrate real world data into Cardano or that make existing Cardano data more easy to use and accessible.
Our proposed solution makes it easier for developers to access financial exchange data (both CEXs and DEXs) with open software tools. We hope to enable enhanced data accessibility to this information and empower developers to build applications. We strive to catalyze their creativity by building on top of the software tools we make available.
We believe that achieving this goal directly answers the challenge proposed in the <u>Development and Infrastructure</u> category. This tool will enable ourselves (Wolfram Blockchain Labs) and other developers to contribute to the overall growth of the Cardano ecosystem.
How do you intend to measure the success of your project?
The success of our project will be reflected in the availability of our software to the Cardano developer community with instructions to easily spin up in their own hardware and/or cloud service. We will measure the success of our project in the following ways:
- Coverage and Data Quality:
- Assess the extent of coverage in terms of the number of exchanges, trading pairs, and Cardano DEXs incorporated into the data collectors.
- Additionally, evaluate the quality and reliability of the collected data from exchanges by analyzing factors like accuracy, consistency, and timeliness.
- We will also provide access to monitor the status of the server architecture with the intention to understand the overall effectiveness of system in action.
- Scalability:
- Server infrastructure, data storage, and resource utilization have been designed with scalability in mind.
- Additionally, this architecture can accommodate an increasing number of exchanges, trading pairs, and users.
- Usage
- Any incremental usage from open source developers will be considered a success.
Please describe your plans to share the outputs and results of your project?
The outputs and results of our project will be shared through a dedicated repository. The repository will serve as a central hub where we will make the code, documentation, and other relevant project artifacts available to the public.