not approved
Wolfram Cardano Exchange Data Infrastructure
Current Project Status
Unfunded
Amount
Received
₳0
Amount
Requested
₳824,400
Percentage
Received
0.00%
Solution

Our open-source solution will allow developers to deploy their own data collectors to store trading pair data from CEXs and Cardano based DEXs so they can build applications on top.

Problem

The task of collecting high-resolution financial data and making it useful from a wide range of centralized crypto exchanges (CEXs) and decentralized crypto exchanges (DEXs) is a complex undertaking.

Feasibility
Value for money
Impact / Alignment

Wolfram Blockchain Labs

3 members

Wolfram Cardano Exchange Data Infrastructure

Please describe your proposed solution.

Wolfram Blockchain Labs’ (WBL) mission is to advance the science of blockchains, making blockchain data computational and accessible. The task of collecting high-resolution financial data from a wide range of centralized crypto exchanges (CEXs) and decentralized crypto exchanges (DEXs) AND making it useful to developers is a complex undertaking.

Wolfram Exchange Data Infrastructure

Wolfram Exchange Data Infrastructure: Video Presentation

Real-time data aggregation services are critical components of financial applications. Providing an open-source component should advance the quantity of financial applications in the Cardano ecosystem by making it easier to start development.

Wolfram Blockchain Labs (WBL) is taking two simultaneous tactics to provide this data—collecting vertically through on-chain decentralized exchanges (DEXs) and horizontally across chains and centralized exchanges (CEXs).

Our open-source software will allow developers to deploy their own data collectors to store pair pricing data from the exchange sources with customizable intervals so they can build applications on top. Potential applications might range from visual dashboards to on-chain price feeds.

Horizontal data collection from CEXs

Through the CCXT library, we get quick access to market data for storage. CCXT is an open-source data and trading library that supports over 100 exchanges.

We have set up one collector per exchange that pulls the following trading pair data:

  • Order book
  • Trade
  • Ticker
  • Candlestick

Once the response is received, this data is stored in a database, making it easily retrievable by app developers. In our PoC (available in the GitHub repo attached to this application), we have integrated five exchanges and five trading pairs.

We will expand this coverage to 10+ exchanges and 100+ trading pairs. Also, we will connect the tokens and coins to ITSA.global ITINs for linking additional data.

Vertical data collection from DEXs

We will build a solution that incorporates the top native token pairs listed in Cardano’s decentralized exchanges (DEXs) to empower developers to generate new applications on top. We will also connect the relevant Cardano tokens (up to a maximum of 50) to ITSA ITINs for linking additional data.

Wolfram Cardano DEX Data Infrastructure

Additionally, we plan to incorporate pricing information obtained from the leading three Ethereum DEXs, including Uniswap.

Connecting to ITSA's "International Token Identification Number" establishes an industry standard for token identification which could be helpful for developers across blockchains, applications and interoperability protocols.

Development

To ensure the smooth and consistent deployment of our services, we have implemented a CI/CD pipeline that constructs Docker images of the updated services, allowing them to be retrieved by other developers on the target machine or cloud service.

Service

As a benefit to the Cardano community, we will provide an API with access to the data we collect. This API will operate continuously for a duration of two years, ensuring sustained availability of the service, provided the volatility of ADA is low. To maintain a reasonable and secure usage environment, appropriate rate limits will be implemented to prevent misuse that could potentially impact the overall performance and reliability of the system. By offering this open API, we aim to empower developers and stakeholders within the Cardano ecosystem with convenient and reliable access to our collected data, fostering innovation and enabling the development of diverse applications and services.

General Improvements

We will also add several general improvements to ensure that the project is more usable as a component for financial applications, such as queuing to improve data collection, historical data compression and system status monitoring. Additionally, WBL recognizes the significance of the broader cryptocurrency exchange ecosystem for pricing data. For example, significant volume and economic activity occurs on decentralized exchanges like Uniswap. With this in mind, the Wolfram Cardano Exchange Data Infrastructure project aims to incorporate DEXs across the spectrum. These DEX services represent significant data sources for obtaining a more comprehensive understanding of the broader cryptocurrency ecosystem.

Summary of Project Objectives:

  • Make significant architectural improvements to enhance the stability of data collection and enable simultaneous searching across multiple exchanges, among other features
  • Generate an archive with historical pricing data for relevant pairs
  • Enable data streaming of aggregate pricing data from Centralized Exchanges through an API
  • Incorporate Cardano DEX pairs into our current data collectors
  • Incorporate top three Ethereum DEXs into our current DEX data collectors for additional pricing data
  • Provide an open-source package that enables other developers to generate a pricing database according to their financial application needs

Overall Benefits:

  • Open-source pricing database
  • API (REST and WebSocket) of open, high, low, close, pricing, and coin/token volume data
  • Ability to adapt/fork the database
  • Expansion to the top 100 trading pairs with connections to ITSA.global ITINs for additional information
  • Expansion to incorporate Cardano DEXs and price data for the top 100 Cardano tokens
  • Expansion to incorporate top 3 Ethereum DEXs
  • Reduced friction for third-party developers to build dashboards, trading strategies, smart contracts, etc.
  • Database compression algorithm to save space and reduce cost overtime.
  • Tool to enable system monitoring of the servers collecting the data.

Our team has experience integrating various blockchain ecosystems into our technology stack and is excited to continue incorporating Cardano exchange data into the foundation of our services.

How does your proposed solution address the challenge and what benefits will this bring to the Cardano ecosystem?

The <u>Development and Infrastructure</u> challenge asks, "what research, tools or software can improve the developer ecosystem or infrastructure to make it easier to build and scale on the Cardano blockchain?"

Our proposed solution addresses this challenge through the following sub-category:

  • Data oracles, indexers & explorers
  • Tools and software that integrate real world data into Cardano or that make existing Cardano data more easy to use and accessible.

Our proposed solution makes it easier for developers to access financial exchange data (both CEXs and DEXs) with open software tools. We hope to enable enhanced data accessibility to this information and empower developers to build applications. We strive to catalyze their creativity by building on top of the software tools we make available.

We believe that achieving this goal directly answers the challenge proposed in the <u>Development and Infrastructure</u> category. This tool will enable ourselves (Wolfram Blockchain Labs) and other developers to contribute to the overall growth of the Cardano ecosystem.

How do you intend to measure the success of your project?

The success of our project will be reflected in the availability of our software to the Cardano developer community with instructions to easily spin up in their own hardware and/or cloud service. We will measure the success of our project in the following ways:

  • Coverage and Data Quality:
  • Assess the extent of coverage in terms of the number of exchanges, trading pairs, and Cardano DEXs incorporated into the data collectors.
  • Additionally, evaluate the quality and reliability of the collected data from exchanges by analyzing factors like accuracy, consistency, and timeliness.
  • We will also provide access to monitor the status of the server architecture with the intention to understand the overall effectiveness of system in action.
  • Scalability:
  • Server infrastructure, data storage, and resource utilization have been designed with scalability in mind.
  • Additionally, this architecture can accommodate an increasing number of exchanges, trading pairs, and users.
  • Usage
  • Any incremental usage from open source developers will be considered a success.

Please describe your plans to share the outputs and results of your project?

The outputs and results of our project will be shared through a dedicated repository. The repository will serve as a central hub where we will make the code, documentation, and other relevant project artifacts available to the public.

What is your capability to deliver your project with high levels of trust and accountability?

Wolfram Blockchain Labs (WBL) is a subsidiary of the 36-year-old Wolfram Research that is exclusively licensed with high-performance blockchain technologies and specifically designed to extend ecosystem tools for application development for distributed ledger technologies (DLTs). WBL often uses additional expertise in specialized areas from both Wolfram Research and Wolfram Technical Consulting teams. The latter is an organization that has completed over three hundred consulting projects in the past decade.

Our capability to deliver the project with high levels of trust and accountability is grounded in several key factors:

  • Track Record: Wolfram Blockchain Labs (WBL) has a proven track record in successfully completing substantial projects (Catalyst included), for which we have completed four projects already and have three more near completion. We've gone through the Milestone pilot in Fund 9 and the subsequent revision process, through two sets of proof-of-achievement processes and one of our projects was used as an example for good documentation (DADF - Fund 9). We understand the governance process and support the mission to improve and grow auditability of the Catalyst governance system.
  • Experienced Team: Our team comprises professionals from diverse disciplines, including blockchain technology, finance and project management. Our multidisciplinary approach brings a comprehensive and holistic perspective to the project, ensuring that all aspects are addressed with a high level of expertise.
  • All of four completed projects across Fund 8 and Fund 9 have occurred within budget.

This experience relays our ability to meet project objectives, timelines, and budgetary constraints and deliver results. We believe we have been good stewards of the Catalyst process and are well-suited to deliver the project with high levels of trust and accountability.

What are the main goals for the project and how will you validate if your approach is feasible?

We know this approach is feasible and have a proof-of-concept completed to show for it. The main goals of the project are to expand data coverage and to create a tool for others to use as well. Below we explain the feasibility of our approach:

  • Data Coverage: Expand data coverage in terms of the number of exchanges (10), trading pairs (100), Cardano DEXs (5) and several Ethereum DEXs (3) incorporated into the DEX data collectors. We will monitor the progress by tracking the percentage increase in coverage over time.

  • Open Tool: Freely accessible tool for developers to employ their own software and hardware infrastructure to collect data from both centralized and decentralized exchanges. This will be made available through our open repository.

Please provide a detailed breakdown of your project’s milestones and each of the main tasks or activities to reach the milestone plus the expected timeline for the delivery.

  1. Historical data and source expansion (1 month)
  2. Expand number of pairs and exchanges
  3. Identify publicly available sources for historical pricing data of the specified pairs
  4. Implement data fetching from these sources
  5. Testing
  6. DEX Collectors (2 months)
  7. Implement driver for Cardano node
  8. Implement DEX contract drivers
  9. Implement collectors for DEX contracts
  10. DEX Collector REST API and Testing
  11. Update REST API
  12. Testing
  13. Data compression (2 months)
  14. Data compression algorithm
  15. Data compression Testing
  16. System Status Monitoring (2 months)
  17. Data visualization
  18. Stress testing
  19. Equity Data Infrastructure (2 months)
  20. Equity data infrastructure design
  21. Equity data infrastructure implementation
  22. Equity data infrastructure testing

Please describe the deliverables, outputs and intended outcomes of each milestone.

  1. Historical data and source expansion
  2. Deliverable: Function(s) that allow historical data to be loaded in the deployed database
  3. Output: Functional component of the docker image which provides access to populate historical (CEX) financial data easily
  4. Outcome: Devs will have the ability of populating historical data for their desired use case
  5. DEX Collectors
  6. Deliverable: Software module that connects to the Cardano Blockchain to get DEX pricing data. Software module that connects to Ethereum DEXs to get pricing data.
  7. Output: Functional component of the docker image which provides access to start collecting DEX financial data.
  8. Outcome: Devs will have the ability to store native Cardano pair pricing from various exchanges as a basis for their applications.
  9. Data compression
  10. Deliverable: Platform functions that after a determined amount of time take a section of the data and replace it with a single aggregate line.
  11. Output: A function to compress data across time intervals to enable greater storage capacity.
  12. Outcome: Data compression and long term resource optimization.
  13. System Status Monitoring
  14. Deliverable: A simple tool that allows monitoring of the deployed servers.
  15. Output: Metrics for understanding system status.
  16. Outcome: The system status monitoring tool will give developers valuable information to ensure the reliability of their deployed servers.
  17. Equity Data Infrastructure
  18. Deliverable: A data collector infrastructure for traditional equity data.
  19. Output: Functional component of the docker image which provides access to start collecting traditional equity financial data.
  20. Outcome: This collector will allow developers to store traditional equity data (not including the required data licenses).

Please provide a detailed budget breakdown of the proposed work and resources.

  1. Historical data and source expansion (1 month) - ₳78,400
  2. Expand number of pairs and exchanges
  3. Identify publicly available sources for historical pricing data of the specified pairs.
  4. Implement data fetching from these sources.
  5. Testing
  6. DEX Collectors (2 months) - ₳187,500
  7. Implement driver for Cardano node
  8. Implement DEX contract driver
  9. Implement collector for DEX contracts
  10. DEX Collector REST API and Testing
  11. Update REST API
  12. Testing
  13. Data compression (2 months) - ₳187,500
  14. Data compression algorithm
  15. Data compression Testing
  16. System Status Monitoring (2 months) - ₳119,300
  17. Data visualization
  18. Stress testing
  19. Equity Data Infrastructure (2 months) - ₳255,700
  20. Equity Data Infrastructure Design
  21. Equity Data Infrastructure implementation
  22. Equity Data Infrastructure Testing

Who is in the project team and what are their roles?

Jon Woodard, CEO

Jon Woodard is the CEO at Wolfram Blockchain Labs, where Jon coordinates the decentralized projects that connect the Wolfram Technology ecosystem to different DLT ecosystems. Previously at Wolfram Research Jon worked on projects at the direction of Wolfram Research CEO Stephen Wolfram and prior to that was a member of the team who worked on the monetization strategies and execution for Wolfram|Alpha. Jon has a background in economics and computational neuroscience. He enjoys cycling in his spare time.

Johan Veerman, CTO

Johan Veerman is General Manager at Wolfram Research South America and CTO at Wolfram Blockchain Labs. Previously he has been Science Advisor at the Ministry of Foreign Affairs in Peru and Chief Scientist on two Antarctic expeditions. Johan's background is on physics and business management. He enjoys playing soccer and is a certified cave diver.

Steph Macurdy, Head of Research and Education

Steph Macurdy has a background in economics, with a focus on complex systems. He attended the Real World Risk Institute in 2019, lead by Nassim Taleb, and has been investing in the crypto asset space since 2015. He previously worked for Tesla as an energy advisor and Cambridge Associates as an investment analyst. Steph is a youth soccer coach in the Philadelphia area and is interested in permaculture.

Gabriela Guerra Galan, Project Manager

Gabriela Guerra Galan: Gabriela has 15+ years of experience leading projects. She is a certified PMP and Product Owner with bachelor's degree in Mechatronical Engineering, complemented by a master's degree in Automotive Engineering. As the co-founder of Bloinx, a startup that secured funding from the UNICEF Innovation Fund, she has demonstrated a passion for driving innovation and social impact.

Daniel Suarez, Manager Blockchain Integrations

https://www.linkedin.com/in/dsuarezm84/

Jesus Hernandez, Senior Technologist

Jesús Hernández is Senior Technologist at Wolfram Research where he helps various business units explore new opportunities and partnerships. He has been with the company for ten years and is continually learning how Wolfram technology can be applied in new areas. He has a background in theoretical atomic physics. Jesús enjoys cooking, listening to almost all forms of music, and watching his favorite sports teams inevitably lose.

Significantly, our project execution will benefit from the invaluable contribution of numerous additional skilled team members.

How does the cost of the project represent value for money for the Cardano ecosystem?

WBL wants to supply Cardano and it's users with real-time crypto financial data and price feeds. In order to bring this application to market, we needed to build the infrastructure so that we could access the complex spectrum of crypto financial data. Crypto financial data is difficult and expensive to work with. The reason it is difficult is because the true value of crypto financial data is that it needs to be as real-time as possible. If crypto financial data is not reflecting the real price in real time, then the application or entity providing the data can be arbitraged. Arbitrage is the process by which a difference in price is exploited in order to remove the discrepancy. When dealing in crypto financial markets, this can be a catastrophic error and cost applications or entities potentially billions of dollars.

For this reason, crypto financial data requires high performing hardware, efficient software and data management, and human expertise to implement a solution.

Furthermore, apart from cryptocurrency financial data, the solution has the capability to accommodate equities financial data as well. However, it's important to note that this particular price does not cover subscriptions to that data (which can run into multiple hundreds of thousands a year).

Because WBL operates within the Cardano ecosystem, our solution to this problem is to open source the tool we've built to eventually use for ourselves in the application for delivering financial data and price feeds back to the community. We've consulted many organizations around this problem and have found expertise in many areas and feel confident that we can deliver this solution. This has been a time consuming and expensive process, it will require further expertise to bring the full solution to market, and has several moving parts and teams to manage. This is why we are requesting the amount of $200,000 USD to complete and deliver the project.

close

Playlist

  • EP2: epoch_length

    Authored by: Darlington Kofa

    3m 24s
    Darlington Kofa
  • EP1: 'd' parameter

    Authored by: Darlington Kofa

    4m 3s
    Darlington Kofa
  • EP3: key_deposit

    Authored by: Darlington Kofa

    3m 48s
    Darlington Kofa
  • EP4: epoch_no

    Authored by: Darlington Kofa

    2m 16s
    Darlington Kofa
  • EP5: max_block_size

    Authored by: Darlington Kofa

    3m 14s
    Darlington Kofa
  • EP6: pool_deposit

    Authored by: Darlington Kofa

    3m 19s
    Darlington Kofa
  • EP7: max_tx_size

    Authored by: Darlington Kofa

    4m 59s
    Darlington Kofa
0:00
/
~0:00