Please describe your proposed solution
<u>HIGH LEVEL INTRODUCTION</u>
We propose to develop a fully automated, self-service, scalable pull-based oracle solution that enables Cardano builders to integrate CNT price pair data into their DApps within minutes. This solution leverages our existing Charli3 Dendrite DEX aggregator and enhances it with a user-friendly interface and robust backend architecture.
User Story
As an end-user,
- I want to be able to go to a front-end landing page on Charli3.io and view a catalogue of available data feeds. We will support almost every possible permutation of CNTs available on the major Cardano DEXs including pairs that aren't listed like (WMT/SNEK).
- I want to have clear and transparent pricing information and documentation on the architecture. We are in the process of open-sourcing many parts of our solution and pricing will be transparently displayed for users.
- I want to select the price pair(s) to integrate into my protocol and only pay when I want the data. In under a minute, I select the kind of feed I want and I will instantly receive a specific configuration file and steps to download/use an Open Sourced SDK.
- After downloading the SDK and following simple steps, I can use the configuration file to immediately start consuming a price feed. There will be no need to talk with Charli3, sign any contracts, nor wait for a network / data feed to launch. (!!!!)
- Use Charli3 price feed only when I want and pay only when I use it with clear transparent pricing given at the start.
- I want to access the same front-end to add new pairs and modify existing pairs without interacting with the Charli3 team and can easily repeat the previous steps to do so.
Key Components of the solution:
- Self-Service Backend Architecture:
- Refactoring of current pull-based Oracle architecture to handle on-demand deployment requests for new price pairs
- Integration and optimization of Charli3 Dendrite for efficient single-pair and multi-pair data retrieval
- Implementation of a redeemer-based approach to significantly reduce on-chain costs
- Development of a scalable node network capable of handling hundreds of requests per minute
- User-Friendly Frontend Interface:
- Web-based catalog of all available CNT price pairs
- Intuitive selection process for users to choose desired price pairs
- Automated documentation generation for integration steps
- Transparent pricing and usage statistics
- Open-Source SDK:
- Development of a comprehensive SDK for easy integration into DApps
- Support for multiple Cardano client frameworks (e.g., lucid-evolution, blaze, mesh-js, ctl)
- Extensive documentation and usage examples
- Scaling and Optimization:
- Implementation of metadata-based UTXO distinction for efficient multi-pair handling
- Optimization of node software to process multiple simultaneous requests
- Development of load balancing and failover mechanisms for high availability
Technical Innovation:
Our solution introduces several key innovations.
- On-Demand Price Pair Creation: Unlike traditional oracle services, our system can dynamically create and serve new price pairs as requested, without manual intervention.
- Efficient Multi-Pair Handling: By optimizing our backend to handle multiple price pairs simultaneously, we significantly reduce infrastructure costs and improve response times.
- Redeemer-Based Cost Reduction: Our novel approach using redeemers for data requests will substantially lower on-chain costs for DApp developers.
- Scalable Node Network: Our enhanced node software will be capable of handling hundreds of requests per minute, ensuring high performance even under peak load.
Foundations for the project:
Project 1100090: Charli3 Pull based oracle functionality
- Enables users to request data and a network puts it on-chain
- Limitations: Charli3 needs to set up a network for every price pair, Projects need to contract with Charli3 and agree on pricing and SLA terms, teams need to work with Charli3 to modify data feeds, it is not set up to scale/handle 100s of requests at once
Project 1100093: Charli3 Dendrite - Open source Pythonic Dex Aggregator
- Allows users to use the solution to extract raw data on liquidity pools and trading activity on most Cardano DEXs
- Limitations: Charli3 Dendrite is a powerful tool, but does not come configured out of the box for this use case. As a data sourcing tool, it is not optimized for one node network to source all CNT price pairs. The team will need to restructure a network to use the tooling effectively for the intended purposes of this project.
<u>SOLUTION IMPLEMENTATION PLAN</u>
1 Backend Development (18 weeks):
- Rebuild pull-based oracle architecture
- Building the Network Architecture:
- Establish a mechanism for identifying one price pair request from others and establishing a network ready state for supplying that response to the user
- Distinguish which UTXO belongs to which feed via metadata, asset minting, or another suitable solution
- Optimise current architecture to a redeemer-based approach to significantly lower consumer costs
- Integrate and optimize Charli3 Dendrite
- Implement redeemer-based approach
- Develop scalable node network
- Restructuring the Node Back-end architecture:
- Implement dynamic handling of price pair requests without pre-configuration
- Ability to handle a sufficient number of feed pair requests and respond to them in an acceptable time frame / Enhance the backend to efficiently manage concurrent requests and maintain performance under varying loads
- Develop and optimise the backend to handle high traffic and extreme price volatility, ensuring that the oracle services remain stable and reliable during peak demand.
Note:
For scaling, the challenge we have to overcome is handling multiple calls from hundreds if not thousands of consumers and to avoid possible DDOS attacks. A significant amount of the developer hours will be spent on this step.
2 Frontend Development (5 weeks):
- Utilise one of the available Cardano frameworks (Lucid-Evolution (TS), Blaze (TS), Mesh-js (TS), CTL (Purescript)) to streamline development and integration.
- Include practical documentation ensuring ease of SDK integration and deployment.
- Build out the front-end to include a catalogue of all supported CNT price pairs as a menu for users to select as input for the back-end architecture
- Implement automated configuration generation tool
- Include practical documentation ensuring ease of SDK integration and deployment
- Pricing information
- SDK documentation
- Steps for using configuration generation tool
3 SDK Development (4 weeks):
- Create core SDK functionalities
- Implement support for multiple Cardano client frameworks
- Develop comprehensive documentation and examples
4 Testing and Optimization (6 weeks):
- Conduct thorough testing of all components
- Perform load testing and optimize for high concurrency
- Refine user experience based on feedback
5 Deployment and Launch (2 weeks):
- Deploy backend infrastructure
- Launch public-facing frontend
- Release open-source SDK
OUT OF SCOPE FUTURE ENHANCEMENTS
- Non CNT price pairs. We will only provide CNT price pairs supported by Charli3 Dendrite. This should encompass almost any price pair available on a major Cardano DEX and permutations of those feeds. We will provide non-cnt price pairs in this service as a future enhancement (e.g. btc/usd)
OTHER INFORMATION:
Pull based oracle functionality
Project 1100090 Information
1100090 Statement of Milestones: <https://milestones.projectcatalyst.io/projects/1100090>
1100090 Proposal: <https://projectcatalyst.io/funds/11/cardano-use-cases-solution/charli3-pull-oracle-real-time-price-feeds-on-request-on-demand-validation-service-mvp>
In project 1100090, Charli3 and the Metalamp team successfully delivered a fully functioning MVP that you view here:
Demo video: <https://drive.google.com/file/d/1trqOYkJ5BL6NJQnGPo1OCRigboIU0T8I/view?usp=sharing>
Contract address (Odv-oracle-contract): <https://cexplorer.io/address/addr1w93vmc4m23cc6j507lhjf2duqxpjp5mdfey3fs767etxxjcyfmfv3/tx#data>
Github Repo to interact with the Dummy use-case:
<https://github.com/Charli3-Official/swap-demo-contract>