Please describe your proposed solution.
We will offer on demand all the on chain data in a relational database that can be used by any client. In this way the pain of querying on-chain data, combining realtime and historical data will be mitigated.
We are developing and maintaining a high-availability infrastructure running Db-Sync to minimise downtime. Cardano projects can be hooked up to our Db-Sync Enterprise solution in short time.
Db-sync is the main system for accessing Cardano on-chain data. Unfortunately the cost of running it is high, setting it up is complex and it is not very stable: having to fully re-sync takes days.
Please describe your proposed solution.
We will offer on demand all the on chain data in a relational database that can be used by any client. In this way the pain of querying on-chain data, combining realtime and historical data will be mitigated.
Please define the positive impact your project will have on the wider Cardano community.
We will offer to Cardano’s ecosystem a low barrier entry for querying realtime and historical on-chain data. It will be “Db-Sync as a Service”, pay as you go, and scalable to the user’s project needs.
What is your capability to deliver your project with high levels of trust and accountability? How do you intend to validate if your approach is feasible?
We have more than two years experience with our own developed Db-Sync Enterprise protocol which minimises downtime of a Db-Sync pipeline.
The project team covers all aspects required to achieve the stated goals.
What are the key milestones you need to achieve in order to complete your project successfully?
M1: setup redundant and high-availability infrastructure for the project (2 months)
This process entails setting up the servers to run the Cardano node, db-sync, and a Postgres database. The setup includes configuring two fully functional db-sync pipelines, including the mentioned components. The criteria for acceptance will focus on ensuring these pipelines run smoothly, testing failover capabilities.
>M2: create monitoring and dashboard view (2 months)
This milestone's outputs will be the monitoring of the whole system. It will involve monitoring and alerting at different levels: underlying servers, running systems, database usage.
Server monitoring:
We will monitor CPU, I/O, disk and memory for every server, setting up relevant alerts for unexpected usage levels,
Running systems:
We will monitor process health and uptime
Database usage:
We will monitor typical database metrics (replica lag, queries) as well as db-sync related metrics (eg. block height lag).
The acceptance criteria will be having 3 dashboards where all the server/process instances will be monitored and test the successful delivery of alerts for all the relevant metrics
>M3: automatise user signup (2 months)
This milestone involves setting up a web App, where each user will be able to signup and select what type/how many database instances would like to use. After the successful signup, the relevant resources will be created automatically and the connection details will be shared with the user.
The user will have the option to tear down the spawned instances and pay only for the time they were used.
Acceptance criteria: Perform a successful user signup and use the connection details to connect to the read replica and perform a query successfully. Perform a successful tear down. Validate pay-as-you-go charging.
Who is in the project team and what are their roles?
Alexander Diemand (cardanobigquery - <https://www.linkedin.com/in/axeld/>): architecture & design, project management, communication, documentation
Thomas Kaliakos (thomaska - <https://www.linkedin.com/in/tkaliakos/>): data engineering, data quality responsibility, documentation
Bitseat Tadesse (bitseatt - <https://www.linkedin.com/in/bitseat/>): data science, social networks, documentation
Please provide a cost breakdown of the proposed work and resources.
B1: 20 PD architecture & design, project management, communication
B2: 20 PD system engineering (Devops) for pipeline setup
B3: 20 PD software engineering
B4: 20 PD data science: dashboard development
B5: hardware costs: est. $1000 per month (x6 months)
(PD = person day; 8 hrs/day; 1 hr = $90)
Total Budget: $63600
How does the cost of the project represent value for money for the Cardano ecosystem?
By sharing our redundant high-availability Db-Sync pipeline we can provide users with a sophisticated and reliable Db-Sync connection for a fair price.
We believe this offering enables a number of Cardano projects as it lowers entry barriers.