Please describe your proposed solution.
Context
- Having node diversity (more than one implementation of the Cardano node) will make the whole ecosystem more resilient.
- A full node is a major endeavor, we estimate it to be a multi-year project with multidisciplinary engineers on the team.
- Given the magnitude, we consider that our team wouldn’t be able to achieve the goal within the context of a single Catalyst proposal.
- A Divide-and-Conquer strategy can be used to split the project into several pieces, each one achievable as stepping stones for a final implementation some time from now.
- The “Dolos” project provides a lightweight implementation of a node which is focused solely on basic data tasks (keeping track of the chain, accessing UTxO state, serving data to client applications).
Scope of the Proposal
- Slowly build on top of “Dolos” new features, each one bringing us one step closer to a full implementation.
- Maintain a flexible architecture that allows different flavors of the end application so that new features don’t compromise the lightweight characteristics of Dolos.
- As the next big feature, we plan on adding the business logic to perform “Phase-1 Validations” of the block transactions, providing as benefit a higher level of self-accountability of the chain correctness.
Technical Implementation
- Roughly speaking, Phase-1 validation is defined as all the checks required for a transaction to be added to the ledger excluding the execution of Plutus scripts. Nodes are not compensated for this work, and transactions that fail this validation are rejected without being included into a block and without paying any fee or collateral.
- All the validation rules are specified and implemented in the Cardano Ledger. The specification is splitted into several documents, one for each era, describing incrementally the modifications and additions that each era introduced. For each era there is also as part of the specification a CDDL file that precisely defines the CBOR schema used for blocks and transactions. For any aspect of the rules not specified in the documentation, the implementation of the Cardano Ledger in Haskell serves as a reference.
- To obtain a comprehensive and precise definition of the Phase-1 validation rules, it is necessary to perform a complete reverse engineering work that requires studying the specification documents for all the eras (Byron, Shelley, Mary, Alonzo and Babbage) and the Haskell source code of the ledger implementation.
- The big set of Phase-1 rules can be divided into groups according to the aspects they refer to, and to the contextual information required to check them. We arbitrarily choose to organize the rules into the following categories:
- Basic transaction structure: CDDL compliance, integrity hashes, size limits, etc.
- Basic ledger rules: input UTxOs existence, current slot in validity interval, etc.
- Fees and collateral: correct amounts according to protocol parameters.
- Output value: input/output balance, collateral balance, min lovelace per output, etc.
- Witnesses: required signatures, scripts, redeemers and data.
- Phase-1 scripts: native script execution.
- Staking: credentials and withdrawals, etc.
- Other: protocol parameter update proposals, bootstrap witnesses, etc.
- For this proposal, we plan to implement the validation rules in the first five categories (a, b, c, d and e). We understand that this set of rules accounts for an important part of the current ledger and that, given the budget constraints, it is a reasonable increment in the path to a fully functional Rust node.
- With this implementation, we will be able to synchronize all the public networks (mainnet, preprod and preview) with an important coverage of the ledger rules. This will open the road to further iterations where the remaining aspects of phase-1 validation can be completed and phase-2 validation can be fully implemented.
How does your proposed solution address the challenge and what benefits will this bring to the Cardano ecosystem?
As a general concern and an indirect, long-term effect of this proposal, node diversity provides several benefits to the ecosystem:
- Resilience against vulnerability and bugs. In a single-implementation ecosystem, any zero-day vulnerability or severe bug could be “catastrophic” to the whole network.
- Resilience against organizational issues. In an ecosystem where the development of the node is handled by a single team, any organizational issues (developer rotation, shifts in priority, budget limitation, etc) can have drastic consequences on the evolution of the protocol.
- Foster different and novel approaches to existing problems. Independent teams or different tech-stacks will yield different ways of solving the same problems, potentially encountering better ways to solve already implemented concerns.
There’s also several new problems that arise from node diversity, like potential incompatibilities between implementations. We won’t discuss them as part of the proposal, but these are certainly issues that need to be addressed once a complete, full implementation of a second node is achieved.
On a more direct relationship, this proposal has short-term benefits too:
- Improve the level of self-accountability that Dolos has over the data that it receives from upstream nodes. Currently, Dolos needs to “trust” the upstream node providing the data. Phase-1 validations
- Generate detailed documentation (potentially a spec) of how Phase-1 validations works, which could be used by other teams in different node implementations.
- Abstract Phase-1 validations into a Rust library that can be embedded into other Rust-based tooling, allowing Phase-1 validations to be executed offline, without the need for a node (eg: Aiken, Lucid).
- Create a testbench for black-box testing of a ledger implementation in a way that it can be used for any current and future implementations.
How do you intend to measure the success of your project?
We consider the following dimensions for measuring the success of the project:
- Activity in the open-source Github repository through metrics such as, but not limited to: # of issues, clones, external contributors, stars, visitors, etc.
- Number of downloads of binary releases.
- Web traffic on documentation site.
- Number of external repositories that include this project as a dependency.
Please describe your plans to share the outputs and results of your project?
Being an open-source project, the outputs will be available to any developer in the ecosystem at every step of the development process:
- Latest version of the source-code will be available in the Github repository.
- Source code changes will be applied through a pull-request process.
- Alpha and Beta versions will be released at every milestone.
Upon reaching the end of the development process, we’ll provide:
- A LTS release version of the engine available to download in many formats (binary, Docker image, etc)
- A CLI (command line interface) binary to serve as entry point for developers
- A documentation website with instructions for usage and deployment
- An executable testing framework to run Phase-1 validation unit-test on the provided code.
- A Rust library that can be used to evaluate Phase-1 validations over arbitrary transaction data.
- A tutorial video showing a walkthrough of how to run Dolos in different environments and configurations.