-
Notifications
You must be signed in to change notification settings - Fork 3
feat: initial implementation of SPV client in rust-dashcode #75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
77 commits
Select commit
Hold shift + click to select a range
715ffbf
feat: add chainlock to inv
PastaPastaPasta 3c69899
add chainlock / islock stuff; request chainlocks we see in inv
PastaPastaPasta 7604f94
bloom no work
PastaPastaPasta 592dcca
compact filters
PastaPastaPasta f609bf0
dash-spv crate
PastaPastaPasta baab12a
feat: implement BIP158 filter matching and comprehensive SPV monitoring
PastaPastaPasta 1c5d02e
"Add Improved Network Message Handling and Block Processing
PastaPastaPasta e3acc3c
"fix: Update Regtest network constants and genesis block"
PastaPastaPasta 24ef04e
feat: add batch header loading and reverse index to storage
PastaPastaPasta 1afdc6d
perf: optimize masternode sync header feeding by 1000x
PastaPastaPasta e0e3297
feat: add modern terminal UI with real-time status display
PastaPastaPasta 0d0b1ba
feat: integrate terminal UI with SPV client
PastaPastaPasta 97d9683
feat: add terminal UI support to CLI and improve logging
PastaPastaPasta 0f3de38
refactor: minor improvements to sync modules
PastaPastaPasta 1733c03
remove redundant chainlock storage
PastaPastaPasta fa786a7
adjust how blocks are fed to masternode engine to avoid redundant blo…
PastaPastaPasta b423c28
reduce verbose logging
PastaPastaPasta c4b2d51
adds batch of tests, some that should've been commited earlier
PastaPastaPasta 0af12d1
p2p: connect to multiple nodes, multiple threads
PastaPastaPasta 953a29a
fixup network constants
PastaPastaPasta bfff00b
fix: correct genesis_block static values for mainnet
PastaPastaPasta 167159c
feat: implement UTXO tracking and wallet functionality
PastaPastaPasta 8c1ea6f
refactor: resolve cargo check warnings
PastaPastaPasta 9facca2
feat: improve network architecture and multi-peer management
PastaPastaPasta 8f33d4b
feat: enhance sync system with robust coordination and recovery
PastaPastaPasta 7a44359
feat: implement centralized message routing and coordination
PastaPastaPasta d429ad0
test: add comprehensive sync testing and verification utilities
PastaPastaPasta 50725a3
fix: correct sync state management to prevent premature completion
PastaPastaPasta c9c2b81
fix: add proper sync state completion handling in client
PastaPastaPasta 022dc5f
debug: add comprehensive logging to header sync manager
PastaPastaPasta 24fdba2
improve: enhance network error handling for checksum failures
PastaPastaPasta dcee516
docs: add comprehensive project documentation
PastaPastaPasta bb9bcd5
fix: resolve header sync state management issues
PastaPastaPasta ba077a2
fix: resolve critical race condition in header sync timing
PastaPastaPasta 6b245b1
fix: resolve race condition by coordinating sync with monitoring loop
PastaPastaPasta c6a7b49
feat: implement interleaved header and filter header sync
PastaPastaPasta f0bd5c9
fix: ensure filter header requests are sent even when sync is active
PastaPastaPasta 2d126f5
fix: remove hardcoded 10000 height limit in filter header sync
PastaPastaPasta ec59841
improve: enhance network monitoring resilience during peer disconnect…
PastaPastaPasta 7f098fd
fix: improve storage concurrency safety in header storage
PastaPastaPasta 2d9c5a9
improve: add adaptive timeout handling for header sync
PastaPastaPasta 85fa18f
refactor: simplify filter header sync coordination logic
PastaPastaPasta b12b437
fix: auto-trigger masternode sync after header sync completion
PastaPastaPasta 2c5bfdb
improve: enhance dash-spv CLAUDE.md with debugging and implementation…
PastaPastaPasta 6c4875d
fix: resolve CFilter message processing and add comprehensive debug l…
PastaPastaPasta dd8f419
fix: resolve storage layer race condition in segmented eviction
PastaPastaPasta 4fabfee
fix: resolve UTXO serialization and balance calculation issues
PastaPastaPasta ea7fdae
fix: resolve filter header sync storage consistency issue
PastaPastaPasta adda3ba
fix: implement exclusive peer connection mode and adjust peer discove…
PastaPastaPasta f57ac13
fix: resolve WatchItem deserialization issue with earliest_height field
PastaPastaPasta c53819b
feat: add ISLock message support to network message parsing
PastaPastaPasta 64ed637
fix: enhance ProTx parsing logic for BasicBLS version and platform fi…
PastaPastaPasta 16f1bc5
refactor: complete overhaul of filter processing architecture for bet…
PastaPastaPasta 591e29b
fix: address over-reading issue in coinbase payload decoding for vers…
PastaPastaPasta 789e70b
debug: log block hash for blocks the fail deser
PastaPastaPasta 9dc9fd7
feat: implement request timeout handling and tracking for network mes…
PastaPastaPasta 78bbb28
feat: add blocks_processed counter to SpvStats with logging
PastaPastaPasta 5efdbf9
feat: add MNHF Signal transaction support in special transaction hand…
PastaPastaPasta 46a48dd
feat: implement in-memory UTXO cache with disk persistence and addres…
PastaPastaPasta cba4029
feat: implement asynchronous block processing with dedicated worker a…
PastaPastaPasta e46fee7
feat: integrate storage access for current blockchain tip height retr…
PastaPastaPasta ba8d380
feat: clean up whitespace and improve code readability in mod.rs
PastaPastaPasta 17d7d81
refactor: modularize SPV client and add wallet management functionality
PastaPastaPasta 7e2c6f8
feat: implement filter sync tracking and progress reporting
PastaPastaPasta e9f76b2
feat: enhance filter synchronization with flow control and request tr…
PastaPastaPasta a029a98
refactor: de-duplicate filter header chain verification logic
PastaPastaPasta 2051bda
feat: add CFHeader gap detection and auto-restart functionality
PastaPastaPasta 050efa7
refactor: remove redundant CLSig and ISLock structs
PastaPastaPasta 95fe46a
fix: build failure re CLSig and ISLock messages
PastaPastaPasta 99aaec3
fix: add NetworkExt import to multiple files
PastaPastaPasta bdb33b0
refactor: simplify WatchManager usage by removing instance creation
PastaPastaPasta ac91ae9
fix: change logging initialization to use try_init with error handling
PastaPastaPasta dac0a7c
feat: implement handshake timeout mechanism with message polling
PastaPastaPasta e7b1bb0
fix: add terminal size check before drawing status bar
PastaPastaPasta d0fbb6c
fix: update filter segment paths to use the correct directory
PastaPastaPasta 05de85f
fix: improve error handling for directory creation in disk module
PastaPastaPasta b1fe1c7
fix: ensure proper sync state handling during masternode synchronization
PastaPastaPasta File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,225 @@ | ||
| # CLAUDE.md | ||
|
|
||
| This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. | ||
|
|
||
| ## Project Overview | ||
|
|
||
| **dash-spv** is a Rust implementation of a Dash SPV (Simplified Payment Verification) client library built on top of the `dashcore` library. It provides a modular, async/await-based architecture for connecting to the Dash network, synchronizing blockchain data, and monitoring transactions. | ||
|
|
||
| ## Architecture | ||
|
|
||
| The project follows a layered, trait-based architecture with clear separation of concerns: | ||
|
|
||
| ### Core Modules | ||
| - **`client/`**: High-level client API (`DashSpvClient`) and configuration (`ClientConfig`) | ||
| - **`network/`**: TCP connections, handshake management, message routing, and peer management | ||
| - **`storage/`**: Storage abstraction with memory and disk backends via `StorageManager` trait | ||
| - **`sync/`**: Synchronization coordinators for headers, filters, and masternode data | ||
| - **`validation/`**: Header validation, ChainLock, and InstantLock verification | ||
| - **`wallet/`**: UTXO tracking, balance calculation, and transaction processing | ||
| - **`types.rs`**: Common data structures (`SyncProgress`, `ValidationMode`, `WatchItem`, etc.) | ||
| - **`error.rs`**: Unified error handling with domain-specific error types | ||
|
|
||
| ### Key Design Patterns | ||
| - **Trait-based abstractions**: `NetworkManager`, `StorageManager` for swappable implementations | ||
| - **Async/await throughout**: Built on tokio runtime | ||
| - **State management**: Centralized sync coordination with `SyncState` and `SyncManager` | ||
| - **Modular validation**: Configurable validation modes (None/Basic/Full) | ||
|
|
||
| ## Development Commands | ||
|
|
||
| ### Building and Running | ||
| ```bash | ||
| # Build the library | ||
| cargo build | ||
|
|
||
| # Run the SPV client binary | ||
| cargo run --bin dash-spv -- --network mainnet --data-dir ./spv-data | ||
|
|
||
| # Run with custom peer | ||
| cargo run --bin dash-spv -- --peer 192.168.1.100:9999 | ||
|
|
||
| # Run examples | ||
| cargo run --example simple_sync | ||
| cargo run --example filter_sync | ||
| ``` | ||
|
|
||
| ### Testing | ||
|
|
||
| **Unit and Integration Tests:** | ||
| ```bash | ||
| # Run all tests | ||
| cargo test | ||
|
|
||
| # Run specific test files | ||
| cargo test --test handshake_test | ||
| cargo test --test header_sync_test | ||
| cargo test --test storage_test | ||
| cargo test --test integration_real_node_test | ||
|
|
||
| # Run individual test functions | ||
| cargo test --test handshake_test test_handshake_with_mainnet_peer | ||
|
|
||
| # Run tests with output | ||
| cargo test -- --nocapture | ||
|
|
||
| # Run single test with debug output | ||
| cargo test --test handshake_test test_handshake_with_mainnet_peer -- --nocapture | ||
| ``` | ||
|
|
||
| **Integration Tests with Real Node:** | ||
| The integration tests in `tests/integration_real_node_test.rs` connect to a live Dash Core node at `127.0.0.1:9999`. These tests gracefully skip if no node is available. | ||
|
|
||
| ```bash | ||
| # Run real node integration tests | ||
| cargo test --test integration_real_node_test -- --nocapture | ||
|
|
||
| # Test specific real node functionality | ||
| cargo test --test integration_real_node_test test_real_header_sync_genesis_to_1000 -- --nocapture | ||
| ``` | ||
|
|
||
| See `run_integration_tests.md` for detailed setup instructions. | ||
|
|
||
| ### Code Quality | ||
| ```bash | ||
| # Check formatting | ||
| cargo fmt --check | ||
|
|
||
| # Run linter | ||
| cargo clippy --all-targets --all-features -- -D warnings | ||
|
|
||
| # Check all features compile | ||
| cargo check --all-features | ||
| ``` | ||
|
|
||
| ## Key Concepts | ||
|
|
||
| ### Sync Coordination | ||
| The `SyncManager` coordinates all synchronization through a state-based approach: | ||
| - Header sync via `HeaderSyncManager` | ||
| - Filter header sync via `FilterSyncManager` | ||
| - Masternode list sync via `MasternodeSyncManager` | ||
| - Centralized timeout handling and recovery | ||
|
|
||
| ### Storage Backends | ||
| Two storage implementations via the `StorageManager` trait: | ||
| - `MemoryStorageManager`: In-memory storage for testing | ||
| - `DiskStorageManager`: Persistent disk storage for production | ||
|
|
||
| ### Network Layer | ||
| TCP-based networking with proper Dash protocol implementation: | ||
| - Connection management via `TcpConnection` | ||
| - Handshake handling via `HandshakeManager` | ||
| - Message routing via `MessageHandler` | ||
| - Multi-peer support via `PeerManager` | ||
|
|
||
| ### Validation Modes | ||
| - `ValidationMode::None`: No validation (fast) | ||
| - `ValidationMode::Basic`: Basic structure and timestamp validation | ||
| - `ValidationMode::Full`: Complete PoW and chain validation | ||
|
|
||
| ### Wallet Integration | ||
| Basic wallet functionality for address monitoring: | ||
| - UTXO tracking via `Utxo` struct | ||
| - Balance calculation with confirmation states | ||
| - Transaction processing via `TransactionProcessor` | ||
|
|
||
| ## Testing Strategy | ||
|
|
||
| ### Test Organization | ||
| - **Unit tests**: In-module tests for individual components | ||
| - **Integration tests**: `tests/` directory with comprehensive test suites | ||
| - **Real network tests**: Integration with live Dash Core nodes | ||
| - **Performance tests**: Sync rate and memory usage benchmarks | ||
|
|
||
| ### Test Categories (from `tests/test_plan.md`) | ||
| 1. **Network layer**: Handshake, connection management (3/4 passing) | ||
| 2. **Storage layer**: Memory/disk operations (9/9 passing) | ||
| 3. **Header sync**: Genesis to tip synchronization (11/11 passing) | ||
| 4. **Integration**: Real node connectivity and performance (6/6 passing) | ||
|
|
||
| ### Test Data Requirements | ||
| - Dash Core node at `127.0.0.1:9999` for integration tests | ||
| - Tests gracefully handle node unavailability | ||
| - Performance benchmarks expect 50-200+ headers/second sync rates | ||
|
|
||
| ## Development Workflow | ||
|
|
||
| ### Working with Sync | ||
| The sync system uses a monitoring loop pattern: | ||
| 1. Call `sync_*()` methods to start sync processes | ||
| 2. The monitoring loop calls `handle_*_message()` for incoming data | ||
| 3. Use `check_sync_timeouts()` for timeout recovery | ||
| 4. Sync completion is tracked via `SyncState` | ||
|
|
||
| ### Adding New Features | ||
| 1. Define traits for abstractions (e.g., new storage backend) | ||
| 2. Implement concrete types following existing patterns | ||
| 3. Add comprehensive unit tests | ||
| 4. Add integration tests if network interaction is involved | ||
| 5. Update error types in `error.rs` for new failure modes | ||
|
|
||
| ### Error Handling | ||
| Use domain-specific error types: | ||
| - `NetworkError`: Connection and protocol issues | ||
| - `StorageError`: Data persistence problems | ||
| - `SyncError`: Synchronization failures | ||
| - `ValidationError`: Header and transaction validation issues | ||
| - `SpvError`: Top-level errors wrapping specific domains | ||
|
|
||
| ## MSRV and Dependencies | ||
|
|
||
| - **Minimum Rust Version**: 1.80 | ||
| - **Core dependencies**: `dashcore`, `tokio`, `async-trait`, `thiserror` | ||
| - **Built on**: `dashcore` library with Dash-specific features enabled | ||
| - **Async runtime**: Tokio with full feature set | ||
|
|
||
| ## Key Implementation Details | ||
|
|
||
| ### Storage Architecture | ||
| - **Segmented storage**: Headers stored in 10,000-header segments with index files | ||
| - **Filter storage**: Separate storage for filter headers and compact block filters | ||
| - **State persistence**: Chain state, masternode data, and sync progress persisted between runs | ||
| - **Storage paths**: Headers in `headers/`, filters in `filters/`, state in `state/` | ||
|
|
||
| ### Async Architecture Patterns | ||
| - **Trait objects**: `Arc<dyn StorageManager>`, `Arc<dyn NetworkManager>` for runtime polymorphism | ||
| - **Message passing**: Tokio channels for inter-component communication | ||
| - **Timeout handling**: Configurable timeouts with recovery mechanisms | ||
| - **State machines**: `SyncState` enum drives synchronization flow | ||
|
|
||
| ### Debugging and Troubleshooting | ||
|
|
||
| **Common Debug Commands:** | ||
| ```bash | ||
| # Run with tracing output | ||
| RUST_LOG=debug cargo test --test integration_real_node_test -- --nocapture | ||
|
|
||
| # Run specific test with verbose output | ||
| cargo test --test handshake_test test_handshake_with_mainnet_peer -- --nocapture --test-threads=1 | ||
|
|
||
| # Check storage state | ||
| ls -la data*/headers/ | ||
| ls -la data*/state/ | ||
| ``` | ||
|
|
||
| **Debug Data Locations:** | ||
| - `test-debug/`: Debug data from test runs | ||
| - `data*/`: Runtime data directories (numbered by run) | ||
| - Storage index files show header counts and segment info | ||
|
|
||
| **Network Debugging:** | ||
| - Connection issues: Check if Dash Core node is running at `127.0.0.1:9999` | ||
| - Handshake failures: Verify network (mainnet/testnet/devnet) matches node | ||
| - Timeout issues: Node may be syncing or under load | ||
|
|
||
| ## Current Status | ||
|
|
||
| This is a refactored SPV client extracted from a monolithic example: | ||
| - ✅ Core architecture implemented and modular | ||
| - ✅ Compilation successful with comprehensive trait abstractions | ||
| - ✅ Extensive test coverage (29/29 implemented tests passing) | ||
| - ⚠️ Some wallet functionality still in development (see `PLAN.md`) | ||
| - ⚠️ ChainLock/InstantLock signature validation has TODO items | ||
|
|
||
| The project transforms a 1,143-line monolithic example into a production-ready, testable library suitable for integration into wallets and other Dash applications. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| [package] | ||
| name = "dash-spv" | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
| authors = ["Dash Core Team"] | ||
| description = "Dash SPV (Simplified Payment Verification) client library" | ||
| license = "MIT" | ||
| repository = "https://github.com/dashpay/rust-dashcore" | ||
| rust-version = "1.80" | ||
|
|
||
| [dependencies] | ||
| # Core Dash libraries | ||
| dashcore = { path = "../dash", features = ["std", "serde", "core-block-hash-use-x11", "message_verification"] } | ||
| dashcore_hashes = { path = "../hashes" } | ||
|
|
||
|
Comment on lines
+13
to
+15
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion
Crates with [package]
publish = false🤖 Prompt for AI Agents |
||
| # CLI | ||
| clap = { version = "4.0", features = ["derive"] } | ||
|
|
||
| # Async runtime | ||
| tokio = { version = "1.0", features = ["full"] } | ||
| async-trait = "0.1" | ||
|
|
||
| # Error handling | ||
| thiserror = "1.0" | ||
| anyhow = "1.0" | ||
|
|
||
| # Serialization | ||
| serde = { version = "1.0", features = ["derive"] } | ||
| serde_json = "1.0" | ||
| bincode = "1.3" | ||
|
|
||
| # Logging | ||
| tracing = "0.1" | ||
| tracing-subscriber = "0.3" | ||
|
|
||
| # Utilities | ||
| rand = "0.8" | ||
|
|
||
| # Terminal UI | ||
| crossterm = "0.27" | ||
|
|
||
| # DNS | ||
| trust-dns-resolver = "0.23" | ||
|
|
||
| # Also add log to main dependencies for consistency | ||
| log = "0.4" | ||
|
|
||
| [dev-dependencies] | ||
| tempfile = "3.0" | ||
| tokio-test = "0.4" | ||
| env_logger = "0.10" | ||
| hex = "0.4" | ||
|
|
||
| [[bin]] | ||
| name = "dash-spv" | ||
| path = "src/main.rs" | ||
|
|
||
| [lib] | ||
| name = "dash_spv" | ||
| path = "src/lib.rs" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Magic-byte update is only half-done – networking will mis-detect Regtest
You updated
Network::magic()to0xDCB7C1FC, but:Network::from_magic()(line 39) still returns the old value (0xDAB5BFFA).Don’t forget to adjust the corresponding expectations in
tests::test_network_magicandtests::test_network_from_magic; otherwise the crate will not compile.📝 Committable suggestion
🤖 Prompt for AI Agents