This repository documents DeltaDeFi's experience integrating and operating Hydra - the Layer 2 scaling solution for Cardano.
We extend our appreciation to the Hydra team at IOG/Cardano Scaling for their groundbreaking work on this technology. The issues documented here arise from stress-testing Hydra under production-like conditions for our DEX use case, and we hope this documentation contributes back to the ecosystem.
Purpose: Document issues, investigations, and fixes related to Hydra usage in DeltaDeFi's trading infrastructure.
hydra-issues/
├── README.md # This file - overview and top issues
├── issues/ # Detailed issue documentation
│ ├── incremental/ # IC/ID stale snapshot issues
│ ├── stress-test/ # TPS and load testing
│ ├── snapshot-confirm-instability/ # Snapshot confirmation issues
│ ├── error-message-inconsistency/ # False error responses
│ ├── sideload-instability/ # Sideload recovery issues
│ └── memory-bloat/ # Memory usage issues (resolved)
├── knowledge-base/ # Research and documentation
└── debug/ # Debug scripts and tools
| Issue | Description | Details | Upstream |
|---|---|---|---|
| TPS Limitation | Hydra processes transactions sequentially, limiting throughput to ~10 TPS. Need 1k TPS for multi-pair trading. | stress-test | - |
| Snapshot Confirm Instability | At ~10 TPS, snapshots fail to confirm. Head becomes unresponsive due to single sequential event queue blocking consensus. | snapshot-confirm-instability | - |
| Incremental Commit/Decommit Instability | IC/ID operations become stale after L1 finalization due to version race conditions. Recovery via sideload possible but not ideal. | incremental | #2446 |
| Issue | Description | Details | Related |
|---|---|---|---|
| Error Message Inconsistency | Tx submit returns error but transaction accepted by snapshot, causing state mismatch. | error-message-inconsistency | #2434 |
| Sideload Instability | Sideload snapshot timeout during recovery. Edge case only. | sideload-instability | - |
| Issue | Resolution | Details |
|---|---|---|
| Make 1.3.0 changes optional | #2432 | Random downtime from security fix not applicable to our trust model |
| Memory Bloat | StrictData | Higher than linear space complexity with UTXO growth |
Issues and fixes are tracked in the DeltaDeFi Hydra Fork.