CIP-???? | Non-segregated Block Body Serialization

[teodanciu]

Simplify serialization of transactions in a block

---

(latest version of proposal)

[rphair]

@teodanciu @lehins - looks good in format & readability; we'll introduce this in Triage at the next CIP meeting: https://hackmd.io/@cip-editors/119

[rphair]

@teodanciu please let's avoid force pushing again, as requested here: https://github.com/cardano-foundation/CIPs/blob/master/CIP-0001/README.md#1a-authors-open-a-pull-request

This is a shared resource for commentary & editing, not a mirror of the author's code. We need to keep track of review points and correlate these with changes to the branch, and force pushing effectively wipes out that history. 🙏

[teodanciu]

That makes sense, I apologize. Thank you for the information!

[rphair]

@teodanciu @lehins today's CIP meeting has not yet confirmed this as a CIP candidate mainly (as I understood from meeting discussion) because we can't verify that other node development teams have endorsed this. We invited people at the meeting to tag relevant reviewers from those teams & so hopefully that will proceed next.

Including the Nested Transactions already mentioned in the proposal, we also noted that potential changes to the block body make this proposal considerable with these other CIPs (and so tagging authors & some reviewers of these proposals):

• CIP-0118? | Nested Transactions #862 - cc @polinavino @fallen-icarus @WhatisRT
• CIP-???? | Merkle Root of Transactions in Block Header #964 - cc @aslesarenko @colll78
• CIP-0150 | Block Data Compression - cc @sierkov @Quantumplation

[lehins]

One important point we failed to mention in this CIP is that this rearrangement will also change the block body hash computation, while making it less complicated.
Currently block body hash is computed with this weird indirection:

blockBodyHash = hash(hash(txBodies) + hash(txWitnesses) + hash(txAuxData) + hash(txSeqIsValids))

We should mention in the CIP that block body hash computation will change to a single invocation of the hash function on the serialized block body. This simplification will be especially useful for Peras and Leios, since we'll be adding their respective certificates to the block body when those features get implemented:

blockBodyHash = hash(block_body)

where a block body has a clear definition:

block_body =
  [ transactions : [* transaction]
  , invalid_transactions : [* transaction_index]
  ]

[rphair]

@teodanciu @lehins it's been suggested at today's CIP meeting (where this came up on a list of non-progressed proposals) that this could be done at an upcoming inter-era hard fork & the Haskell Ledger team might have this as a planned feature anyway... could we get this updated in the meantime to be sure the CIP itself is accurate by such a time? Will leave Waiting for Author in the meantime.

[lehins]

this could be done at an upcoming inter-era hard fork & the Haskell Ledger team might have this as a planned feature anyway...

@rphair No, unfortunately, this CIP cannot be implemented at an intra-era hard fork, since it introduces changes to serialization. This is however planned for Dijkstra era, since if we don't implement this CIP for Dikstra era, Nested Transaction implementation will make current approach of transactions serialization in a block incomprehensible.

[ch1bo]

I like this proposal. We are in desperate need of simplicity on Cardano.

However, I'm wondering why we do not lean into this part of the motivation and also inline the in-/validity flags?

[rphair]

@teodanciu I think we're just waiting for you to progress systematically through all unaddressed comments in the reviews above before removing the Waiting for Author tag & getting this on a steady review track again.

[rphair]

@teodanciu @lehins @ch1bo - at the rate we are progressing now — with older issues addressed and response to some newer issues — we'd be able to confirm this as a candidate at our next CIP meeting, so I've put this on the agenda for Review there: https://hackmd.io/@cip-editors/127

[ch1bo]

This is in a good state and I'd be happy to work with this new block format. The ecosystem impact is going to be substantial, but the timing of this is good because of other similar changes that all seem to be targeting Dijkstra.

[rphair]

@ch1bo I would also like to see this confirmed as a candidate so if @Ryun1 @perturbing can provide quorum here on GitHub we can assign a CIP number without having to wait for the next meeting & so begin more ambitious review.