feat(prover)!: Add EIP4844 support for provers subsystem #1200
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
EmilLuta
commented
Feb 26, 2024
EmilLuta
changed the title
EmilLuta
commented
Feb 26, 2024
perekopskiy
reviewed
Feb 27, 2024
core/lib/dal/migrations/20240226120530_add_support_for_eip4844.up.sql
Outdated
Show resolved
Hide resolved
core/lib/dal/migrations/20240226120530_add_support_for_eip4844.up.sql
Outdated
Show resolved
Hide resolved
prover/prover_dal/migrations/20240226120310_add_support_for_eip4844.up.sql
Outdated
Show resolved
Hide resolved
Co-authored-by: Roman Brodetski <rb@matterlabs.dev>
Co-authored-by: Roman Brodetski <rb@matterlabs.dev>
perekopskiy
previously approved these changes
Feb 27, 2024
|
Had a sync with @RomanBrodetski, he's approving, but doesn't have access to github ATM. |
…e01b13f chore: Update generated Prover FRI GPU setup-data keys from branch evl-4844-branch
Deniallugo
previously approved these changes
Feb 29, 2024
…e776783 chore: Update generated Prover FRI GPU setup-data keys from branch evl-4844-branch
montekki
approved these changes
Feb 29, 2024
This was referenced Feb 29, 2024
RomanBrodetski
pushed a commit
that referenced
this pull request
Mar 1, 2024
🤖 I have created a release *beep* *boop* --- ## [21.0.0](core-v20.8.0...core-v21.0.0) (2024-03-01) ### ⚠ BREAKING CHANGES * **prover:** Add EIP4844 support for provers subsystem ([#1200](#1200)) * Set 21 as latest protocol version ([#1262](#1262)) ### Features * A way to disable slow query logging for some queries ([#1297](#1297)) ([13b82a0](13b82a0)) * **aggregator:** support two operator addresses in sender and aggregator ([#1201](#1201)) ([7be56e5](7be56e5)) * change EN main node fetcher concurrency factor ([#1317](#1317)) ([d4235b5](d4235b5)) * **commitment-generator:** Commitment for 1.4.2 ([#1234](#1234)) ([9b773eb](9b773eb)) * **config:** add pubdata sending method to config ([#1261](#1261)) ([cebf55a](cebf55a)) * **config:** Added blobs to commit batches and made da source part of config ([#1243](#1243)) ([e01d1b6](e01d1b6)) * **config:** update eth_sender to use blobs ([#1295](#1295)) ([e81f080](e81f080)) * **core:** Adds support for 4844 transaction signing and rlp encoding ([#1254](#1254)) ([10e3a3e](10e3a3e)) * **dal:** `zksync_types::Transaction` to use protobuf for wire encoding (BFT-407) ([#1047](#1047)) ([ee94bee](ee94bee)) * **eth_sender:** set blob gas fee when blobs DA is used ([#1285](#1285)) ([57c5526](57c5526)) * **gas-adjuster:** gas adjuster for EIP4844 ([#1255](#1255)) ([1da97ed](1da97ed)) * Metrics for block tip ([#1144](#1144)) ([85d4b12](85d4b12)) * **node_framework:** Add Web3 API layers ([#1258](#1258)) ([105f4cc](105f4cc)) * **node_framework:** Support Proof Data Handler in the framework ([#1233](#1233)) ([2191218](2191218)) * **prover:** Add EIP4844 support for provers subsystem ([#1200](#1200)) ([6953e89](6953e89)) * Remove methods to sign an arbitraty message ([#1294](#1294)) ([8904123](8904123)) * Set 21 as latest protocol version ([#1262](#1262)) ([30579ef](30579ef)) * support loading verifier and protocol version from db ([#1293](#1293)) ([533f013](533f013)) ### Bug Fixes * Add EIP4844 to fri_prover_group_config ([#1309](#1309)) ([edf9397](edf9397)) * **api:** Use better gas per pubdata in API (pure server changes) ([#1311](#1311)) ([54f8d8c](54f8d8c)) * **docker:** change default password for postgres ([#1250](#1250)) ([f6bff74](f6bff74)) * **en:** fail fast if we don't request correct number of txs from man node ([#1269](#1269)) ([1bcbf17](1bcbf17)) * Fix scheduler enqueuer bug ([#1322](#1322)) ([95deb92](95deb92)) * **snapshots_creator:** Fix snapshot generation query ([#1289](#1289)) ([e279456](e279456)) * **vm:** fix match clause in `get_max_gas_per_pubdata_byte` ([#1292](#1292)) ([eaf5a50](eaf5a50)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
github-merge-queue bot
pushed a commit
that referenced
this pull request
Mar 5, 2024
🤖 I have created a release *beep* *boop* --- ## [12.0.0](prover-v11.0.0...prover-v12.0.0) (2024-03-04) ### ⚠ BREAKING CHANGES * **prover:** Add EIP4844 support for provers subsystem ([#1200](#1200)) * Set 21 as latest protocol version ([#1262](#1262)) ### Features * Adding ability to generate 4844 setup key and refactor ([#1143](#1143)) ([975f54b](975f54b)) * **api:** Remove unused and obsolete token info ([#1071](#1071)) ([e920897](e920897)) * **dal:** `zksync_types::Transaction` to use protobuf for wire encoding (BFT-407) ([#1047](#1047)) ([ee94bee](ee94bee)) * **db:** Soft-remove `storage` table ([#982](#982)) ([601f893](601f893)) * **en:** Integrate snapshots recovery into EN ([#1032](#1032)) ([c7cfaf9](c7cfaf9)) * **healthcheck:** Various healthcheck improvements ([#1166](#1166)) ([1e34148](1e34148)) * improving verification key generation ([#1050](#1050)) ([6f715c8](6f715c8)) * Prover interface and L1 interface crates ([#959](#959)) ([4f7e107](4f7e107)) * **prover:** Add EIP4844 support for provers subsystem ([#1200](#1200)) ([6953e89](6953e89)) * **prover:** Added --recompute-if-missing option to key generator ([#1151](#1151)) ([cad7278](cad7278)) * **prover:** Added 4844 circuit to verification keys ([#1141](#1141)) ([8b0cc4a](8b0cc4a)) * **prover:** Adding first support for 4844 circuit ([#1155](#1155)) ([6f63c53](6f63c53)) * **prover:** adding keystore object to handle reading and writing of prover keys ([#1132](#1132)) ([1471615](1471615)) * **prover:** merging key generation into a single binary ([#1101](#1101)) ([6de8b84](6de8b84)) * **prover:** Moved setup key generation logic to test harness ([#1113](#1113)) ([469ab06](469ab06)) * **prover:** Use new shivini function for 4844 circuits ([#1205](#1205)) ([376c09e](376c09e)) * Set 21 as latest protocol version ([#1262](#1262)) ([30579ef](30579ef)) * **vlog:** Remove env getters from vlog ([#1077](#1077)) ([00d3429](00d3429)) ### Bug Fixes * fix link ([#1007](#1007)) ([f1424ce](f1424ce)) * make `zk status prover` use the new prover table ([#1044](#1044)) ([9b21d7f](9b21d7f)) * **prover:** Decouple core/ prover database management ([#1029](#1029)) ([37674fd](37674fd)) * **prover:** Fix initial prover migration ([#1083](#1083)) ([6d54010](6d54010)) * **prover:** QoL socket utilization ([#1020](#1020)) ([13a6816](13a6816)) * update harness to include fix to new boojum OOM ([#1053](#1053)) ([4976941](4976941)) ### Performance Improvements * bump harness version ([#1003](#1003)) ([1cbb4c9](1cbb4c9)) * reduce memory consumption of witness generation ([#696](#696)) ([dea6768](dea6768)) * upgrade harness version to improve witness generation memory spike ([#1034](#1034)) ([09bbb84](09bbb84)) * use jemalloc in witness generator ([#1014](#1014)) ([917b2dc](917b2dc)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for EIP4844 as part of 1.4.2 release.
To support EIP4844, the prover needs to:
EIP4844 is special cased for 1.4.2. Later (1.5.0) it will behave like any other circuit.
How EIP4844 works?
Blob proofs are generated at basic/ aggregation_round 0 stage and are marked as final nodes to be inserted into scheduler. If there is a single blob, the witness for the second blob needs to be zeroed out whilst the proof still needs to be injected (any proof will work). Blobs are stored in
witness_inputs_friandscheduler_dependency_trackeris aware of the proofs ascircuit_255_final_prover_job_id_0for the first blob andcircuit_255_final_prover_job_id_1for the second blob. (circuit id 255 is blob, until the code-base migrates to 1.5.0, when it will become a regular circuit).