Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(vm): Fast VM integration #1949

Merged
merged 128 commits into from
Aug 9, 2024
Merged

feat(vm): Fast VM integration #1949

merged 128 commits into from
Aug 9, 2024

Commits on May 15, 2024

  1. start integrating vm2

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    fab3811 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7aadf9a View commit details
    Browse the repository at this point in the history
  3. add BootloaderState

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    0846b25 View commit details
    Browse the repository at this point in the history
  4. putting TX in bootloader

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    26bcbcb View commit details
    Browse the repository at this point in the history
  5. able to fail one test

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    2643d7e View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    fa9e2fa View commit details
    Browse the repository at this point in the history
  7. fix formatting

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    8f5ff12 View commit details
    Browse the repository at this point in the history
  8. pass first test

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    3022d1d View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    b9eac2a View commit details
    Browse the repository at this point in the history
  10. enable oracle test

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    d15348a View commit details
    Browse the repository at this point in the history
  11. clippy fixes

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    3e211cd View commit details
    Browse the repository at this point in the history
  12. enable more tests

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    5f7c227 View commit details
    Browse the repository at this point in the history
  13. enable more tests

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    e567d19 View commit details
    Browse the repository at this point in the history
  14. give bootloader free memory

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    84c2a7f View commit details
    Browse the repository at this point in the history
  15. refactor hooks

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    d954782 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    3b9100c View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    420f9cf View commit details
    Browse the repository at this point in the history
  18. Configuration menu
    Copy the full SHA
    b1f3297 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    27538ac View commit details
    Browse the repository at this point in the history
  20. implement event merging

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    6ddf8cc View commit details
    Browse the repository at this point in the history
  21. integrate l2 to l1 messages

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    556bcbb View commit details
    Browse the repository at this point in the history
  22. make send_and_transfer pass

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    3c7bd29 View commit details
    Browse the repository at this point in the history
  23. add pubdata

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    85594a3 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    533d92a View commit details
    Browse the repository at this point in the history
  25. implements is_write_initial for world

    montekki authored and joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    59cefe3 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    b682773 View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    ca0f399 View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    202f956 View commit details
    Browse the repository at this point in the history
  29. now passes refund test

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    31ac603 View commit details
    Browse the repository at this point in the history
  30. enable the majority of tests

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    99ed680 View commit details
    Browse the repository at this point in the history
  31. populate initialness of storage log

    Unfortunately this is slow as unnecessarily consults the DB
    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    03585ba View commit details
    Browse the repository at this point in the history
  32. emit published bytecodes

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    39c373d View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    caa95e8 View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    8d55601 View commit details
    Browse the repository at this point in the history
  35. even more snapshots

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    a3f4b03 View commit details
    Browse the repository at this point in the history
  36. Configuration menu
    Copy the full SHA
    b2d7dbe View commit details
    Browse the repository at this point in the history
  37. Configuration menu
    Copy the full SHA
    1b59fe5 View commit details
    Browse the repository at this point in the history
  38. update tests

    joonazan committed May 15, 2024
    Configuration menu
    Copy the full SHA
    6487107 View commit details
    Browse the repository at this point in the history
  39. Configuration menu
    Copy the full SHA
    3dde7a7 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. Configuration menu
    Copy the full SHA
    de72ad8 View commit details
    Browse the repository at this point in the history
  2. integrate into zksync_core

    This required changing the VM interface so that the lifetime is compatible with
    the other VMs.
    joonazan committed May 16, 2024
    Configuration menu
    Copy the full SHA
    6945a4e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3465369 View commit details
    Browse the repository at this point in the history
  4. fix lints

    joonazan committed May 16, 2024
    Configuration menu
    Copy the full SHA
    6880ddb View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. adapt to VM changes

    joonazan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    35ad661 View commit details
    Browse the repository at this point in the history
  2. depend on newest vm2

    joonazan committed May 21, 2024
    Configuration menu
    Copy the full SHA
    ea34526 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    fae9e9a View commit details
    Browse the repository at this point in the history
  4. rollback the changes

    montekki committed May 21, 2024
    Configuration menu
    Copy the full SHA
    96c44db View commit details
    Browse the repository at this point in the history
  5. Merge pull request #2003 from matter-labs/fvs-integrade-vm2-step-1

    feat(vm2): implement inspect_transaction_with_bytecode_compression
    joonazan authored May 21, 2024
    Configuration menu
    Copy the full SHA
    e3b6fe8 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. remove dependency overrides

    joonazan committed May 22, 2024
    Configuration menu
    Copy the full SHA
    a85d090 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f8bce48 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. Configuration menu
    Copy the full SHA
    694bea4 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #2028 from matter-labs/fvs-track-tx-numbers-and-de…

    …dup-logs
    
    fixes to logs deduplication and tx number tracking
    joonazan authored May 23, 2024
    Configuration menu
    Copy the full SHA
    eb2c87c View commit details
    Browse the repository at this point in the history
  3. adapt to new vm2

    joonazan committed May 23, 2024
    Configuration menu
    Copy the full SHA
    1668af7 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1f3f963 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. Configuration menu
    Copy the full SHA
    b8b317f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3e66243 View commit details
    Browse the repository at this point in the history
  3. adapt to iterators

    joonazan committed May 24, 2024
    Configuration menu
    Copy the full SHA
    dbd8d78 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. Configuration menu
    Copy the full SHA
    54dbd25 View commit details
    Browse the repository at this point in the history
  2. updates to newest vm2 code

    montekki committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    8771ed1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    316359b View commit details
    Browse the repository at this point in the history
  4. Merge pull request #2149 from matter-labs/fvs-update-integration-bran…

    …ch-jms-vm2
    
    chore: updates to newest vm2 code
    joonazan authored Jun 5, 2024
    Configuration menu
    Copy the full SHA
    90647e3 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1d207a6 View commit details
    Browse the repository at this point in the history
  6. update to latest vm2

    joonazan committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    99a4d00 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Configuration menu
    Copy the full SHA
    9414144 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Configuration menu
    Copy the full SHA
    6d8ef7f View commit details
    Browse the repository at this point in the history
  2. adapt to latest main

    joonazan committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    2c553df View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. support Decommit

    joonazan committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    c8dffb7 View commit details
    Browse the repository at this point in the history
  2. fix lints

    joonazan committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    ba16fa4 View commit details
    Browse the repository at this point in the history
  3. adapt to removing tx_number

    joonazan committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    b4a40af View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Configuration menu
    Copy the full SHA
    9103f44 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Use specific vm2 revision

    slowli committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    b25ed57 View commit details
    Browse the repository at this point in the history
  2. Sketch VM shadowing

    slowli committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    8b96b54 View commit details
    Browse the repository at this point in the history
  3. Fix shadowing

    slowli committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    a5abc2b View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. set_refund_for_current_tx (#2361)

    ## What ❔
    
    Fixes refund-related bugs in the glue code for the new VM:
    
    - Computes `pubdata_published` as a difference with the previous transaction
    - Sets refund for the bootloader state
    montekki authored Jul 2, 2024
    Configuration menu
    Copy the full SHA
    8468cd3 View commit details
    Browse the repository at this point in the history
  2. Refactor VM divergence checks

    slowli committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    2e794e2 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Configuration menu
    Copy the full SHA
    acb22cf View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f969530 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ad6ce5e View commit details
    Browse the repository at this point in the history
  4. Remove some garbage

    slowli committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    4d3a007 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    83d11dd View commit details
    Browse the repository at this point in the history
  6. Disable failing unit tests

    slowli committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    c372bfe View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. Configuration menu
    Copy the full SHA
    b23c351 View commit details
    Browse the repository at this point in the history
  2. Split off VmFactory trait

    slowli committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    5222c55 View commit details
    Browse the repository at this point in the history
  3. Update new VM

    slowli committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    38517bb View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d11d580 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #2378 from matter-labs/jms-vm2+aov

    feat(vm): Shadow mode + fixes
    montekki authored Jul 4, 2024
    Configuration menu
    Copy the full SHA
    fa7f237 View commit details
    Browse the repository at this point in the history

Commits on Jul 23, 2024

  1. Configuration menu
    Copy the full SHA
    c393bcc View commit details
    Browse the repository at this point in the history
  2. adapt to newest vm2

    joonazan committed Jul 23, 2024
    Configuration menu
    Copy the full SHA
    26ddf4e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    616e324 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. perf: writing tx to memory is no longer quadratic

    It used to grow proportionally to the number of transactions squared.
    joonazan committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    bbe33bc View commit details
    Browse the repository at this point in the history
  2. remove interim version

    joonazan committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    0b7069b View commit details
    Browse the repository at this point in the history
  3. convert less

    joonazan committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    02651c8 View commit details
    Browse the repository at this point in the history
  4. remove clone

    joonazan committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    f880405 View commit details
    Browse the repository at this point in the history
  5. fix compilation

    joonazan committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    0c81bb3 View commit details
    Browse the repository at this point in the history
  6. remove another clone

    joonazan committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    c2518e6 View commit details
    Browse the repository at this point in the history
  7. Merge pull request #2477 from matter-labs/fix-quadratic-blocks

    perf: writing tx to memory is no longer quadratic
    joonazan authored Jul 24, 2024
    Configuration menu
    Copy the full SHA
    f58a026 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    e88cf85 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Configuration menu
    Copy the full SHA
    8c21f48 View commit details
    Browse the repository at this point in the history
  2. Update from main

    slowli committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    4c67c2c View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. update vm2 revision

    joonazan committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    4c8878c View commit details
    Browse the repository at this point in the history
  2. try fix branch

    joonazan committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    6090791 View commit details
    Browse the repository at this point in the history
  3. Fix refund computation

    slowli committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    cf8dc20 View commit details
    Browse the repository at this point in the history
  4. Test refund computation

    slowli committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    a018fa8 View commit details
    Browse the repository at this point in the history
  5. fix: Fill storage_refunds / pubdata_costs data (#2431)

    These fields don't seem to be used by ENs, but are expected to be filled
    during the main state keeper execution cycle.
    slowli authored Jul 26, 2024
    Configuration menu
    Copy the full SHA
    fb800d1 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2024

  1. Update from upstream

    slowli committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    d46b173 View commit details
    Browse the repository at this point in the history
  2. depend on vm2 master

    joonazan committed Jul 29, 2024
    Configuration menu
    Copy the full SHA
    5db0cf5 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #2519 from matter-labs/aov-pla-999-fix-operator-re…

    …fund-divergency
    
    fix(vm): Fix operator refund divergency
    joonazan authored Jul 29, 2024
    Configuration menu
    Copy the full SHA
    2d03f44 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    7b163a9 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2024

  1. Merge pull request #2481 from matter-labs/jms-snapshot-compat

    chore: compatibility with the snapshot changes
    joonazan authored Jul 30, 2024
    Configuration menu
    Copy the full SHA
    7a44759 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f3d929f View commit details
    Browse the repository at this point in the history
  3. infinite loop

    montekki committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    bdce2de View commit details
    Browse the repository at this point in the history
  4. adds a random access

    montekki committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    c2c0569 View commit details
    Browse the repository at this point in the history
  5. changes test again

    montekki committed Jul 30, 2024
    Configuration menu
    Copy the full SHA
    2f2a454 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. 16000 array size

    montekki committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    9725643 View commit details
    Browse the repository at this point in the history
  2. remove sub

    montekki committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    b4ee16d View commit details
    Browse the repository at this point in the history
  3. Merge pull request #2542 from matter-labs/jms-vm2+fvs+bench

    chore: adds a benchmark for heaps read and writes
    joonazan authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    e4ddb30 View commit details
    Browse the repository at this point in the history
  4. perf(vm): Improve snapshot management in batch executor (#2513)

    ## What ❔
    
    - Does not embed snapshots into each other, and instead use a linear
    workflow.
    - Discards an old snapshot, if any, when processing a new transaction
    (currently, old snapshots are seemingly never discarded unless the
    transaction is rolled back).
    
    ## Why ❔
    
    - Linear snapshow workflow is easier to grasp and it could allow
    optimizations in the future.
    - Accumulating snapshots introduces an overhead, esp. in the new VM.
    slowli authored Jul 31, 2024
    Configuration menu
    Copy the full SHA
    69be78c View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. feat(vm): Allow switching between VMs for latest protocol version (#2508

    )
    
    ## What ❔
    
    - Allows using old (latest) VM by default, new VM only or old + new VM
    in a shadow mode in the `MainBatchExecutor`.
    - Allows configuring this mode for a new `VmRunner`-powered component,
    VM playground.
    
    ## Why ❔
    
    A separate component seems a relatively safe place to start integration from.
    slowli authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    77b6d81 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. refactor(vm-runner): Allow switching between VMs for latest protocol …

    …version – follow ups (#2567)
    
    ## What ❔
    
    Various follow-ups after
    #2508:
    
    - Adds VM playground config to `etc/env`.
    - Adds a health check for the VM playground.
    - Runs VM playground in server integration tests and checks it on
    teardown.
    
    ## Why ❔
    
    Improves maintainability and test coverage.
    slowli authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    c75f6e1 View commit details
    Browse the repository at this point in the history
  2. test(vm): Fix fast VM unit tests (#2560)

    ## What ❔
    
    Fixes unit tests that previously failed because of incorrect `Heap`
    comparisons.
    
    ## Why ❔
    
    Better test coverage.
    slowli authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    ca37336 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Update from upstream

    slowli committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    7bf449b View commit details
    Browse the repository at this point in the history
  2. Re-enable TEE verifier

    slowli committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    295fa8d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    459da8e View commit details
    Browse the repository at this point in the history
  4. Brush up TODOs in multivm

    slowli committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    7ec7ba7 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. Use vm2::FIRST_HEAP

    slowli committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    0dfba61 View commit details
    Browse the repository at this point in the history
  2. Merge pull request #2598 from matter-labs/aov-pla-1009-merge-vm-featu…

    …re-branch-into-main
    
    chore(vm): Update VM revision and TODOs
    joonazan authored Aug 7, 2024
    Configuration menu
    Copy the full SHA
    e481efb View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Update from upstream

    slowli committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    9a83049 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. feat(vm): Allow running new VM in state keeper (#2616)

    ## What ❔
    
    - Allows running the new VM in the state keeper.
    - Adds a CI load test with the new VM.
    
    ## Why ❔
    
    Allows to monitor new VM performance regressions.
    slowli authored Aug 9, 2024
    Configuration menu
    Copy the full SHA
    7da372f View commit details
    Browse the repository at this point in the history