Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Enhanced Orthogonal Persistence (64-Bit with Graph Copy) (#4475)
* Graph copy: Work in progress * Implement stable memory reader writer * Add skip function * Code refactoring * Continue stabilization function * Support update at scan position * Code refactoring * Code refactoring * Extend unit test * Continue implementation * Adjust test * Prepare memory compatibility check * Variable stable to-space offset * Deserialize with partitioned heap * Prepare metadata stabilization * Adjust stable memory size * Stabilization version management * Remove code redundancies * Fix version upgrade * Put object field hashes in a blob * Support object type * Code refactoring * Support blob, fix bug * Renaming variable * Adjust deserialization heap start * Handle null singleton * Fix version upgrade * Support regions * Backup first word in stable memory * Support additional fields in upgraded actor * Make unit tests runnable again * Dummy null singleton in unit test * Add test cases * Support boxed 32-bit and 64-bit numbers * Support more object types * Support more object types * Handle `true` bool constant * Grow main memory on bulk copy * Update benchmark results * Support bigint * Clear deserialized data in stable memory * Update test results * Add documentation * Reformat * Add missing file * Update design/GraphCopyStabilization.md Co-authored-by: Claudio Russo <claudio@dfinity.org> * Update rts/motoko-rts/src/stabilization.rs Co-authored-by: Claudio Russo <claudio@dfinity.org> * Update rts/motoko-rts/src/stabilization.rs Co-authored-by: Claudio Russo <claudio@dfinity.org> * Graph Copy: Explicit Stable Data Layout (#4293) Refinement of Graph-Copy-Based Stabilization (#4286): Serialize/deserialize in an explicitly defined and fixed stable layout for a long-term perspective. * Supporting 64-bit pointer representations in stable format, even if main memory currently only uses 32-bit addresses. Open aspect: * Make `BigInt` stable format independent of Tom's math library. * Update rts/motoko-rts/src/stabilization.rs Co-authored-by: Claudio Russo <claudio@dfinity.org> * Update rts/motoko-rts/src/stabilization/layout.rs Co-authored-by: Claudio Russo <claudio@dfinity.org> * Handle non-stable fields in stable records * Add object type `Some` * Add test case * Adjust stabilization to incremental GC * Update benchmark results * Distinguish assertions * Fix RTS unit test * Update benchmark results * Adjust test * Adjust test * Fix: Handle all non-stable types during serialization * Fix typos and complete comment * Experiment: Simplified Graph-Copy-Based Stabilization (#4313) # Experiment: Simplified Graph-Copy-Based Stabilization **Simplified version of #4286, without stable memory buffering and without memory flipping on deserialization.** Using graph copying instead of Candid-based serialization for stabilization, to save stable variables across upgrades. ## Goals * **Stop-gap solution until enhanced orthogonal persistence**: More scalable stabilization than the current Candid(ish) serialization. * **With enhanced orthogonal persistence**: Upgrades in the presence of memory layout changes introduced by future compiler versions. ## Design Graph copy of sub-graph of stable objects from main memory to stable memory and vice versa on upgrades. ## Properties * Preserve sharing for all objects like in the heap. * Allow the serialization format to be independent of the main memory layout. * Limit the additional main memory needed during serialization and deserialization. * Avoid deep call stack recursion (stack overflow). ## Memory Compatibility Check Apply a memory compatibility check analogous to the enhanced orthogonal persistence, since the upgrade compatibility of the graph copy is not identical to the Candid subtype relation. ## Algorithm Applying Cheney’s algorithm [1, 2] for both serialization and deserialization: ### Serialization * Cheney’s algorithm using main memory as from-space and stable memory as to-space: * Focusing on stable variables as root (sub-graph of stable objects). * The target pointers and Cheney’s forwarding pointers denote the (skewed) offsets in stable memory. * Using streaming reads for the `scan`-pointer and streaming writes for the `free`-pointer in stable memory. ### Deserialization * Cheney’s algorithm using stable memory as from-space and main memory as to-space: * Starting with the stable root created during the serialization process. * Objects are allocated in main memory using the default allocator. * Using random read/write access on the stable memory. ## Stable Format For a long-term perspective, the object layout of the serialized data in the stable memory is fixed and independent of the main memory layout. * Pointers support 64-bit representations, even if only 32-bit pointers are used in current main memory address space. * The Brooks forwarding pointer is omitted (used by the incremental GC). * The pointers encode skewed stable memory offsets to the corresponding target objects. * References to the null objects are encoded by a sentinel value. ## Specific Aspects * The null object is handled specifically to guarantee the singleton property. For this purpose, null references are encoded as sentinel values that are decoded back to the static singleton of the new program version. * Field hashes in objects are serialized in a blob. On deserialization, the hash blob is allocated in the dynamic heap. Same-typed objects that have been created by the same program version share the same hash blob. * Stable records can dynamically contain non-stable fields due to structural sub-typing. A dummy value can be serialized for such fields as a new program version can no longer access this field through the stable types. * For backwards compatibility, old Candid destabilzation is still supported when upgrading from a program that used older compiler version. * Incremental GC: Serialization needs to consider Brooks forwarding pointers (not to be confused with the Cheney's forwarding information), while deserialization can deal with partitioned heap that can have internal fragmentation (free space at partition ends). ## Complexity Specific aspects that entail complexity: * For each object type, not only serialization and deserialization needs to be implemeneted but also the pointer scanning logic of its serialized and deserialized format. Since the deserialization also targets stable memory the existing pointer visitor logic cannot be used for scanning pointers in its deserialized format. * The deserialization requires scanning the heap which is more complicated for the partitioned heap. The allocator must yield monotonously growing addresses during deserialization. Free space gaps are allowed to complete partitions. ## Open Aspects * Unused fields in stable records that are no longer declared in a new program versions should be removed. This could be done during garbage collection, when objects are moved/evacuated. * The binary serialization and deserialization of `BigInt` entails dynamic allocations (cf. `mp_to_sbin` and `mp_from_sbin` of Tom's math library). ## Related PRs * Motoko Enhanced Orthogonal Persistence: #4225 * Motoko Incremental Garbage Collector: #3837 ## References [1] C. J. Cheney. A Non-Recursive List Compacting Algorithm. Communications of the ACM, 13(11):677-8, November 1970. [2] R. Jones and R. Lins. Garbage Collection: Algorithms for Automatic Dynamic Memory Management. Wiley 2003. Algorithm 6.1: Cheney's algorithm, page 123. * Bug fix: Allocations are not monotonically growing in partitioned heap for large objects * Update benchmark results * Update benchmark results * Drop content of destabilized `Any`-typed actor field * Refactor `is_primitive_type` in Candid parser and subtype check * Do not use the cache for the main actor type compatibility check * Update benchmark results * Increase chunk size for stable memory clearing * Custom bigint serialization * Update benchmark results * Update documentation * Update documentation * Optimize array deserialization * Update benchmark results * Code refactoring of upgrade version checks * Remove redundant math functions * Eliminate size redundancy in the `Object` header * Also adjust the `Object` header in the compiler * Revert "Also adjust the `Object` header in the compiler" This reverts commit f75bb76. * Revert "Eliminate size redundancy in the `Object` header" This reverts commit 0fe3926. * Record the upgrade instruction costs * Update tests for new `Prim.rts_upgrade_instructions()` function * Make test more ergonomic * Incremental Graph-Copy-Based Upgrades (#4361) # Incremental Graph-Copy-Based Upgrades Refinement of #4286 Supporting arbitrarily large graph-copy-based upgrades beyond the instruction limit: * Splitting the stabilization/destabilization in multiple asynchronous messages. * Limiting the stabilization work units to fit the update or upgrade messages. * Blocking other messages during the explicit incremental stabilization. * Restricting the upgrade functionality to the canister owner and controllers. * Stopping the GC during the explicit incremental upgrade process. ## Usage For large upgrades: 1. Initiate the explicit stabilization before the upgrade: ``` dfx canister call CANISTER_ID __motoko_stabilize_before_upgrade "()" ``` * An assertion first checks that the caller is the canister owner or a canister controller. * All other messages to the canister will be blocked until the upgrade has been successfully completed. * The GC is stopped. * If defined, the actor's pre-upgrade function is called before the explicit stabilization. * The stabilzation runs in possibly multiple asynchronous messages, each with a limited number of instructions. 2. Run the actual upgrade: ``` dfx deploy CANISTER_ID ``` * Run and complete the stabilization if not yet done in advance. * Perform the actual upgrade of the canister on the IC. * Start the destabilization with a limited number of steps to fit into the upgrade message. * If destabilization cannot be completed, the canister does not start the GC and does not accept messages except step 3. 3. Complete the explicit destabilization after the upgrade: ``` dfx canister call CANISTER_ID __motoko_destabilze_after_upgrade "()" ``` * An assertion checks that the caller is the canister owner or a canister controller. * All other messages remain blocked until the successful completion of the destabilization. * The destabilzation runs in possibly multiple asynchronous messages, each with a limited number of instructions. * If defined, the actor's post-upgrade function is called at the end of the explicit destabilization. * The GC is restarted. ## Remarks * Steps 1 (explicit stabilization) and/or 2 (explicit destabilization) may not be needed if the corresponding operation fits into the upgrade message. * Stabilization and destabilization steps are limited to the increment limits: Operation | Message Type | IC Instruction Limit | **Increment Limit** ----------|--------------|----------------------|-------------------- **Explicit (de)stabilization step** | Update | 20e9 | **16e9** **Actual upgrade** | Upgrade | 200e9 | **160e9** * The stabilization code in the RTS has been restructured to be less monolithic. * Manual merge conflict resolution (work in progress) * Adjust tests, resolve some merge bugs * Adjust RTS test case * Make RTS tests run again * Add missing function export * Adjust imports, manual merge conflict resolution * Manual merge conflict resolution * Manual merge conflict resolution * Adjust persistence initialization * Adjust persistence version management * Adjust stable memory metadata for enhanced orthogonal persistence Distinguish enhanced orthogonal persistence from Candid legacy stabilization * Add comment * Adjust graph stabilization initialization * Adjust GC mode during destabilization * Adjust object visitor for graph destabilization * Adjust incremental graph destabilization * Adjust error message * Adjust tests * Adjust tests * Update benchmark results * Adjust test * Upgrade stable memory version after graph destabilization * Adjust memory sanity check * Clear memory on graph destabilization as first step * Adjust big int serialization for 64-bit * Fix: Clear memory on graph destabilization * Add test case for graph stabilization * Add test case for incremental graph stabilization * Add tests for graph stabilization * Add more tests for graph stabilization * Add more test cases for graph stabilization * Add more test cases for graph stabilization * More conservative persistence version check * Adjust expected test results * Adjust test * Adjust tests * Adjust tests * Adjust RTS test for stabilization * Adjust tests * Adjust test results * Remove unwanted binary files * Adjust comment * Code refactoring * Fix merge mistake * Manual merge conflict resolution * Add test cases * Manual merge conflict resolution * Fix typo in documentation Co-authored-by: Claudio Russo <claudio@dfinity.org> * Fix typo in documentation Co-authored-by: Claudio Russo <claudio@dfinity.org> * Bug fix: Allow stabilization beyond compiler-specified stable memory limit * Adjustment to RTS unit tests * Add comments * Code refactoring * Fix difference between debug and release test execution * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Delete unused file * Code refactoring * Use correct trap for an unreachable case * Remove dead code * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Fix typo in function identifier * Fix indendation Co-authored-by: Claudio Russo <claudio@dfinity.org> * Removing unused code * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Fix RTS compile error * Bug fix: Object size lookup during stabilization * experiment: refactoring of ir extensions in graph-copy PR (#4543) * refactoring of ir * fix arrange_ir.ml --------- Co-authored-by: luc-blaeser <luc.blaeser@dfinity.org> * Manual merge conflict resolution * Adjust test case, remove file check * Manual merge conflict resolution * Manual merge conflict resolution * test graph copy of text and blob iterators (#4562) * Optimize instruction limit checks * Bug fix graph copy limit on destabilization * Incremental stable memory clearing after graph copy * Parameter tuning for graph copy * Manual merge conflict resolution * Manual merge conflict resolution * Remove redundant code * Manual merge conflict resolution: Remove `ObjInd` from graph-copy stabilization * Manual merge conflict resolution * Merge Preparation: Latest IC with Graph Copy (#4630) * Adjust to new system API * Port to latest IC 64-bit system API * Update to new IC with Wasm64 * Updating nix hashes * Update IC dependency (Wasm64 enabled) * Update expected test results * Fix migration test * Use latest `drun` * Adjust expected test results * Updating nix hashes * Update expected test results * Fix `drun` nix build for Linux * Disable DTS in `drun`, refactor `drun` patches * Update expected test results for new `drun` * Limiting amount of stable memory accessed per graph copy increment * Reformat * Adjust expected test result --------- Co-authored-by: Nix hash updater <41898282+github-actions[bot]@users.noreply.github.com> * Message-dependent stable memory access limit * Graph copy: Fix accessed memory limit during stabilization * Enhanced Orthogonal Persistence (Complete Integration) (#4488) * Prepare two compilation targets * Combined RTS Makefile * Port classical compiler backend to combined solution * Adjust nix config file * Start combined RTS * Reduce classical compiler backend changes * Continue combined RTS * Make RTS compilable for enhanced orthogonal persistence * Make RTS tests runnable again for enhanced orthogonal persistence * Adjust compiler backend of enhanced orthogonal persistence * Unify Tom's math library binding * Make classical non-incremental RTS compile again * Make classical incremental GC version compilable again * Make all RTS versions compile again * Adjust memory sanity check for combined RTS modes * Prepare RTS tests for combined modes * Continue RTS test merge * Continue RTS tests combined modes * Continue RTS tests support for combined modes * Adjust LEB128 encoding for combined mode * Adjust RTS test for classical incremental GC * Adjust RTS GC tests * Different heap layouts in RTS tests * Continue RTS GC test multi-mode support * Make all RTS run again * Adjust linker to support combined modes * Adjust libc import in RTS for combined mode * Adjust RTS test dependencies * Bugfix in Makefile * Adjust compiler backend import for combined mode * Adjust RTS import for combined mode * Adjust region management to combined modes * Adjust classical compiler backend to fit combined modes * Reorder object tags to match combined RTS * Adjust test * Adjust linker for multi memory during Wasi mode with regions * Adjust tests * Adjust bigint LEB encoding for combined modes * Adjust bigint LEB128 encoding for combined modes * Adjust test * Adjust tests * Adjust test * Code refactoring: SLEB128 for BigInt * Adjust tests * Adjust test * Reformat * Adjust tests * Adjust benchmark results * Adjust RTS for unit tests * Reintroduce compiler flags in classical mode * Support classical incremental GC * Add missing export for classical incremental GC * Adjust tests * Adjust test * Adjust test * Adjust test * Adjust test * Adjust test * Adjust test * Pass `keep_main_memory` upgrade option only for enhanced orthogonal persistence * Adjust test * Update nix hash * Adjust Motoko base dependency * Adjust tests * Extend documentation * Adjust test * Update documentation * Update documentation * Manual merge conflict resolution * Manual merge refinement * Manual merge conflict resolution * Manual merge conflict resolution * Refactor migration test from classical to new persistence * Adjust migration test * Manual merge conflict resolution * Manual merge conflict resolution * Adjust compiler reference documentation * Test CI build * Test CI build * Adjust performance comparison in CI build * Manual merge conflict resolution * Add test for migration paths * Adjust test for integrated PR * Adjust test case * Manual merge conflict resolution * Manual merge conflict resolution * Manual merge conflict resolution * Manual merge conflict resolution * Code refactoring * Fix typo in comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Manual merge conflict resolution * Add static assertions, code formatting * Manual merge conflict resolution * Add test case * Refine comment Co-authored-by: Claudio Russo <claudio@dfinity.org> * Manual merge conflict resolution * Manual merge conflict resolution * Code refactoring * Manual merge conflict resolution * Adjust test run script messages * Manual merge conflict resolution * Manual merge conflict resolution * Manual merge conflict resolution * Manual merge conflict resolution * Merge Preparation: Dynamic Memory Capacity for Integrated EOP (#4586) * Tune for unknown memory capacity in 64-bit * Adjust benchmark results * Fix debug assertion, code refactoring * Manual merge conflict resolution * Manual merge conflict resolution * Code refactoring: Improve comments * Reformat * Fix debug assertion * Re-enable memory reserve for upgrade and queries See PR #4158 * Manual merge conflict resolution * Manual merge conflict resolution * Update benchmark results * Manual merge conflict resolution * Manual merge conflict resolution * Merge Preparation: Latest IC with Integrated EOP (#4638) * Adjust to new system API * Port to latest IC 64-bit system API * Update to new IC with Wasm64 * Updating nix hashes * Update IC dependency (Wasm64 enabled) * Update expected test results * Fix migration test * Use latest `drun` * Adjust expected test results * Updating nix hashes * Update expected test results * Fix `drun` nix build for Linux * Disable DTS in `drun`, refactor `drun` patches * Update expected test results for new `drun` * Limiting amount of stable memory accessed per graph copy increment * Reformat * Manual merge conflict resolution * Manual merge conflict resolution * Adjust expected test result --------- Co-authored-by: Nix hash updater <41898282+github-actions[bot]@users.noreply.github.com> * Manual merge conflict resolution * Documentation Update for Enhanced Orthogonal Persistence (#4670) --------- Co-authored-by: Claudio Russo <claudio@dfinity.org> Co-authored-by: Nix hash updater <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: Claudio Russo <claudio@dfinity.org> Co-authored-by: Nix hash updater <41898282+github-actions[bot]@users.noreply.github.com>
- Loading branch information
3 people
authored
Aug 26, 2024
1 parent
0de3aeb
commit a9184eb