feat: Integrate StorageService for large controller data

[andrepimenta]

Integrate StorageService for Large Controller Data

Description

Integrates @metamask/storage-service to enable controllers to offload large, infrequently-accessed data from Redux state to persistent storage.

Problem

State bloat impacting mobile performance:

• 10.79 MB total Engine state
• 92% (9.94 MB) in just 2 controllers storing rarely-accessed data
• Slow app startup parsing 10.79 MB on every launch
• High memory usage (all data loaded even if not needed)
• Slow persist operations (up to 6.26 MB per controller change)

Root cause: Controllers store large data (snap source code, token caches) in Redux state instead of on disk.

Solution

Add StorageService integration - Controllers can now store large data via messenger:

// Store data (out of state, on disk)
await messenger.call(
  'StorageService:setItem',
  'ControllerA',
  'data-key',
  largeData,
);

// Load on demand (lazy loading)
const data = await messenger.call(
  'StorageService:getItem',
  'ControllerA',
  'data-key',
);

What's Included

StorageService Integration:

• ✅ storage-service-init.ts - FilesystemStorage adapter for mobile
• ✅ storage-service-messenger.ts - Messenger factory
• ✅ Engine registration - Service available to all controllers
• ✅ Type definitions - StorageService in Engine types
• ✅ Messenger delegation - Actions/events configured
• ✅ Tests - Integration verified (5 tests passing)

FilesystemStorage Adapter (mobile-specific):

• Implements StorageAdapter interface
• Handles namespace filtering (getAllKeys, clear)
• Uses STORAGE_KEY_PREFIX constant (storageService:)
• iOS/Android platform handling
• Error logging

Impact

Immediate (infrastructure ready):

• ✅ Service available for controllers to use
• ✅ Zero performance impact (no controllers using yet)

When controllers migrate (separate PRs):

• 92% state reduction (10.79 MB → 0.85 MB)
• SnapController: 5.95 MB sourceCode → disk
• TokenListController: 3.99 MB cache → disk
• Faster app startup (92% less data to parse)
• Memory freed: 9.94 MB

Architecture

Platform-agnostic service:

Controller → messenger.call('StorageService:setItem', ...)
  ↓
StorageService (core) → Publishes events
  ↓
FilesystemStorage (mobile) → Persists to disk

Key format: storageService:{namespace}:{key}
Example: storageService:SnapController:snap-id:sourceCode

Testing

# Run integration tests
yarn jest app/core/Engine/controllers/storage-service-init.test.ts
yarn jest app/core/Engine/messengers/storage-service-messenger.test.ts

# Expected: 5 tests pass

Test coverage:

• ✅ Service initialization
• ✅ FilesystemStorage adapter integration
• ✅ iOS/Android handling
• ✅ Messenger creation
• ✅ Error handling

Follow-up Work

Controller migrations (separate PRs):

1. SnapController - Remove 5.95 MB sourceCode from state
2. TokenListController - Remove 3.99 MB cache from state

Each migration requires:

• Controller changes (call StorageService)
• Store migration (move existing data)
• Testing and benchmarking

Related

• ADR: 0017-storage-service-large-data.md
• Core package PR: feat: Add StorageService for offloading large controller data core#7192
• Dependencies: Requires @metamask/storage-service@^1.0.0 (from core)

Checklist

• Tests pass (5/5)
• No linter errors (except spurious messenger type warning)
• TypeScript compiles
• Follows existing service integration patterns (ErrorReportingService)
• Platform-specific code in adapter only
• No breaking changes
• Performance benchmarking (after controller migrations)

Notes

This PR adds infrastructure only - No immediate user-facing changes.

Performance improvements come when controllers are migrated to use StorageService (separate PRs for each controller).

Why FilesystemStorage? Optimized for large files (multi-MB). MMKV is better for small, frequently-accessed data (< 1 MB).

[github-actions]

🔍 Smart E2E Test Selection

• Selected E2E tags: SmokeCore, SmokeWalletPlatform, SmokeAccounts
• Risk Level: high
• AI Confidence: 85%

click to see 🤖 AI reasoning details

Summary

This PR introduces a new StorageService to the core Engine infrastructure of MetaMask Mobile. This is a significant architectural change that adds persistent storage capabilities for controller data.

Changes Made

1. New Dependency: Added @metamask/storage-service package to package.json (local file reference)

2. New Controller Integration: Added StorageService as a new controller in the Engine initialization

3. New Files:

   • storage-service-init.ts: Implements mobile-specific storage adapter using FilesystemStorage
   • storage-service-messenger.ts: Sets up messaging for the service
   • Corresponding test files for both

4. Core Engine Changes:

   • Modified Engine.ts to import and initialize StorageService
   • Updated types.ts to include StorageService in type definitions
   • Updated messenger index to register the new service

Risk Assessment - HIGH

Why High Risk:

1. Core Engine Modification: Any change to the Engine is critical as it's the backbone of MetaMask Mobile
2. Storage Infrastructure: This introduces a new storage layer that could affect data persistence across the entire app
3. FilesystemStorage Usage: Uses the same FilesystemStorage as redux-persist, introducing potential for conflicts or issues with existing persistence
4. Controller Data Storage: Designed to handle "large controller data" which could impact multiple controllers
5. New Dependency: Introduces a new package that will be fundamental to app operations

Potential Impact Areas:

• Core Wallet Functionality: Storage service could be used by any controller to persist state
• Account Management: Controllers managing accounts may use this service
• Data Persistence: Changes affect how data is stored/retrieved from filesystem
• App Initialization: New service initialization happens at Engine startup

Test Tag Selection Rationale

Selected Tags:

1. SmokeCore ✅

   • Reason: This is a fundamental change to the core Engine infrastructure
   • Impact: Core wallet initialization and controller management
   • Justification: Any Engine-level change should trigger core functionality tests

2. SmokeWalletPlatform ✅

   • Reason: Tests core wallet operations, accounts, and platform fundamentals
   • Impact: Storage service may be used by wallet-level controllers
   • Justification: Ensures the core platform still functions correctly with new storage infrastructure

3. SmokeAccounts ✅

   • Reason: Account-related controllers might leverage this storage service
   • Impact: Account data persistence and management
   • Justification: Ensures account operations work correctly with the new storage layer

Why Not Other Tags:

• The change is infrastructure-level and doesn't directly modify feature-specific code
• Other tags (Swaps, Stake, Card, etc.) are feature-specific and less likely to be immediately impacted
• However, if this is a critical integration, running ALL smoke tests would be the safest approach

Confidence: 85%

Why High Confidence:

• Clear understanding of the changes through code inspection
• Well-documented new service with comprehensive tests
• Impact scope is clear: core Engine infrastructure
• Changes follow established patterns in the codebase

Why Not 100%:

• Cannot determine which specific controllers will use this service without deeper investigation
• The local package dependency (file:./storage-service-package.tgz) might have additional impacts not visible in this PR
• Some uncertainty about the full downstream impact on feature-specific functionality

View GitHub Actions results