design_eng

MMKV Design

Memory Preparation

By using mmap to memory-map a file, MMKV provides a memory block that can be written to at any time. The application writes data directly into this memory block, while the operating system handles writing the data back to the file. This ensures data integrity even if the app crashes.

Data Organization

For data serialization, we chose the Protocol Buffers (protobuf) protocol, which offers excellent performance and space efficiency. Since MMKV is a general-purpose key-value (KV) component, keys are restricted to strings, while values can be of various types (int, bool, double, etc.). To generalize value handling, we serialize values into a unified memory buffer using protobuf, then serialize the entire KV object into memory.

message KV {
    string key = 1;
    bytes value = 2;
}

Example usage:

- (BOOL)setInt32:(int32_t)value forKey:(NSString *)key {
    NSData *data = PBEncode(value);
    return [self setData:data forKey:key];
}

- (BOOL)setData:(NSData *)data forKey:(NSString *)key {
    KV kv = { key, data };
    NSData *buf = PBEncode(kv);
    return [self write:buf];
}

Write Optimization

Standard protobuf does not support incremental updates, requiring full writes for every modification. To address frequent write-update scenarios, MMKV implements incremental updates: new KV objects are serialized and appended directly to the end of the memory block. This means multiple versions of the same key may exist, with the latest value always at the end. During initialization, MMKV scans all entries and retains only the last occurrence of each key, ensuring data validity.

Space Management

The append-only approach introduces uncontrolled file growth (e.g., repeated updates to a single key could bloat the file to hundreds of MB or even GB, despite needing only ~1 KB). To balance performance and space, MMKV:

1. Allocates space in memory page size units.
2. Operates in append mode until space is exhausted.
3. Triggers file defragmentation (deduplication) when appending reaches the end.
4. If space remains insufficient after deduplication, doubles the file size until adequate.

Pseudocode:

- (BOOL)append:(NSData *)data {
    if (space >= data.length) {
        append(fd, data);
    } else {
        NSData *newData = deduplicate(allKV);
        if (total_space >= newData.length) {
            write(fd, newData);
        } else {
            while (total_space < newData.length) {
                total_space *= 2;
            }
            ftruncate(fd, total_space);
            write(fd, newData);
        }
    }
}

Data Validity

To guard against file system or OS instability, MMKV adds CRC checksum validation to detect corrupted data. In production environments (e.g., iOS WeChat), we observed approximately 700,000 daily instances of data validation failures.

Android Multi-Process Access

After porting MMKV to Android, many users requested multi-process support—a feature not originally considered (as iOS lacks multi-process support). This required comprehensive design and meticulous implementation. For details, see Android Multi-Process Design and Implementation.