|  | 
| 6 | 6 | 
 | 
| 7 | 7 | This repository provides automated builds of Microsoft's official WSL2 kernel sources, compiling and releasing only the bzImage kernel files. | 
| 8 | 8 | 
 | 
| 9 |  | -## Why This Exists ❓ | 
| 10 |  | - | 
| 11 |  | -Microsoft maintains the [WSL2-Linux-Kernel](https://github.com/microsoft/WSL2-Linux-Kernel) but: | 
| 12 |  | -- Doesn't provide pre-built historical kernel versions | 
| 13 |  | -- Only distributes kernels through Windows Update | 
| 14 |  | -- Makes it difficult to test different kernel versions | 
| 15 |  | - | 
| 16 |  | -I encountered issues while fine-tuning models using **Unsloth**, and I suspect they are related to the WSL2 kernel. To experiment with different WSL kernels, I created this repository. | 
| 17 |  | - | 
| 18 |  | -The specific Unsloth issue can be found here: https://github.com/unslothai/unsloth/issues/1744   | 
| 19 |  | -Problems I discovered include: | 
| 20 |  | - | 
| 21 |  | -- Some WSL kernel versions have serious memory management issues: | 
| 22 |  | -  - Memory is requested but cannot be released, even when `pageReporting=true` is enabled. | 
| 23 |  | -  - `memlock` limits vary—some WSL instances have a higher or even unlimited locked memory cap, but typically the limit is around 2GB. In my tests, it sometimes allows only 2GB, which is entirely dependent on the WSL kernel. | 
| 24 |  | -    - `memlock` has a hard limit that seems unchangeable within WSL. | 
| 25 |  | -    - `memlock` appears to overestimate current locked memory; repeated locking of the same memory can be counted multiple times (as documented in [WSL's infiniband/user_verbs.rst](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning)). | 
| 26 |  | -- Some WSL kernels are incompatible with Docker Desktop. | 
| 27 |  | - | 
| 28 |  | -## Memory Management in WSL Kernels | 
| 29 |  | - | 
| 30 |  | -The WSL2 kernel has specific behaviors around memory allocation and locking: | 
| 31 |  | - | 
| 32 |  | -1. **Performance Monitoring Memory**:  | 
| 33 |  | -   - The `perf_event_mlock_kb` setting governs memory available for performance monitoring | 
| 34 |  | -   - This extends the `RLIMIT_MEMLOCK` limit but only for perf_event mmap buffers | 
| 35 |  | -   - Processes with `CAP_IPC_LOCK` capability bypass these limits | 
| 36 |  | -   - [See documentation](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/admin-guide/perf-security.rst#memory-allocation) | 
| 37 |  | - | 
| 38 |  | -2. **Memory Pinning**: | 
| 39 |  | -   - Direct I/O requires memory regions to stay at fixed physical addresses | 
| 40 |  | -   - The kernel accounts pinned memory in `pinned_vm` | 
| 41 |  | -   - Pages pinned multiple times are counted each time (may overestimate actual usage) | 
| 42 |  | -   - [Documentation reference](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning) | 
| 43 |  | - | 
| 44 |  | -These behaviors explain some of the memory management challenges encountered when running memory-intensive workloads in WSL2. | 
| 45 |  | - | 
| 46 | 9 | ## Features ✨ | 
| 47 | 10 | 
 | 
| 48 | 11 | - Automates building official Microsoft kernel sources exactly as released | 
| @@ -90,9 +53,42 @@ Example: | 
| 90 | 53 | ./scripts/build.sh 5.15.90.1 | 
| 91 | 54 | ``` | 
| 92 | 55 | 
 | 
| 93 |  | -## Supported Versions 📦 | 
|  | 56 | +## Why This Exists ❓ | 
|  | 57 | + | 
|  | 58 | +Microsoft maintains the [WSL2-Linux-Kernel](https://github.com/microsoft/WSL2-Linux-Kernel) but: | 
|  | 59 | +- Doesn't provide pre-built historical kernel versions | 
|  | 60 | +- Only distributes kernels through Windows Update | 
|  | 61 | +- Makes it difficult to test different kernel versions | 
|  | 62 | + | 
|  | 63 | +I encountered issues while fine-tuning models using **Unsloth**, and I suspect they are related to the WSL2 kernel. To experiment with different WSL kernels, I created this repository. | 
|  | 64 | + | 
|  | 65 | +The specific Unsloth issue can be found here: https://github.com/unslothai/unsloth/issues/1744   | 
|  | 66 | +Problems I discovered include: | 
|  | 67 | + | 
|  | 68 | +- Some WSL kernel versions have serious memory management issues: | 
|  | 69 | +  - Memory is requested but cannot be released, even when `pageReporting=true` is enabled. | 
|  | 70 | +  - `memlock` limits vary—some WSL instances have a higher or even unlimited locked memory cap, but typically the limit is around 2GB. In my tests, it sometimes allows only 2GB, which is entirely dependent on the WSL kernel. | 
|  | 71 | +    - `memlock` has a hard limit that seems unchangeable within WSL. | 
|  | 72 | +    - `memlock` appears to overestimate current locked memory; repeated locking of the same memory can be counted multiple times (as documented in [WSL's infiniband/user_verbs.rst](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning)). | 
|  | 73 | +- Some WSL kernels are incompatible with Docker Desktop. | 
|  | 74 | + | 
|  | 75 | +## Memory Management in WSL Kernels | 
|  | 76 | + | 
|  | 77 | +The WSL2 kernel has specific behaviors around memory allocation and locking: | 
|  | 78 | + | 
|  | 79 | +1. **Performance Monitoring Memory**:  | 
|  | 80 | +   - The `perf_event_mlock_kb` setting governs memory available for performance monitoring | 
|  | 81 | +   - This extends the `RLIMIT_MEMLOCK` limit but only for perf_event mmap buffers | 
|  | 82 | +   - Processes with `CAP_IPC_LOCK` capability bypass these limits | 
|  | 83 | +   - [See documentation](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/admin-guide/perf-security.rst#memory-allocation) | 
|  | 84 | + | 
|  | 85 | +2. **Memory Pinning**: | 
|  | 86 | +   - Direct I/O requires memory regions to stay at fixed physical addresses | 
|  | 87 | +   - The kernel accounts pinned memory in `pinned_vm` | 
|  | 88 | +   - Pages pinned multiple times are counted each time (may overestimate actual usage) | 
|  | 89 | +   - [Documentation reference](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning) | 
| 94 | 90 | 
 | 
| 95 |  | -All versions from Microsoft's [WSL2-Linux-Kernel](https://github.com/microsoft/WSL2-Linux-Kernel/tags) can be built. | 
|  | 91 | +These behaviors explain some of the memory management challenges encountered when running memory-intensive workloads in WSL2. | 
| 96 | 92 | 
 | 
| 97 | 93 | ## License 📄 | 
| 98 | 94 | 
 | 
|  | 
0 commit comments