Skip to content

Commit 3b76304

Browse files
committed
docs: update readme
1 parent 150088a commit 3b76304

File tree

1 file changed

+35
-39
lines changed

1 file changed

+35
-39
lines changed

README.md

Lines changed: 35 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -6,43 +6,6 @@
66

77
This repository provides automated builds of Microsoft's official WSL2 kernel sources, compiling and releasing only the bzImage kernel files.
88

9-
## Why This Exists ❓
10-
11-
Microsoft maintains the [WSL2-Linux-Kernel](https://github.com/microsoft/WSL2-Linux-Kernel) but:
12-
- Doesn't provide pre-built historical kernel versions
13-
- Only distributes kernels through Windows Update
14-
- Makes it difficult to test different kernel versions
15-
16-
I encountered issues while fine-tuning models using **Unsloth**, and I suspect they are related to the WSL2 kernel. To experiment with different WSL kernels, I created this repository.
17-
18-
The specific Unsloth issue can be found here: https://github.com/unslothai/unsloth/issues/1744
19-
Problems I discovered include:
20-
21-
- Some WSL kernel versions have serious memory management issues:
22-
- Memory is requested but cannot be released, even when `pageReporting=true` is enabled.
23-
- `memlock` limits vary—some WSL instances have a higher or even unlimited locked memory cap, but typically the limit is around 2GB. In my tests, it sometimes allows only 2GB, which is entirely dependent on the WSL kernel.
24-
- `memlock` has a hard limit that seems unchangeable within WSL.
25-
- `memlock` appears to overestimate current locked memory; repeated locking of the same memory can be counted multiple times (as documented in [WSL's infiniband/user_verbs.rst](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning)).
26-
- Some WSL kernels are incompatible with Docker Desktop.
27-
28-
## Memory Management in WSL Kernels
29-
30-
The WSL2 kernel has specific behaviors around memory allocation and locking:
31-
32-
1. **Performance Monitoring Memory**:
33-
- The `perf_event_mlock_kb` setting governs memory available for performance monitoring
34-
- This extends the `RLIMIT_MEMLOCK` limit but only for perf_event mmap buffers
35-
- Processes with `CAP_IPC_LOCK` capability bypass these limits
36-
- [See documentation](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/admin-guide/perf-security.rst#memory-allocation)
37-
38-
2. **Memory Pinning**:
39-
- Direct I/O requires memory regions to stay at fixed physical addresses
40-
- The kernel accounts pinned memory in `pinned_vm`
41-
- Pages pinned multiple times are counted each time (may overestimate actual usage)
42-
- [Documentation reference](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning)
43-
44-
These behaviors explain some of the memory management challenges encountered when running memory-intensive workloads in WSL2.
45-
469
## Features ✨
4710

4811
- Automates building official Microsoft kernel sources exactly as released
@@ -90,9 +53,42 @@ Example:
9053
./scripts/build.sh 5.15.90.1
9154
```
9255

93-
## Supported Versions 📦
56+
## Why This Exists ❓
57+
58+
Microsoft maintains the [WSL2-Linux-Kernel](https://github.com/microsoft/WSL2-Linux-Kernel) but:
59+
- Doesn't provide pre-built historical kernel versions
60+
- Only distributes kernels through Windows Update
61+
- Makes it difficult to test different kernel versions
62+
63+
I encountered issues while fine-tuning models using **Unsloth**, and I suspect they are related to the WSL2 kernel. To experiment with different WSL kernels, I created this repository.
64+
65+
The specific Unsloth issue can be found here: https://github.com/unslothai/unsloth/issues/1744
66+
Problems I discovered include:
67+
68+
- Some WSL kernel versions have serious memory management issues:
69+
- Memory is requested but cannot be released, even when `pageReporting=true` is enabled.
70+
- `memlock` limits vary—some WSL instances have a higher or even unlimited locked memory cap, but typically the limit is around 2GB. In my tests, it sometimes allows only 2GB, which is entirely dependent on the WSL kernel.
71+
- `memlock` has a hard limit that seems unchangeable within WSL.
72+
- `memlock` appears to overestimate current locked memory; repeated locking of the same memory can be counted multiple times (as documented in [WSL's infiniband/user_verbs.rst](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning)).
73+
- Some WSL kernels are incompatible with Docker Desktop.
74+
75+
## Memory Management in WSL Kernels
76+
77+
The WSL2 kernel has specific behaviors around memory allocation and locking:
78+
79+
1. **Performance Monitoring Memory**:
80+
- The `perf_event_mlock_kb` setting governs memory available for performance monitoring
81+
- This extends the `RLIMIT_MEMLOCK` limit but only for perf_event mmap buffers
82+
- Processes with `CAP_IPC_LOCK` capability bypass these limits
83+
- [See documentation](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/admin-guide/perf-security.rst#memory-allocation)
84+
85+
2. **Memory Pinning**:
86+
- Direct I/O requires memory regions to stay at fixed physical addresses
87+
- The kernel accounts pinned memory in `pinned_vm`
88+
- Pages pinned multiple times are counted each time (may overestimate actual usage)
89+
- [Documentation reference](https://github.com/microsoft/WSL2-Linux-Kernel/blob/main/Documentation/infiniband/user_verbs.rst#memory-pinning)
9490

95-
All versions from Microsoft's [WSL2-Linux-Kernel](https://github.com/microsoft/WSL2-Linux-Kernel/tags) can be built.
91+
These behaviors explain some of the memory management challenges encountered when running memory-intensive workloads in WSL2.
9692

9793
## License 📄
9894

0 commit comments

Comments
 (0)