Claude/optimistic mayer w5n5 g - #3
Open
Sia-Sheerland wants to merge 13 commits into
Open
Conversation
Key changes: - Add VmAreaStruct (vm_area_struct) data structure and pool allocator - Add PageRefCount for per-page reference counting (COW support) - Rewrite MemoryDescriptor with VMA list, heap fields, per-page ops (MapPageDirect, GetPTE, UnmapPage, FindVma, AddVma, FreeAllVmas) - Rewrite page fault handler (Exception::PageFault) with three cases: A) first access: allocate page, zero-fill, load file-backed data B) swap-in: allocate, read from swap, free swap block C) copy-on-write: copy page if shared (refcount>1), else mark writable - Rewrite NewProc (fork) for COW: mark R/W pages R/O, share shadow PTEs, increment refcounts, clone VMA list, use SwtchUStruct without RetU - Rewrite Exec for demand paging: per-page allocation with refcount=1, MapPageDirect for each text/data/stack page, build VMA list describing all virtual regions (text/data/BSS/heap/stack), lazy heap allocation - Add section info arrays to PEParser for demand-page fault resolution - Update mm/Makefile to build new VmAreaStruct.o and PageRefCount.o https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
Exit(): call FreeUserPages + FreeAllVmas before Release() to free all individually-allocated data/stack pages; fix FreeMemory to release only the ppda (USIZE) since p_size is no longer the total process size. SStack(): replace old EstablishUserPageTable+Expand+CopySeg with single-page alloc + MapPageDirect + VMA extension; stack grows down one page per fault without disturbing existing mapped pages. SBreak(): replace old EstablishUserPageTable+Expand+CopySeg with pure VMA/m_HeapEnd update; new heap pages are lazily allocated on fault; shrink path releases pages via PageRefCount + UnmapPage. Also expose FreeUserPages as ProcessManager::FreeUserPages (public static) so Process.cpp can call it from Exit(). https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
Our demand-paging additions (~22 KB: VmAreaStruct pool, PageRefCount array, new handler code) push kernel.bin past the original 90 KB limit. The bootloader now loads up to 200 KB so the full kernel is mapped before jumping to 0xC0100000. https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
The original MapToPageTable() always mapped virtual 0x0 → physical frame 0, where main() copies the runtime() stub at boot. Our rewrite dropped this, so EIP=0x0 (set by Exec) faulted with no VMA. Restored the special hwPT[0][0] = frame 0 entry at the end of MapToPageTable(), and added a pageVA==0 early-return in the page-fault handler so the runtime page never triggers a spurious SIGSEGV. https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
Two bugs: 1. Physical RAM at 0x3FF000 (User struct/ppda) retains stale ProcessOpenFileTable entries across QEMU warm reboots, causing AllocFreeSlot() to return fd=1 instead of fd=0 on the second boot (triggering "STDIN Error!"). Fix: zero the entire ppda page in SetupProcessZero() on every boot. 2. MapPageDirect() updated the page table entry in memory but never called FlushPageDirectory(), leaving stale TLB entries. In COW (Case C of the page fault handler), after upgrading a page from R/O to R/W, the TLB still held the old R/O entry. The CPU retried the write, got another protection fault, and looped infinitely until kernel stack overflow → double fault → triple fault → CPU reset → reboot → STDIN Error. Fix: always FlushPageDirectory() at the end of MapPageDirect(). https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
Replace the full-page zero (which caused the initialization hang) with a targeted clear of u_ofiles.ProcessOpenFileTable[]. Only these 15 File* pointers need to be NULL on boot; zeroing all 4096 bytes of the ppda page was interfering with the initialization sequence. Root cause: stale ProcessOpenFileTable entries at physical 0x3FF000 (the ppda page) survive QEMU warm reboot. AllocFreeSlot() finds slot 0 non-NULL and returns fd=1 instead of 0, triggering "STDIN Error!" on the second and subsequent boots. https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
Two bugs fixed: 1. NewProc() missing child/parent detection after SaveU: When the child process was scheduled and resumed in NewProc() via RetU(), it re-executed the parent code path (creating spurious grandchild processes, corrupting page tables and u_procp). Added a check after SaveU(u.u_rsav): if u.u_procp == child (child's ppda is active) return 1 immediately so Fork() correctly identifies the child. Without this, the child corrupted kernel state causing EIP to land past .text end → kernel page fault. 2. COW page copy in PageFault handler corrupts kernel PTE ReadWriter: kpt[258].m_ReadWriter was set to 0 (R/O) for the copy source but never restored, leaving the kernel code page at 0xC0102000 marked read-only. Now saves and restores m_ReadWriter for both PTEs. https://claude.ai/code/session_01H8VezxP5dsHrc3QKUfvFpE
Exception::PageFault starts at VA 0xC0102FB4, which falls inside kpt[258]'s page (0xC0102000-0xC0102FFF). Its body (case-C COW logic) extends into kpt[259]'s page (0xC0103000-0xC0103FFF). The old COW copy borrowed exactly kpt[258] and kpt[259], remapped them to the old/new user physical pages, then called FlushPageDirectory(). After the TLB flush the CPU fetched the next instruction from VA 0xC0103xxx -- now mapped to the freshly-allocated zero-filled page -- executed 0x00 0x00 (add [eax],al) in a loop and crashed with "Kernel PageFault CR2=0xC0115 EIP=0xC0112xxx". Fix: use CopySeg() instead. CopySeg is at 0xC0111362 (kpt[273]) and borrows kpt[256]/[257] (VA 0xC0100000-0xC0101FFF), which are entirely outside PageFault's code range. PageFault continues executing from its own pages unaffected while CopySeg does the copy.
After CopySeg copies the parent's ppda to the child's ppda, the child's u_procp still points to the parent process struct. The child-detection check `if (u.u_procp == child)` at the top of NewProc could therefore never be true: the child would fall through, re-execute the parent-side NewProc logic (allocating phantom shadow pages and ppda), and eventually return 0 to Fork() — causing fork() to return the grandchild's PID to the child instead of 0. The child then called wait() instead of execv(), and the kernel state degraded after each command until SIGSEGV. Fix: add `cu.u_procp = child` to the SwtchUStruct(child) ppda-setup block so the child-detection check fires correctly on the child's first schedule, returning 1 to Fork() and giving fork() return value 0. https://claude.ai/code/session_01H8VezxP5dsHrc3QKUfvFpE
In NewProc(), Clone() copies p_textp to the child but never increments x_count/x_ccount. When the child calls Exec() -> XFree(), x_count drops to 0 and the physical text pages are freed — even though the parent shell still has them mapped. This caused the "first command works, second no response, third freeze" degradation. Fix: increment x_count and x_ccount for the child immediately after Clone(). https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
…rement FreeUserPages() skips text frames only when p_textp is non-NULL. Previously XFree() was called first, setting p_textp=NULL, so FreeUserPages() would find text frames in the shadow page table, decrement their refcount (incremented to 1 by NewProc::Inc), hit 0, and call FreeMemory — freeing the parent's text pages while the parent was still executing from them. Fix: call FreeUserPages()+FreeAllVmas() before XFree() so p_textp is still set and text frames are correctly skipped. XFree then handles releasing the text segment reference via x_count/x_ccount. https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
Documents the demand paging implementation, COW fork, and all bugs fixed during development. https://claude.ai/code/session_018UqurieEFK6k8JYX72eTv7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.