docs: Add FreeRTOS implementation summary.

Document covering the journey from master branch to FreeRTOS SMP,
including architecture, service task framework API, and usage notes.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>

Author: pi-anl

FREERTOS_IMPL_SUMMARY.md

@@ -0,0 +1,338 @@
+# MicroPython FreeRTOS Threading Backend: Implementation Summary
+
+**Date:** December 2025
+**Branch:** `freertos`
+**Status:** Functional on RP2 (Pico W)
+
+---
+
+## Executive Summary
+
+This document summarizes the implementation of a universal FreeRTOS threading backend for MicroPython, currently integrated with the RP2 (Raspberry Pi Pico) port. The implementation replaces the limited Pico SDK multicore approach (2 threads max, one per core) with FreeRTOS SMP, enabling:
+
+- Unlimited Python threads (limited only by RAM)
+- True parallel execution on both RP2040 cores
+- Background service tasks for USB, WiFi, and soft timers
+- A reusable threading backend designed for other ports (STM32, mimxrt, etc.)
+
+---
+
+## Motivation: Why FreeRTOS?
+
+### Master Branch Limitations
+
+The upstream RP2 port uses Pico SDK's multicore API:
+- Limited to exactly 2 threads (one per core)
+- GIL disabled (`MICROPY_PY_THREAD_GIL=0`)
+- No priority-based preemption
+- USB/WiFi compete with Python for CPU time
+
+### FreeRTOS Benefits
+
+- **N threads**: Not limited by core count
+- **SMP scheduling**: FreeRTOS distributes work across cores
+- **Priority preemption**: USB/WiFi run at higher priority, remain responsive
+- **Industry standard**: Well-documented, actively maintained
+- **Portable**: Same backend works on STM32, mimxrt, nRF, etc.
+
+---
+
+## Journey from Master
+
+### Phase 1: Scaffolding and Core Backend
+
+Created `extmod/freertos/` with shared infrastructure:
+
+| File | Purpose |
+|------|---------|
+| `mpthreadport.h` | Thread API definitions, data structures |
+| `mpthreadport.c` | Thread creation, GC integration, mutex implementation |
+| `mp_freertos_hal.h/c` | HAL functions (delay, ticks, yield) |
+| `mp_freertos_service.h/c` | Service task framework for deferred callbacks |
+
+### Phase 2: QEMU Validation
+
+Built and tested on QEMU ARM (MPS2-AN385) to validate the core backend without hardware. All 32 thread tests pass (1 skip: `disable_irq.py`).
+
+### Phase 3: STM32 Integration
+
+Integrated with STM32 port (NUCLEO-F429ZI). Validated threading, GC scanning, and stress tests on real hardware.
+
+### Phase 4: RP2 Integration
+
+Major restructuring of the RP2 port:
+
+1. **Replaced Pico SDK multicore with FreeRTOS SMP**
+   - FreeRTOS-Kernel-SMP from Pico SDK
+   - Hardware spinlocks 0 and 1 reserved for FreeRTOS
+   - `configNUMBER_OF_CORES = 2`
+
+2. **Resolved PendSV Handler Conflict**
+   - Both MicroPython's soft timers and FreeRTOS use PendSV
+   - Solution: Assembly wrapper that chains both handlers
+   - Uses `configUSE_DYNAMIC_EXCEPTION_HANDLERS`
+
+3. **Replaced PendSV Dispatch with Service Task**
+   - Soft timers and scheduled callbacks now run in a high-priority FreeRTOS task
+   - Avoids interrupt-level conflicts
+
+4. **WiFi Investigation and Fix**
+   - Initial builds hung during WiFi scan
+   - Root cause: `cyw43_yield()` blocked forever waiting for events
+   - Fix: Changed yield behavior for FreeRTOS SMP context
+
+### Phase 5: True SMP Investigation
+
+Initial FreeRTOS builds pinned all threads to core 0 for safety. Investigation into enabling true dual-core Python execution:
+
+| Configuration | Result |
+|--------------|--------|
+| GIL=1, core 0 only | Works, but no parallelism |
+| GIL=1, tskNO_AFFINITY | Works, threads migrate but still serialize |
+| GIL=0, tskNO_AFFINITY | Works with caveats (see below) |
+
+**Key Finding:** Master's GIL=0 implementation has the same race condition vulnerability. The official thread tests require 4 threads which master can't run, hiding the issue.
+
+**Final Configuration:** GIL=0 with `tskNO_AFFINITY` - matches master's semantics but with N threads instead of 2.
+
+---
+
+## Architecture Overview
+
+### Thread Model
+
+```
+┌──────────────────────────────────────────────────────────────────┐
+│                     FreeRTOS SMP Scheduler                        │
+│                   (runs on both RP2040 cores)                     │
+├──────────────────────────────────────────────────────────────────┤
+│  Priority 3: Service Task (soft timers, scheduled callbacks)     │
+├──────────────────────────────────────────────────────────────────┤
+│  Priority 1: Main MicroPython Task                               │
+│  Priority 1: Python Thread 1                                     │
+│  Priority 1: Python Thread 2                                     │
+│  Priority 1: ... (unlimited threads)                             │
+├──────────────────────────────────────────────────────────────────┤
+│  Priority 0: FreeRTOS Idle Task                                  │
+└──────────────────────────────────────────────────────────────────┘
+```
+
+### Memory Management
+
+All thread memory is GC-allocated using static FreeRTOS APIs:
+
+```c
+// Thread creation (mp_thread_create)
+tcb = m_new(StaticTask_t, 1);           // GC-allocated TCB
+stack = m_new(StackType_t, stack_len);  // GC-allocated stack
+xTaskCreateStatic(...);                  // FreeRTOS uses our memory
+```
+
+**Benefits:**
+- Thread stacks are scanned by GC (no dangling references)
+- No separate FreeRTOS heap needed
+- Memory automatically reclaimed when threads finish
+
+### Thread Cleanup (Reaper)
+
+A thread cannot free its own stack while running. The "reaper" mechanism cleans up finished threads:
+
+1. Thread marks itself `FINISHED` and yields forever
+2. Next `mp_thread_create()` calls reaper
+3. Reaper iterates thread list, frees `FINISHED` threads
+4. Memory returned to GC heap
+
+---
+
+## Service Task Framework
+
+### Purpose
+
+Replaces PendSV-based deferred execution (soft timers, scheduled callbacks) with a FreeRTOS task. This avoids conflicts with FreeRTOS's own PendSV usage.
+
+### API
+
+```c
+// Initialize (called once during startup)
+void mp_freertos_service_init(void);
+
+// Schedule a callback to run in service task context
+// slot: Dispatch slot index (0 to MICROPY_FREERTOS_SERVICE_MAX_SLOTS-1)
+// callback: Function to call (runs at high priority)
+void mp_freertos_service_schedule(size_t slot, mp_freertos_dispatch_t callback);
+
+// Suspend/resume dispatch processing (for critical sections)
+void mp_freertos_service_suspend(void);
+void mp_freertos_service_resume(void);
+```
+
+### How C Modules Use Service Tasks
+
+C modules that need deferred callback execution should:
+
+1. **Define a dispatch slot** in `mpconfigport.h`:
+   ```c
+   #define MP_FREERTOS_SLOT_SOFT_TIMER   0
+   #define MP_FREERTOS_SLOT_PENDSV       1
+   #define MP_FREERTOS_SLOT_MY_DRIVER    2
+   ```
+
+2. **Schedule work** from ISR or task context:
+   ```c
+   void my_driver_irq_handler(void) {
+       // Set up data for callback...
+       mp_freertos_service_schedule(MP_FREERTOS_SLOT_MY_DRIVER, my_deferred_handler);
+   }
+   ```
+
+3. **Implement the callback**:
+   ```c
+   static void my_deferred_handler(void) {
+       // Runs at high priority, but in task context (can call FreeRTOS APIs)
+       // Process deferred work here
+   }
+   ```
+
+### ISR Context Detection
+
+Ports must provide `mp_freertos_service_in_isr()` to detect interrupt context:
+
+```c
+// Example for Cortex-M (in mphalport.c or similar)
+bool mp_freertos_service_in_isr(void) {
+    uint32_t ipsr;
+    __asm volatile ("mrs %0, ipsr" : "=r" (ipsr));
+    return ipsr != 0;
+}
+```
+
+This allows `mp_freertos_service_schedule()` to use the correct FreeRTOS API (`FromISR` variants in ISR context).
+
+---
+
+## Current Configuration (RP2)
+
+### Key Settings
+
+| Setting | Value | File |
+|---------|-------|------|
+| `MICROPY_PY_THREAD` | 1 | mpconfigport.h |
+| `MICROPY_PY_THREAD_GIL` | 0 | mpconfigport.h |
+| `MP_THREAD_CORE_AFFINITY` | tskNO_AFFINITY | mpthreadport.h |
+| `configNUMBER_OF_CORES` | 2 | FreeRTOSConfig.h |
+| FreeRTOS heap | 8KB | FreeRTOSConfig.h |
+| Main task stack | 8KB | main.c |
+
+### Threading Semantics
+
+With GIL=0 and true SMP:
+
+- **Multiple Python threads execute in parallel on both cores**
+- **Mutable objects (dict, list, set, bytearray) are NOT thread-safe**
+- **Users MUST protect shared mutable objects with locks**
+
+Check `sys.implementation._thread` at runtime:
+- `"GIL"` - GIL enabled, mutable objects implicitly protected
+- `"unsafe"` - GIL disabled, explicit locking required
+
+---
+
+## Usage Examples
+
+### Safe Multi-threaded Access
+
+```python
+import _thread
+import time
+
+shared_data = {}
+lock = _thread.allocate_lock()
+
+def worker(name, count):
+    for i in range(count):
+        with lock:  # REQUIRED for GIL=0
+            shared_data[f"{name}_{i}"] = i
+        time.sleep(0.001)
+
+# Start threads on potentially different cores
+_thread.start_new_thread(worker, ("A", 100))
+_thread.start_new_thread(worker, ("B", 100))
+```
+
+### Checking Thread Safety Mode
+
+```python
+import sys
+
+if hasattr(sys.implementation, '_thread'):
+    if sys.implementation._thread == "unsafe":
+        print("GIL disabled - use locks for shared objects")
+    else:
+        print("GIL enabled - implicit protection")
+else:
+    print("Threading not available")
+```
+
+---
+
+## Files Modified/Created
+
+### New Files (extmod/freertos/)
+
+| File | Lines | Purpose |
+|------|-------|---------|
+| mpthreadport.h | 136 | Thread API, data structures |
+| mpthreadport.c | ~400 | Thread lifecycle, GC integration |
+| mp_freertos_hal.h | 85 | HAL API declarations |
+| mp_freertos_hal.c | 125 | Delay, ticks, yield implementation |
+| mp_freertos_service.h | 129 | Service task API |
+| mp_freertos_service.c | 208 | Service task implementation |
+
+### RP2 Port Modifications
+
+| File | Changes |
+|------|---------|
+| CMakeLists.txt | FreeRTOS integration, conditional compilation |
+| FreeRTOSConfig.h | SMP configuration, tick rate, priorities |
+| freertos_hooks.c | Static allocation callbacks, stack overflow hook |
+| main.c | FreeRTOS task creation, scheduler startup |
+| mpconfigport.h | Threading config, backend selection |
+| mpthreadport_rp2.c | Atomic sections (FreeRTOS critical + IRQ disable) |
+| pendsv.c | PendSV wrapper for FreeRTOS coexistence |
+| mphalport.c | ISR detection, FreeRTOS-aware delays |
+
+---
+
+## Test Status
+
+### Passing
+
+- All 32 standard thread tests (on QEMU and hardware)
+- WiFi scan and connect (Pico W)
+- Soft timers via service task
+- GC under threading load
+- Concurrent dict access with explicit locks
+
+### Known Limitations
+
+- `mutate_dict` tests fail without locks (expected with GIL=0)
+- `disable_irq.py` skipped (incompatible with FreeRTOS)
+
+---
+
+## Future Work
+
+1. **Service framework for USB/WiFi** - Move TinyUSB and CYW43 to dedicated high-priority tasks
+2. **STM32 port completion** - Finalize FreeRTOS integration for STM32F4/F7
+3. **mimxrt/nRF ports** - Apply same pattern to other FreeRTOS-capable ports
+4. **Thread pool API** - Higher-level threading primitives
+5. **Debug instrumentation removal** - Remove temporary debug counters from service task
+
+---
+
+## References
+
+- `FREERTOS_THREADING_REQUIREMENTS.md` - Technical specification (v1.5)
+- `FREERTOS_IMPLEMENTATION_PLAN.md` - Phased implementation plan
+- `ports/rp2/FREERTOS_STATUS.md` - RP2-specific integration notes
+- FreeRTOS SMP Documentation: https://www.freertos.org/symmetric-multiprocessing-introduction.html