Implement file descriptor management and pipes #96
Description
Goal
Implement comprehensive file descriptor management including dup/dup2 operations and pipe/pipe2 for inter-process communication.
Context
File descriptor management and pipes are fundamental for Unix-like systems, enabling I/O redirection, process communication, and complex shell operations. Essential for full shell functionality and application support.
Definition of Done
- dup/dup2 syscalls: Duplicate file descriptors with proper semantics
- pipe/pipe2 syscalls: Create bidirectional communication pipes
- FD table management: Robust file descriptor table implementation
- Close-on-exec: FD_CLOEXEC flag support for exec operations
- Resource limits: RLIMIT_NOFILE enforcement and management
- Error handling: Comprehensive error handling and edge cases
- Performance: Efficient FD operations with minimal overhead
File Descriptor Management
FD Table Implementation
- Per-process file descriptor table
- Dynamic FD table growth and shrinkage
- FD allocation and deallocation strategies
- FD inheritance across fork/exec
- Close-on-exec flag handling
dup/dup2 Implementation
- File descriptor duplication with shared file descriptions
- Reference counting for underlying file objects
- FD_CLOEXEC flag inheritance and management
- Error handling for invalid FDs
- Atomic operations for dup2
Resource Management
- FD limit enforcement (RLIMIT_NOFILE)
- FD leak detection and prevention
- Efficient FD table operations
- Memory usage optimization
- Garbage collection of unused FDs
Pipe Implementation
Pipe Data Structure
- Circular buffer for pipe data
- Read/write position tracking
- Blocking/non-blocking I/O support
- Pipe capacity management
- Buffer allocation and growth
Pipe Operations
- pipe() syscall creating read/write FD pair
- pipe2() with flags support (O_CLOEXEC, O_NONBLOCK)
- Read/write operations with proper blocking
- EOF handling when write end closes
- SIGPIPE generation for broken pipes
Synchronization
- Reader/writer synchronization
- Multiple reader/writer support
- Select/poll integration (future)
- Signal handling for pipe operations
- Atomic operations for pipe state
System Call Interface
int dup(int oldfd);
int dup2(int oldfd, int newfd);
int dup3(int oldfd, int newfd, int flags);
int pipe(int pipefd[2]);
int pipe2(int pipefd[2], int flags);
int fcntl(int fd, int cmd, ...);Advanced Features
fcntl Implementation
- F_DUPFD and F_DUPFD_CLOEXEC
- F_GETFD and F_SETFD for FD flags
- F_GETFL and F_SETFL for file status flags
- F_SETLK and F_SETLKW for file locking (future)
- F_GETOWN and F_SETOWN for signal delivery
Named Pipes (FIFOs)
- FIFO creation via mkfifo()
- Filesystem integration for named pipes
- Permission and access control
- Blocking behavior for open operations
- Multiple reader/writer support
Advanced Pipe Features
- Pipe capacity querying and setting
- Non-blocking I/O support
- Scatter-gather I/O operations
- Pipe monitoring and statistics
- Memory-mapped pipe optimization
Implementation Strategy
Phase 1: Basic FD Management
- Core FD table implementation
- Basic dup/dup2 functionality
- FD allocation and cleanup
- Integration with existing file operations
Phase 2: Pipe Implementation
- Basic pipe creation and operations
- Reader/writer synchronization
- Blocking I/O implementation
- Error handling and edge cases
Phase 3: Advanced Features
- fcntl implementation
- Non-blocking I/O support
- Close-on-exec handling
- Performance optimization
Phase 4: Integration and Polish
- Shell integration testing
- Application compatibility
- Performance tuning
- Comprehensive testing
Files to Create/Modify
- syscall/dup.c - dup family system calls
- syscall/pipe.c - pipe system calls
- syscall/fcntl.c - fcntl implementation
- src/kernel/fd_table.c - FD table management
- src/kernel/pipe.c - Pipe implementation
- include/unistd.h - POSIX interface definitions
Technical Implementation
File Descriptor Table
- Array-based or tree-based FD storage
- Efficient FD allocation (bitmap or free list)
- Reference counting for file objects
- Lock-free operations where possible
- Memory-efficient representation
Pipe Buffer Management
- Ring buffer implementation
- Dynamic buffer sizing
- Memory allocation strategies
- Copy-on-write optimization (future)
- Zero-copy operations (future)
Synchronization Design
- Per-pipe locks for concurrent access
- Reader/writer wait queues
- Signal handling during blocking operations
- Deadlock prevention mechanisms
- SMP-safe operations
Testing Strategy
Functionality Testing
- Basic dup/dup2 operations
- Pipe creation and communication
- FD inheritance across fork/exec
- Error condition handling
- Resource limit enforcement
Integration Testing
- Shell redirection and pipes
- Multi-process communication
- Complex FD manipulation scenarios
- Application compatibility testing
- System call interaction validation
Performance Testing
- FD operation latency
- Pipe throughput and latency
- Concurrent access performance
- Memory usage optimization
- Scalability with FD count
Performance Goals
- dup/dup2 latency < 1μs
- Pipe read/write latency < 10μs for small data
- Pipe throughput > 1GB/s for large transfers
- FD table operations O(1) for common cases
- Memory overhead < 64 bytes per FD
Security Considerations
- FD permission validation
- Pipe access control
- Resource exhaustion protection
- Information leak prevention
- Privilege escalation prevention
Dependencies
- File system infrastructure
- Process management (for FD inheritance)
- Memory management for pipe buffers
- Signal handling system
- Virtual file system (VFS) layer
Related Issues
- Enables advanced shell features (Issue Evolve mosh shell to be the default userland shell for meniOS #54)
- Required for process communication
- Foundation for I/O redirection
- Critical for Unix-like application support
- Enables build systems and complex applications
Use Cases
- Shell I/O redirection (cmd > file, cmd < file)
- Shell pipes (cmd1 | cmd2)
- Process communication via pipes
- File descriptor passing
- Complex multi-process applications
- Build systems and development tools