A neural network implementation built entirely from scratch in C for playing Connect Four, trained using reinforcement learning through self-play.
This project implements a complete neural network from the ground up in C, without using any external machine learning libraries. The network learns to play Connect Four through reinforcement learning, achieving an ~85% win rate against a random agent after training on 1 million games.
- From-scratch implementation: Neural network built entirely in C with manual implementation of:
- Forward propagation
- Backpropagation algorithm
- Custom activation functions
- Custom loss functions
- Reinforcement learning: Self-play training against random agents
- Memory management: Manual memory allocation and optimization
- Matrix operations: Optimized matrix computations for faster training
- Serialization system: Save and resume training progress from files
- Performance tracking: Monitor win rates and training progress
- Input Layer: Connect Four board state representation (6x7 grid = 42 neurons)
- Hidden Layers:
- Output Layer: Move evaluation (7 neurons for each column)
- Algorithm: Reinforcement Learning with self-play
- Opponent: Random agent for training
- Training Games: 1 million games
- Performance: ~85% win rate achieved
- GCC compiler
- Make (optional)
- Standard C library
# Clone the repository
git clone https://github.com/marcomit/connected-four-nn.git
cd connected-four-nn
# Compile the project
gcc -o connect_four *.c -lm
# Or if using Makefile
make# It creates rl file, if you want to train the model from scratch delete the rl file
./connect_four| Training Games | Win Rate vs Random | Training Time |
|---|---|---|
| 100,000 | ~65% | ~1 minutes |
| 500,000 | ~78% | ~2 minutes |
| 1,000,000 | ~85% | ~5 minutes |
- Activation Functions: ReLU
- Loss Function: entrophy
- Manual memory management for all matrices and vectors
- Optimized matrix multiplication routines
- Efficient board state representation
- Policy: epsilon-greedy
- Reward System: +1 win, -1 loss, 0 parity
- Experience Collection: Self-play game generation
- Manual Backpropagation: Implemented gradient calculation and weight updates without frameworks
- Memory Management: Efficient allocation/deallocation for large training datasets
- Numerical Stability: Handling floating-point precision in deep calculations
- Training Convergence: Balancing exploration vs exploitation in self-play
- Performance Optimization: Fast matrix operations for real-time gameplay
- Implement more sophisticated activation functions
- Add support for different network architectures
- Implement Monte Carlo Tree Search (MCTS)
- Add GUI for human vs AI gameplay
- Optimize training with parallel processing
- Implement tournament evaluation against different strategies
Contributions are welcome! Please feel free to submit a Pull Request. Areas for contribution:
- Performance optimizations
- Additional activation functions
- GUI implementation
- Documentation improvements
This project is open source and available under the BSD License.
Marco Menegazzi
- GitHub: @marcomit
- Email: marcomenegazzi.05@gmail.com
- Inspired by reinforcement learning research in game AI
- Connect Four game rules and strategy analysis
- Neural network fundamentals and C programming best practices
This project demonstrates low-level machine learning implementation and reinforcement learning concepts using only standard C libraries.