The idea is to develop a general CNN accelerator using Winograd convolution, this will be targeted for Keyword Spotting (KWS) detection using MFCC matrix. This will have a Wishbone interface and will need MFC coeffients to be calculated and available from Caravel harness.
I plan to have a systolic/bit-serial architecture inspired by bit serial processors like SERV. Evaluating one CNN layer at a time on a small footprint and then serializing for the entire network. An FSM control will manage the workflow, orchestrating the loading of weights and storing intermediate results to a buffer.
The NN architecture, training and inference flow is followed from the following papers,
Streaming keyword spotting on mobile devices
Hello Edge: Keyword Spotting on Microcontrollers
- DS-CNN with exact hyperparameters to be decided upon experiments and iterations
- Parallel Convolution Execution unit to calculate 'n' 2D Convolutions in parallel
- FSM to serialize 'n' parallel convolution blocks over all features
- 16 kHz sampling rate with INT8 arithmetic
- Wishbone bus to the Caravel SoC
- The MFC Coefficients will be streamed from Caravel SoC
The starting point of development is exactly from the referenced papers, the optimizations to NN models (pruning and quantization) will be done based on results of training, inference performance and resource requirements obtained during experiments.
| ❗ Important Note |
|---|
Refer to README for a quickstart of how to use caravel_user_project
Refer to README for this sample project documentation.
Refer to the following readthedocs for how to add cocotb tests to your project.