This git repo is made during project71 of the Minor Embedded Systems at the Hogeschool Rotterdam by Renzo Daal, Calvin Kong, Bas van der Laan en Colin van der Spek. This repo contains files that are supposed to run on the FRDM-MCXN947 microcontroller and files that are supposed to run on a computer using Python.
The Python files in this repository have been tested and ran successfully using the following library versions:
Librosa=0.10.2.post1 Matplotlib=3.9.2 Numpy=1.26.4 Pandas=2.2.3 Tensorflow=2.17.0 Keras=3.5.0
The first folder named Python contains all the code that is meant to be run on a computer.
The first file named makeAndTrain.py is used for making and training the model. This file uses the UrbanSoun8K Dataset which can be found here https://urbansounddataset.weebly.com/urbansound8k.html. This dataset contains multiple categories of sound, which are sorted in a .csv file. We set the path to this .csv file on line 47 and the path to the audio files on line 48. When this script is run, it will make and train the model, and consequently display a few graphs to visualize the model. Finally it will quantize the model and output it as fully_quantized_gunshot_model_v5.tflite.
To be able to test the model on a computer we have made the python script testTheModel.py. This file takes a .wav audio file, and using the model, will determine if it is a gunshot or not. The audio file and model paths can be found on line 90 and 91.
Since you cannot use a .wav file on the microcontroller directly, you can use this script to turn it into a .h file which can be used on the microcontroller.
Finally we have made a Python file that does the MFCC and quantization to a .wav file and returns a .bin file which can then be converted to a .h file using the xxd command. This .h file can then be used to test the model on the microcontroller, whilst still doing the MFCC and quantization on a computer. File paths can be found on line 74 to 77.
The microcontroller folder contains the code to run the model on a the FRDM-MCXN947 microcontroller. To do this you will need to follow these steps:
- Download the MCUXpresso IDE which can be found here: https://www.nxp.com/design/design-center/software/development-software/mcuxpresso-software-and-tools-/mcuxpresso-integrated-development-environment-ide:MCUXpresso-IDE.
- Clone the folder Microcontroller into the workspace directory of the MCUXpresso IDE.
- You will have to refresh the
Project Explorerto be able to see the new project in there. If it doesn't appear, restarting the IDE should do the trick. - Build and debug the code.
The project has a source folder which contains the relevant code. The other folder worth mentioning is the eiq folder which contains the TensorFlow Lite Micro library. The source folder contains the main file and 2 more folders called audio and model. The audio folder contains all the MFCC processing logic and the audio files in .h format themselves, whilst the model folder contains all the logic related to running the model on the microcontroller and the model in .h format itself. In main.cpp we include the audio file that will be analyzed by the model. Note that only one audio file can be included at once, and the name of the array in the .h audio file must be audio_sample_data.