Biofeedback Nintendo Entertainment System (BioNES)

A plug-and-play MATLAB based tool to use NES games for multimodal biofeedback
Explore the docs »

---

Table of Contents

• About the Project
  • Built with
  • Features
  • Why this tool?
• Getting started
  • Prerequisites
  • Installation and Running
• Usage
  • Examples
• Documentation
• Roadmap
• Contributing
• License
• Contact
• Acknowledgment
• Citation

About the project

BioNES (Biofeedback Nintendo Entertainment System) is an open-source plug-and-play MATLAB-based tool to use NES games for multimodal biofeedback. It can be used to deliver the HRV biofeedback via any game designed for the NES system. It can receive real-time heartbeat interval (RR) values from Arduino + ear-clip PPG sensor (or any sensor capable to send real-time heart rate pulses to Arduino). After the acquisition, it computes real-time heart rate and heart rate variability (HRV), biofeedback parameters and then sends feedback to the FCEUX emulator which is used to play the NES game. Besides in-game biofeedback, it also offers real-time data visualization in time-series plots and local saving of data for offline analysis. This tool can benefit the researchers (especially from biofeedback or physiological measurement domain) or hobbyist, who want to quickly deploy a biofeedback system, want to explore the NES games for biofeedback, or just want to record the physiological signals.

Built with

• GUI is built using the Guide tool in MATLAB-v2017b.
• Arduino Mega board for hardware.
• Grove ear-clip sensor for photoplethysmograph (PPG) data acquisition.
• FCEUX emulator to play NES game.
• Super Mario Bros. game for feedback.

Features

• Plug and play biofeedback system for SMB game (using other NES games may need minor changes).
• Data acquisition sampling rate of 10 Hz to acquire RR intervals from Arduino.
• Feedback is displayed at a fixed refresh rate at 60 Hz.
• Real-time data visualization (in time-series plot).
• Local saving of data in .mat format for offline analysis.
• Screenshot of current GUI for reference purpose.
• 3 independent modes of working.
  • Data acquisition only
  • Gameplay without biofeedback
  • Gameplay with biofeedback

Why this tool?

Biofeedback therapy has potential benefits for managing stress. However, traditional biofeedback is an expensive and monotonous process. To increase the engagement in biofeedback sessions, video games can be used. But the problem is the limited availability of affordable video-game-based biofeedback solutions. BioNES is developed keeping in mind the cost-effectiveness and ease of use to deliver biofeedback while simultaneously providing reliability as a research tool and flexibility to allow researchers/developers to modify as per their needs. The efficacy of BioNES is validated using observational study and the paper is currently under review in a reputed journal.

Getting started

Prerequisites

• MATLAB

MATLAB-ver2017b was used to develop the BioNES. It was also tested with ver2015b and ver2018b for backward and forward compatibility respectively.

• Arduino hardware board.
• Grove ear-clip sensor.
• FCEUX.
• NES game ROM file. This version of BioNES uses the Super Mario Bros. game without any modification, for other NES games you have to make minor changes to the BioNES.lua file.

Installation and running

1. Download the latest stable release and extract contents into your MATLAB working directory. Alternatively, you can also download the latest code from the repository.
2. Open BioNES.m in MATLAB.
3. Connect Arduino board to PC with USB cable. Upload BioNES.ino sketch to Arduino board.
4. Connect the ear-clip sensor to the Arduino board.
5. Run BioNES.m and wait for the GUI to appear.
6. In the GUI select the COM port for the Arduino and press the Connect button.
7. After a successful connection, appropriate settings can be selected.
8. Select the biofeedback checkbook (if biofeedback gameplay is needed) and press the Connect button in the Game panel. The FCEUX window will open with the pre-configured settings.
9. Start the acquisition from the Start/Stop button in the Control panel.
10. The acquisition runs till a set time or can be stopped manually.
11. After successful completion, the information panel shows various statistics related to the acquisition.
12. Finally, the data can be saved for offline processing.

Usage

Use this space to show useful examples of how a project can be used. Additional screenshots, code examples, and demos work well in this space. You may also link to more resources.
For more examples, please refer to the Documentation.

Examples

See usage section in documentation.

Documentation

The documentation is available at https://kulbhushanchand.github.io/BioNES/

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Any contributions you make are greatly appreciated. You can contribute to this project in the following ways :

• Add new functionality
• Review code
• Raise issues about bugs/features/doubts
• Proof-read the documentation
• Cite if used in a publication
• Star on GitHub
• Share with others

Please note that this project is released with a Contributor Code of Conduct. By contributing to this project you agree to abide by its terms.

License

This project is distributed under the GPLv3 License. See LICENSE for more information.

Citation

If you are using this software, please cite the following publications

Chand, K., & Khosla, A. (2022a). BioNES: A plug-and-play MATLAB-based tool to use NES games for multimodal biofeedback. SoftwareX, 19, 101184. https://doi.org/10.1016/j.softx.2022.101184

Chand, K., & Khosla, A. (2022b). Efficacy of Using Retro Games in Multimodal Biofeedback Systems for Mental Relaxation. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 14(1), 1–23. https://doi.org/10.4018/IJGCMS.295874

OSF | Efficacy of a multimodal biofeedback system for mental relaxation using NES games. (n.d.). Retrieved September 15, 2022, from https://osf.io/q5ez3/

Chand, K. (2021). Kulbhushanchand/BioNES [MATLAB]. https://github.com/kulbhushanchand/BioNES (Original work published 2021)

Acknowledgment

I would like to thank my PhD supervisor Prof. Arun Khosla at Dr B R Ambedkar National Institute of Technology for his guidance and kind support. I also want to acknowledge the open-source tools used in some parts of this project.

• FCEUX is used as an NES emulator.
• drawio-desktop is used to create the logo and diagrams.

Contact

Kulbhushan Chand