Bonsai Python Codes

This repository contains Python, Arduino, analysis, and plotting code used for a closed-loop brain-machine interface (BMI) and rotary encoder behavioural setup. The code was developed for a Master End Project focused on building and validating a pre-clinical BMI platform for head-fixed mice.

The setup combines Bonsai, Python, Arduino, Open Ephys, a motorized lick-port stage, and HDF5-based data logging. The repository includes current experiment scripts, analysis scripts, figure-generation scripts, and setup documentation.

Scientific Overview

The project implements synchronized multi-modal recording and control for behavioural and BMI experiments.

The main system supports:

• Neural recording through Open Ephys and Bonsai
• Real-time threshold-based spike-event detection
• Exponential moving average firing-rate estimation
• Closed-loop BMI decoder output
• Arduino-controlled motorized stage movement
• Rotary encoder behavioural control
• Position-based reward delivery
• HDF5 data logging
• Offline reconstruction, validation, and figure generation

The code is intended for research use. It is not a clinical BMI system.

Repository Structure

The repository is organized into the following main folders:

Bonsai-Python-codes/
├── Analysis and plotting codes/
│   ├── Analysis codes/
│   └── Plotting codes/
├── Arduino codes/
├── Bonsai workflows/
├── Experiment codes/
├── config_template.py
├── .env.example
├── SETUP_CONFIG.md
├── requirements.txt
├── .gitignore
└── README.md

Analysis and plotting codes

This folder contains scripts for post-experiment processing and figure generation.

The analysis scripts are used to:

• Read HDF5 recordings
• Reconstruct runtime, motor, trial, and neural-processing tables
• Extract behavioural and BMI performance metrics
• Run random-baseline comparisons
• Perform offline replay of the neural-processing pipeline
• Inspect raw neural waveforms and Open Ephys recordings
• Generate CSV tables for plotting

The plotting scripts are used to:

• Generate thesis and publication-style figures
• Visualize synchronized runtime behaviour
• Plot BMI performance across recordings
• Plot random-baseline analysis results
• Plot behavioural learning and task execution
• Plot neural-processing validation and movement-aligned activity

Arduino codes

This folder contains the Arduino firmware used to control hardware components such as:

• Stepper motor driver
• Direction and step pins
• Reward solenoid or relay output
• Serial communication with Python
• Position reporting back to the PC

The Arduino communicates with Python through serial commands and status messages.

Bonsai workflows

This folder contains Bonsai workflows used during acquisition and experiment control.

These workflows are part of the full experimental setup and may need to be adjusted to match local hardware, Open Ephys configuration, serial ports, and data paths.

Experiment codes

This folder contains the current experiment scripts used to run behavioural and BMI experiments.

Main experiment scripts include:

• Rotary encoder control scripts
• Large-amplitude BMI control scripts
• Real-time visualization scripts
• Camera frame logging and validation scripts

Older development versions were removed from the public repository to keep the codebase clean. A private local backup can be kept outside Git if older versions are needed later.

HDF5 Data Structure

Recordings are saved in HDF5 format. Depending on the experiment mode and script version, files can contain groups such as:

/meta
/trial
/motor
/neural
/camera
/viz
/firing_rate
/bonsai
/debug

Typical contents include:

/meta
    Recording parameters, sample rate, selected channels, buffer size, task settings

/trial
    Trial counters, trial states, trial outcomes, success or timeout information

/motor
    Motor position, movement direction, command timestamps, step acknowledgements

/neural
    Raw neural buffers, raw timestamps, spike thresholds, threshold crossings

/camera
    Camera frame numbers and timestamps

/viz
    Runtime snapshots, trial state, decoder output, EMA firing rates, thresholds

/firing_rate
    Firing-rate estimates used for decoder analysis

/bonsai
    Bonsai-related timing and event information

/debug
    Optional debugging logs

The exact structure can differ between recording types and script versions. Some analysis scripts include HDF5 inspection utilities to print available groups and datasets.

Installation

Requirements

Use Python 3.8 or higher.

Install the required packages with:

pip install -r requirements.txt

If you use a virtual environment, activate it first.

Example on Windows PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Example on Linux or macOS:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Configuration and File Paths

This repository does not use hardcoded local paths. Earlier development versions used machine-specific paths to local folders and lab network storage, but these paths have been removed for publication.

Most scripts now use environment variables with safe fallback paths.

A typical pattern is:

Path(os.getenv("VARIABLE_NAME", "./data/example_folder"))

This means:

1. If the environment variable is set, the script uses that path.
2. If the environment variable is not set, the script uses a safe local default path inside ./data/.

The ./data/ folder is intended for local data only and should not be committed to GitHub.

Quick Configuration Example

Before running a script, set the relevant environment variables in your terminal.

Example on Windows PowerShell:

$env:LA_RESULTS_PATH = "D:\Data\LA results"
$env:RUNTIME_TABLE_DIR = "D:\Data\LA results\runtime_tables"
$env:ARDUINO_PORT = "COM3"

Then run the script:

python "Analysis and plotting codes\Plotting codes\your_script.py"

These variables only exist in the current PowerShell session. If you close the terminal, you need to set them again.

Persistent Environment Variables on Windows

To save an environment variable permanently for your Windows user account, use setx:

setx LA_RESULTS_PATH "D:\Data\LA results"
setx RUNTIME_TABLE_DIR "D:\Data\LA results\runtime_tables"
setx ARDUINO_PORT "COM3"

After using setx, close and reopen PowerShell or VS Code before running the scripts again.

Environment Variables on Linux or macOS

For the current terminal session:

export LA_RESULTS_PATH="/home/username/data/la_results"
export RUNTIME_TABLE_DIR="/home/username/data/la_results/runtime_tables"
export ARDUINO_PORT="/dev/ttyUSB0"

To make these variables persistent, add them to your shell configuration file, such as ~/.bashrc or ~/.zshrc.

Example:

echo 'export LA_RESULTS_PATH="/home/username/data/la_results"' >> ~/.bashrc
echo 'export RUNTIME_TABLE_DIR="/home/username/data/la_results/runtime_tables"' >> ~/.bashrc
echo 'export ARDUINO_PORT="/dev/ttyUSB0"' >> ~/.bashrc
source ~/.bashrc

Common Environment Variables

Most scripts in this repository use environment variables to find local data folders, input files, output folders, and hardware settings. This keeps the public code free of hardcoded local paths.

A typical script uses a pattern like this:

Path(os.getenv("LA_RESULTS_PATH", "./data/la_results"))

This means that the script first checks whether LA_RESULTS_PATH has been set on your computer. If it has not been set, the script falls back to the safe example folder ./data/la_results.

Variable	Purpose	Example
BONSAI_RECORDINGS_PATH	Folder where experiment scripts save new recordings	D:\Bonsai recordings\Results
LA_RESULTS_PATH	Folder containing BMI or LA analysis data	D:\Data\LA results
RUNTIME_TABLE_DIR	Folder containing runtime, motor, and trial CSV tables	D:\Data\LA results\runtime_tables
RE_ANALYSIS_DATA_DIR	Folder containing rotary encoder HDF5 recordings	D:\Data\RE_Analysis
RE_ANALYSIS_TABLE_DIR	Folder containing rotary encoder analysis tables	D:\Data\RE_Analysis\analysis_tables
BMI_PERFORMANCE_TABLE_DIR	Folder containing BMI performance tables	D:\Data\bmi_performance_tables
BMI_RANDOM_BASELINE_TABLE_DIR	Folder containing BMI random-baseline tables	D:\Data\bmi_random_baseline_tables
OPEN_EPHYS_RECORDING_DIR	Folder containing an Open Ephys recording	D:\Data\OpenEphys\example_recording
ARDUINO_PORT	Arduino serial port	COM3
ARDUINO_BAUDRATE	Arduino serial baud rate	115200

Some scripts also define more specific variables near the top of the file. These are usually related to one figure, one analysis step, or one recording selection.

Examples:

RUNTIME_OVERVIEW_RECORDING_BASENAME
TRIAL_EXECUTION_RECORDING_BASENAME
NEURAL_PROCESSING_RECORDING_BASENAME
MOVEMENT_PSTH_RECORDING_BASENAME
OFFLINE_REPLAY_RECORDING_BASENAME
WAVEFORM_INSPECTION_RECORDING_BASENAME
SPIKE_TRAIN_RECORDING_BASENAME

These variables allow users to select a specific recording without editing the public code.

Environment Variables vs. config.py

The main configuration method in this repository is the use of environment variables. This is the recommended and most reliable way to run the scripts on another computer.

The repository also includes config_template.py. This file can be copied to a private local file called config.py if users prefer to keep their local paths in one place. However, config.py is only used by scripts that explicitly import it. If a script uses os.getenv directly, then the environment variable should be set instead.

In short:

Recommended method: environment variables
Optional helper: config.py
Never commit: config.py or .env

Optional Local Configuration Files

This repository also includes:

config_template.py
.env.example
SETUP_CONFIG.md

These files provide examples for local setup.

You can either use environment variables directly, or create a private local configuration file. For most users, environment variables are the simplest option because this is how the cleaned public scripts are configured. The optional config.py approach is explained below for users who prefer a local configuration file.

Option 1: Environment Variables

This is the recommended option for most users.

Set variables in your terminal before running a script:

$env:LA_RESULTS_PATH = "D:\Data\LA results"
python "Analysis and plotting codes\Plotting codes\your_script.py"

Option 2: Local .env File

Copy the example file:

cp .env.example .env

On Windows PowerShell:

Copy-Item .env.example .env

Then edit .env with your own paths.

The .env file is ignored by Git and should not be committed.

A .env file is only used if your workflow or script loads it, for example with python-dotenv. If the script does not load .env, set the variables directly in the terminal instead.

Option 3: Local config.py

The file config_template.py is included as an example configuration file. Users can copy it to create their own private local configuration file:

cp config_template.py config.py

On Windows PowerShell:

Copy-Item config_template.py config.py

After copying it, open config.py and replace the example paths with paths that match your own computer.

Example:

from pathlib import Path

BONSAI_RECORDINGS_PATH = Path(r"D:\Bonsai recordings\Results")
LA_RESULTS_PATH = Path(r"D:\Data\LA results")
RUNTIME_TABLE_DIR = LA_RESULTS_PATH / "runtime_tables"

RE_ANALYSIS_DATA_DIR = Path(r"D:\Data\RE_Analysis")
RE_ANALYSIS_TABLE_DIR = RE_ANALYSIS_DATA_DIR / "analysis_tables"

BMI_PERFORMANCE_TABLE_DIR = Path(r"D:\Data\bmi_performance_tables")
BMI_RANDOM_BASELINE_TABLE_DIR = Path(r"D:\Data\bmi_random_baseline_tables")

ARDUINO_PORT = "COM3"
ARDUINO_BAUDRATE = 115200

The local config.py file is ignored by Git and should not be committed. This allows each user to keep their own file paths private.

Important: config.py is only used by scripts that explicitly import it.

For example, a script must contain something like:

from config import LA_RESULTS_PATH, RUNTIME_TABLE_DIR

or:

import config

If a script instead uses os.getenv, like this:

Path(os.getenv("LA_RESULTS_PATH", "./data/la_results"))

then the script reads from environment variables, not directly from config.py.

Therefore, there are two ways to use the configuration system:

Method 1: Set environment variables directly
Method 2: Use config.py in scripts that import it

For the cleaned public scripts in this repository, environment variables are the main method. The config.py file is mainly provided as an optional helper or reference for users who prefer to keep their local paths in one private place.

Example: Running an Experiment Script

Before running an experiment script, set the recording output folder and Arduino port.

Windows PowerShell:

$env:BONSAI_RECORDINGS_PATH = "D:\Bonsai recordings\Results"
$env:ARDUINO_PORT = "COM3"
python "Experiment codes\LA_Experiment_2105_StopAtZero.py"

The script will save new recordings to the folder defined by BONSAI_RECORDINGS_PATH.

Example: Running a Plotting Script

If a plotting script needs BMI performance tables, set the table directory first.

$env:BMI_PERFORMANCE_TABLE_DIR = "D:\Data\bmi_performance_tables"
python "Analysis and plotting codes\Plotting codes\your_bmi_performance_figure_script.py"

The script will read input CSV files from BMI_PERFORMANCE_TABLE_DIR and save output figures to the configured output folder.

Example: Running Random-Baseline Figure Scripts

$env:BMI_RANDOM_BASELINE_TABLE_DIR = "D:\Data\bmi_random_baseline_tables"
python "Analysis and plotting codes\Plotting codes\your_random_baseline_figure_script.py"

Example: Running Offline Replay

$env:LA_RESULTS_PATH = "D:\Data\LA results"
$env:RUNTIME_TABLE_DIR = "D:\Data\LA results\runtime_tables"
$env:OFFLINE_REPLAY_RECORDING_BASENAME = "example_recording"

python "Analysis and plotting codes\Analysis codes\your_offline_replay_script.py"

The script will look for files such as:

D:\Data\LA results\example_recording.h5
D:\Data\LA results\runtime_tables\example_recording_runtime_table.csv
D:\Data\LA results\runtime_tables\example_recording_motor_table.csv

Data and Output Files

Large data files are not included in this repository.

Do not commit:

*.h5
*.hdf5
*.npy
*.npz
large CSV output tables
generated figure folders
local data folders
private configuration files

The .gitignore is set up to ignore common local data and configuration files.

Recommended local structure:

data/
├── recordings/
├── la_results/
│   ├── runtime_tables/
│   ├── bmi_performance_tables/
│   └── bmi_random_baseline_tables/
├── re_analysis/
│   └── analysis_tables/
├── open_ephys/
└── figures/

Usage Overview

The codebase generally follows this workflow:

Experiment recording
    ↓
HDF5 data file
    ↓
Runtime and analysis table reconstruction
    ↓
CSV summary tables
    ↓
Figure generation
    ↓
Thesis or publication output

Main Workflows

1. Real-Time Experiment Recording

The experiment scripts can run behavioural or BMI tasks.

Typical steps:

1. Start the relevant Bonsai workflow.
2. Make sure Open Ephys and Bonsai are configured.
3. Connect the Arduino and check the correct serial port.
4. Set BONSAI_RECORDINGS_PATH and ARDUINO_PORT.
5. Run the Python experiment script.
6. Check that the HDF5 output file is saved correctly.

2. Post-Experiment Analysis

Analysis scripts read HDF5 files and reconstruct structured CSV tables.

These tables can include:

• Runtime state samples
• Trial-level outcomes
• Motor position and step events
• Decoder outputs
• Firing-rate estimates
• Random-baseline analysis values
• Behavioural learning metrics

3. Offline Decoder Replay

Offline replay scripts reload saved raw neural buffers and rerun the neural-processing and decoder logic offline.

This is useful for:

• Validating the online pipeline
• Inspecting raw-buffer reconstruction
• Comparing online and offline processing
• Generating corrected neural-processing figures

4. Figure Generation

Plotting scripts read CSV tables and create thesis-style figures.

Figures are generally saved as:

.png
.pdf

Most figures use high-resolution settings suitable for thesis or publication use.

Hardware Notes

The hardware setup can include:

• Open Ephys acquisition system
• Arduino microcontroller
• Stepper motor driver
• Motorized lick-port stage
• Reward solenoid or relay
• Rotary encoder
• Optional camera
• Optional lick or TTL sensor

The Arduino port is set with:

ARDUINO_PORT

Example:

$env:ARDUINO_PORT = "COM3"

The default baud rate depends on the script, but many experiment scripts use:

115200 baud

Important Interpretation Notes

The analysis scripts use threshold-crossing events. These are not spike-sorted single-unit spikes.

Therefore, figures and analyses should be interpreted as:

threshold-crossing activity

not as:

single-unit activity

For older recordings, some scripts include offline raw-buffer reconstruction or offline decoder replay. These scripts are intended to validate and correct the neural-processing pipeline where needed.

Reproducibility Notes

The repository contains code, not raw experimental data.

To reproduce the figures, users need:

1. The relevant HDF5 recordings
2. Reconstructed CSV analysis tables
3. Correct environment variables pointing to their local data folders
4. The required Python packages from requirements.txt

Because local file paths and raw data are not included, users must configure their own local data locations.

Cleaning and Publication Notes

This repository has been cleaned for publication.

The public version:

• Removes machine-specific hardcoded paths
• Uses environment variables for data locations
• Excludes outdated development code
• Excludes local data and generated outputs
• Keeps current experiment, analysis, and plotting scripts

A private backup of older development code may be kept outside Git, but it is not required for using the public repository.

Troubleshooting

File not found

If a script cannot find a file, check the relevant environment variable.

Example:

echo $env:LA_RESULTS_PATH
echo $env:RUNTIME_TABLE_DIR

Make sure the folder exists and contains the expected input files.

Arduino not found

Check that the Arduino is connected and that the correct port is set.

Windows:

$env:ARDUINO_PORT = "COM3"

Linux:

export ARDUINO_PORT="/dev/ttyUSB0"

On Windows, the port can be checked in Device Manager under "Ports (COM & LPT)".

Environment variable not updating

If you used setx on Windows, close and reopen PowerShell or VS Code. Persistent variables are not always available in terminals that were already open.

Script uses default example path

If a script tries to use ./data/..., then the relevant environment variable was probably not set.

Set the correct variable before running the script.

Example:

$env:LA_RESULTS_PATH = "D:\Data\LA results"

config.py does not seem to work

Check whether the script actually imports config.py.

If the script uses:

os.getenv(...)

then set environment variables instead.

Security and Git Notes

Do not commit:

config.py
.env
local data folders
recordings
generated figures
private lab paths
large HDF5 files

Before making the repository public, it is recommended to run a scan such as:

git grep -n -E "C:\\Users|freenasc|WhiskerStore|password|secret|api_key"

Expected result: no private paths or secrets. Placeholder examples in documentation are acceptable if they do not include real private paths.

Author and Contributors

Bonsai Python Codebase Development Team

Project code and analysis developed by Roelof Jager as part of a Master End Project.

License

This repository is released under the MIT License. This means the code can be reused, modified, and distributed, provided that the license notice is included. See the LICENSE file for details.

Further Reading

• Bonsai framework: https://bonsai-rx.org/
• Open Ephys: https://open-ephys.org/
• HDF5 and h5py: https://www.h5py.org/

Last Updated

June 2026