bioRC: Reproducible Pipeline for Empirical & Synthetic Connectome Analysis

This repo contains scripts to:

1. generate synthetic networks (CM, ER, SBM) from empirical connectomes,
2. run bioRC batch analyses in R across multiple signals and SNR levels, and
3. post-process and visualize results (e.g., Z-score heatmaps).

---

Table of Contents

• Requirements
• Directory Layout
• Quick Start
• Generate Synthetic Networks
• Run bioRC Batches (R)
• License & Citation
• Data and attribution
• Takedown and permissions

---

Requirements

OS

• Linux/macOS. (Windows works via WSL; graph-tool is Linux-friendly.)

Python (≥ 3.10 recommended)

• numpy, pandas, matplotlib, seaborn, networkx, igraph
• graph-tool (optional, used for SBM parameter inference via blockmodel)
• joblib, tqdm
• Install example:

  conda create -n biorc python=3.11 -y
  conda activate biorc
  pip install numpy pandas matplotlib seaborn networkx igraph joblib tqdm
  # graph-tool install varies by OS (optional)

R

• A working Rscript on PATH.
• Packages: igraph, plus scripts in this repo:
  • scripts/run_batch_r.R (entrypoint used by Python)
  • batch_analysis.R (fallback if the above is missing)
  • SBM_gen.R (used only by SBM generation; must sit next to generate_networks.py or in the CWD you run from)

Signals

• Place time-series files (e.g. White.txt, IkedaX.txt) under ./signals/.

---

Directory Layout

Below is the expected project structure after running the steps:

.
├── Emp_connectomes/                 # input empirical edge lists (1-based or 0-based, see script notes)
│   ├── CElegans-*.edges
│   └── ...
├── SBM_gen.R                        # R generator for SBM (required for SBM only)
├── generate_networks.py             # Python generator for CM/ER/SBM
├── scripts/
│   ├── run_biorc_batches.py         # Orchestrates batch runs (Emp, CM, ER, SBM)
│   └── run_batch_r.R                # R entrypoint called by Python
├── signals/
│   ├── White.txt
│   ├── IkedaX.txt
│   └── ...
├── Networks/
│   ├── CM_connectomes/0..R-1/*.edges
│   ├── ER_connectomes/0..R-1/*.edges
│   └── SBM_connectomes/0..R-1/*.edges
├── results_Emp/                     # raw R outputs for empirical
├── results_CM/                      # raw R outputs for CM
├── results_ER/                      # raw R outputs for ER
├── results_SBM/                     # raw R outputs for SBM
├── results_Emp_processed/           # CSV summaries (auto)
├── results_CM_processed/
├── results_ER_processed/
├── results_SBM_processed/

---

Quick Start

# Activate env
conda activate biorc

# Generate CM/ER/SBM networks (20 replicates, 8 threads)
python generate_networks.py   --emp-dir ./Emp_connectomes   --do-cm --do-er --do-sbm   --rs 20 --sbm-rscript ./SBM_gen.R

# Run bioRC batches (R) over Emp/CM/ER/SBM families
python scripts/run_biorc_batches.py   --empirical-dir ./Emp_connectomes   --cm-dir ./CM_connectomes   --er-dir ./ER_connectomes   --sbm-dir ./SBM_connectomes   --signals-dir ./signals

If you only have some families, just supply the corresponding --*-dir that exist; the script skips missing ones.

---

Generate Synthetic Networks

The generator reads each empirical edge list, infers parameters if needed (SBM), and writes R replicates to ./<MODEL>_connectomes/<rep>/<file>.edges.

python generate_networks.py   --emp-dir ./Emp_connectomes   [--do-cm] [--do-er] [--do-sbm]   --rs 20   --jobs 8

Flags

• --emp-dir : folder of empirical edge lists.
• --do-cm, --do-er, --do-sbm: choose models to generate.
• --rs : number of replicates (replicate folders 0..rs-1).
• --jobs : parallel workers. If you see resource_tracker issues, the script may switch to a threading backend.

SBM specifics

• SBM_gen.R must be visible from the working directory you call the script in (or next to generate_networks.py).
• The script writes temporary block_sizes.csv and pref_matrix.csv for each file, used by SBM_gen.R.

---

Run bioRC Batches (R)

Use the provided orchestrator to run the R analysis on:

• empirical networks (--empirical-dir),
• CM (--cm-dir),
• ER (--er-dir),
• SBM (--sbm-dir).

It calls scripts/run_batch_r.R and writes raw outputs under ./results_*, then auto-summarizes into ./results_*_processed/all_results.csv.

python scripts/run_biorc_batches.py   --empirical-dir ./Emp_connectomes   --cm-dir ./CM_connectomes   --er-dir ./ER_connectomes   --sbm-dir ./SBM_connectomes   --signals-dir ./signals   --snrs 0,5,10,25,50,100   --mc 20   --replicates 20

Notes

• Omit a --*-dir if you don’t have that family; the script only runs for existing dirs.
• --mc affects empirical runs (Monte Carlo repeats per empirical network).
• --replicates is used for CM/ER/SBM (numbered subfolders present).
• --batch-script defaults to scripts/run_batch_r.R; it falls back to batch_analysis.R if missing.

---

3) Post-process & Plot

Processed CSVs

After running batches, check:

results_Emp_processed/all_results.csv
results_CM_processed/all_results.csv
results_ER_processed/all_results.csv
results_SBM_processed/all_results.csv

For the heatmap zscores run:

python plots/plot_heatmap_zscores.py --signals White,IkedaX,MackeyGlass_t17,fBrownTurb_H075_D1 --tau 4096 --snr 100

---

License & Citation

• Licensing terms for this code follow the repository’s LICENSE file.
• If you use this pipeline in academic work, please cite the original paper and this repository.

---

Data and attribution

We include Emp_connectomes.zip only for convenience and reproducibility. All underlying datasets are the intellectual property of their original authors and repositories. We do not claim ownership. If hosting any mirrored file here raises concerns, contact us and we will remove it promptly.

Please cite the original sources

If you use these data, cite the corresponding papers/repositories listed below.

• C. elegans (hermaphrodite, pharynx/whole) Source repo: https://github.com/ericmjonas/circuitdata/tree/master/celegans_herm

• Ciona intestinalis (larva) Article: eLife - DOI: https://doi.org/10.7554/eLife.16962

• Drosophila medulla (N=1781) Article: Nature - http://www.nature.com/nature/journal/v500/n7461/full/nature12450.html

• Human (N=188, NKI-Rockland) Dataset: NKI-Rockland Sample. Please retrieve from the official project pages and cite accordingly.

• Human (N=998, Hagmann/Lausanne 2008 parcellation) Article: PLoS Biology - DOI: https://doi.org/10.1371/journal.pbio.0060159

• Rat (Rattus norvegicus), Mouse, Rhesus Various mesoscale/EM connectomes commonly distributed via NeuroData / Open Connectome Project. Portal: https://neurodata.io/project/connectomes/

• Rhesus brain Article: PLOS ONE - https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0046497

• Rhesus cerebral cortex Article: Cerebral Cortex - DOI: https://doi.org/10.1093/cercor/bhs270

• Rhesus interareal cortical network Article: PNAS - DOI: https://doi.org/10.1073/pnas.1218972110

Takedown and permissions

We mirror small subsets solely for research reproducibility. If you are a rights holder and prefer we do not mirror your files here, please email manlio.dedomenico@unipd.it with the file names/paths and we will remove them immediately.