Merge branch 'main' into mackenzie/add-wetrun-testing

Author: mackenziesnyder

.github/workflows/python-testing.yml

@@ -113,5 +113,5 @@ jobs:
         run: |
           poetry run autoafids test_data/bids_wetrun_testing test_out participant --cores all --force-output \
           --stereotaxy STN --fidqc && \
-          python ./tests/test_fcsv_output.py --autoafids_fcsv ./test_data/test_out/sub-001/afids-cnn/*.fcsv \
+          python ./test_data/bids_wetrun_testing/test_fcsv_output.py --autoafids_fcsv ./test_data/test_out/sub-001/afids-cnn/*.fcsv \
           --baseline_fcsv ./test_data/bids_wetrun_testing/sub-001*.fcsv

.readthedocs.yaml

@@ -0,0 +1,25 @@
+---
+# .readthedocs.yml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+# Build documentation with MkDocs
+# mkdocs:
+#  configuration: mkdocs.yml
+
+# Optionally build your docs in additional formats such as PDF
+formats: [pdf]
+build:
+  os: ubuntu-20.04
+  tools:
+    python: '3.10'
+python:
+  install:
+    - requirements: docs/requirements.txt
+    - method: pip
+      path: .

README.md

@@ -1,74 +1,36 @@
 # Automatic Anatomical Fiducials (AutoAFIDs)
+[![Documentation Status](https://readthedocs.org/projects/autoafids/badge/?version=stable)](https://autoafids.readthedocs.io/en/stable/?badge=stable)
+![Version](https://img.shields.io/github/v/tag/afids/autoafids?label=version)
+![Python3](https://img.shields.io/badge/python-_3.10_|_3.11-blue.svg)
+[![Tests](https://github.com/khanlab/scattr/actions/workflows/test.yml/badge.svg?branch=main)](https://github.com/khanlab/scattr/actions/workflows/test.yml?query=branch%3Amain)
+![Docker Pulls](https://img.shields.io/docker/pulls/mackenzieasnyder/autoafids)
+
 AIMS Lab Research Team at the Robarts Research Institute - 2023-2024
 
 *This package is under active development. It should be stable and reproducible, but please let any of the active contributing members know if there are any bugs or unusual behaviour.*
 
 This Python package is a standard 3D [U-Net](https://arxiv.org/abs/1505.04597) (Ronneberger et al. 2015) machine learning model based on Snakemake and SnakeBIDS workflow management tools that leverages the recent release of the anatomical fiducial framework to solve the landmark regression problem on 3D MRI images. It is currently in development phase and contains tunable parameters that are not normally exposed in most other machine learning models; the user is highly advised to get familiar with the above mentioned workflow managaments tools and read docstrings and relevant documentation before using this software. Please see the [changelog](CHANGELOG.md) for more details. 
 
-## Table of Contents
-1. [Installation](#installation)
-2. [Workflow](#workflow)
-2. [Train](#train)
-3. [Apply](#apply)
-4. [Known issues](#known-issues)
-5. [Roadmap](#roadmap)
-6. [Questions, Issues, Suggestions, and Other Feedback](#questions--issues)
-
-## Installation 
-
-### Installing Poetry
-We use poetry tool for dependency management and to package the python project. You can find step by step instructions on how to install it by visiting it's official [website](https://python-poetry.org/docs/).
-
-### Local Installation
-
-After installing poetry, clone this repository via:
-
-```bash
-git clone https://github.com/afids/autoafids.git
-```
-
-You can then install the python package using one of the following commands, which should be executed within the repository folder (i.e., autoafids/).
-
-```bash
-poetry install
-```
-If you want to install in _develop mode_, use:
+## Workflow
 
-```bash
-poetry install -e
-```
+A brief summary of the workflow can be found below along with its Directed Acyclic Graph (DAG) (see documentation for a detailed summary):
 
-## Workflow
-Below is a simplified example of the workflow along with its Directed Acyclic Graph (DAG) for the `--procprofile` fast. Each rounded rectangle (vertex) in the DAG represents a rule, which encompasses some code or script that produces output file(s), and the arrows (edges) represent file inputs and outputs to these rules. 
+<img src="docs/workflow/dag.svg" alt="Workflow DAG" width="600">
 
-<img src="docs/dag.svg" alt="Workflow DAG" width="600">
+1. Preprocess input NIfTI files based on image modality
+2. Download and apply the fiducial model 
 
-### Processing landmark data (AFIDs)
+## Processing landmark data (AFIDs)
 1. Extract fiducial points from the landmark files (.fcsv is supported)
 2. Generate a landmark Euclidean distance/probability map with each voxel communicating distance to an AFID of interest
 
-## Train
-Currently, we support generating your own models (i.e., training) in a sperate workflow (i.e., afids-cnn: https://github.com/afids/afids-CNN). For more details, see [Known Issues](#known-issues).
-
-
-## Apply
-Use the classic BIDS App syntax to genereate output AFID .fcsv files. For other derivative outputs, the following flags will be supported: 
-
-`--fidqc`: for quality control of landmark prediction in the form of (*html) format
-
-`--regqc`: for quality control of registration on a BIDS dataset and its derivatives (e.g., fMRIPrep or LeadDBS derivative outputs) 
-
-`--stereotaxy`: predicts a .fcsv file with stereotactic targets (e.g., subthalamaic nucelus) also providing AC-PC transform files in the process 
-
-`--charing`: to make use of AFID charting analysis on a given dataset 
-  
 ## Known Issues
 - Factorize apply workflow to run per landmark of interest
 
-## Roadmap
+### **Full documentation:** [here](https://autoafids.readthedocs.io/en/)
 
-- Model optimization
-- Extension to incorporate new modalities (i.e., CT scans)
+## Revalent Papers: 
+* Link relavent papers to autoafids here 
 
 ## Questions, Issues, Suggestions, and Other Feedback
 Please reach out if you have any questions, suggestions, or other feedback related to this software—either through email (dbansal7@uwo.ca) or the discussions page. Larger issues or feature requests can be posted and tracked via the issues page. Finally, you can also reach out to Alaa Taha, the Science Lead.

autoafids/config/snakebids.yml

@@ -51,7 +51,7 @@ pybids_inputs:
       - reconstruction
       - run
 
-  CT:
+  ct:
     filters:
       suffix: ct
       extension: .nii.gz

autoafids/run.py

@@ -3,6 +3,9 @@
 
 from snakebids import bidsapp, plugins
 
+if "__file__" not in globals():
+    __file__ = "../autoafids/run.py"
+
 app = bidsapp.app(
     [
         plugins.SnakemakeBidsApp(Path(__file__).resolve().parent),
@@ -15,7 +18,7 @@
 
 
 def get_parser():
-    """Exposes parser for sphinx doc generation, cwd is the docs dir."""
+    """Exposes parser for sphinx doc generation, cwd is the docs dir"""
     return app.build_parser().parser
 
 

autoafids/workflow/scripts/apply.py

@@ -7,6 +7,7 @@
 
 # Suppresses INFO, WARNING, and ERROR logs.
 os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
+
 import tarfile
 from os import PathLike
 

docs/Makefile

@@ -0,0 +1,19 @@
+# Minimal makefile for sphinx documentation
+
+# You can set these variables from command line, and also
+# from the environment for the first two.
+SPHINXOPTS	?=
+SPHINXBUILD	?= sphinx-build
+SOURCEDIR	= .
+BUILDDIR	= _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+		@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option. $(0) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+		@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/conf.py

@@ -6,19 +6,22 @@
 
 # -- Path setup --------------------------------------------------------------
 
+import datetime
+
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath('.'))
 
 # -- Project information -----------------------------------------------------
 
 
 project = "autoafids"
-copyright = "2024, Alaa Taha"
+copyright = f"{datetime.date.today().year}, Alaa Taha"
 author = "Alaa Taha"
 
 
@@ -55,4 +58,4 @@
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-# html_static_path = ["_static"]
+html_static_path = ["_static"]

docs/contributing/contributing.md

@@ -0,0 +1,79 @@
+# Contributing to Autoafids
+
+Autoafids python package uses Poetry pacakge manager to manage its dependencies. You’ll need it installed on your machine before contributing to the software. Installation instructions can be found on the 
+[Poetry website](https://python-poetry.org/docs/master/#installation).
+
+Autoafids primarily caters to T1w modality images, in which case it doesn't need to depend on pacakges outside of python but for other modalities it has a single dependency outside of python, i.e., `synthsr`. 
+
+Note: These instructions are only recommended if you are making changes to the Autoafids codebase and committing these back to the repository or if you are using Snakemake’s cluster execution profiles.
+
+## Setup the development environment
+
+Once Poetry is available, clone this repository and install all dependencies (including `dev`):
+
+```
+git clone https://github.com/afids/autoafids.git
+cd autoafids
+poetry install --with dev 
+```
+
+Poetry will automatically create a virtual environment. To customize where 
+these virtual environments are stored, see the poetry docs 
+[here](https://python-poetry.org/docs/configuration/).
+
+Then, you can run autoafids:
+
+```
+autoafids -h
+```
+
+or you can activate a virtual environment shell and run autoafids directly:
+
+```
+poetry shell
+autoafids
+```
+
+You can exit the poetry shell with `exit`
+
+## Running and fixing code format quality
+
+Autoafids uses [poethepoet](https://github.com/nat-n/poethepoet) as a task runner.
+You can see what commands are available by running:
+
+```
+poetry run poe 
+```
+
+We use a few tools, including `ruff`, `snakefmt`, and `yamlfix` to ensure 
+formatting and style of our codebase is consistent. There are two task runners 
+you can use to check and fix your code, which can be invoked with:
+
+```
+poetry run poe quality-check
+poetry run poe quality
+```
+
+_Note: If you are in a poetry shell, you do not need to prepend `poetry run` to
+the command._
+
+## Dry-run / testing your workflow
+
+Using Snakemake\'s dry-run option (`--dry-run`/`-n`) is an easy way to verify
+any changes made to the workflow are working direcctly. The `tests/data` folder 
+contains a _fake_ BIDS dataset (i.e. dataset with zero-sized files) that is 
+useful for verifying different aspects of the workflow. These dry-run tests are 
+part of the automated Github actions that are run for every commit.
+
+You can invoke the pre-configured task via 
+[poethepoet](https://github.com/nat-n/poethepoet) to perform a dry-run:
+
+```
+poetry run poe test_base
+```
+
+This performs a number of tests, involving different scenarios in which a user
+may use autoafids.
+
+## Questions, Issues, Suggestions, and Other Feedback
+Please reach out if you have any questions, suggestions, or other feedback related to this software—either through email (dbansal7@uwo.ca) or the discussions page. Larger issues or feature requests can be posted and tracked via the issues page. Finally, you can also reach out to Alaa Taha, the Science Lead.

docs/getting_started/docker.md

@@ -0,0 +1,30 @@
+# Running Autoafids with Docker
+
+Brief walkthrough on how to prepare the `autoafids` package in a Docker container.
+
+## Local Build Instructions
+To build a docker image of the project locally, do the following:
+1. Download the git repo:
+
+```
+git clone https://github.com/afids/autoafids.git
+```
+2. Ensure you have installed [Docker](https://docs.docker.com/engine/install/) and that it's up and running.
+
+3. Build the docker image using the following command; it should be executed within the repository folder (i.e., autoafids/). 
+
+```
+docker build -t autoafids:<VERSION> -f ./docker/Dockerfile .
+```
+
+4. Once the image is built, you can then run the pipeline as follows:
+
+```
+docker run -it --rm \                     
+-v local/path/to/dataset:/bids_dataset \ 
+-v local/path/to/dataset/derivatives:/output \  
+autoafids:<VERSION> \
+/bids_dataset /output participant --cores all
+```
+
+The first time building this image may take some time because poetry takes a while to install all the required python packages.