Machine Learning Engineering Open Book

Python 3.8+ toolbox for submitting jobs to Slurm

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.

Open source web interface for Slurm HPC clusters

Best practices & guides on how to write distributed pytorch training code

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

Create clusters of VMs on the cloud and configure them with Ansible.

Lightweight fast function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪

Simplify HPC and Batch workloads on Azure

SEML: Slurm Experiment Management Library

A toolset for black-box hyperparameter optimisation.

Slurm-Mail is a drop in replacement for Slurm's e-mails to give users much more information about their jobs compared to the standard Slurm e-mails.

Slurm Docker Container on CentOS 7

A Slurm dashboard for the terminal.

IPython magic for SLURM.

🦠📇 Microbial genomes-to-report pipeline

A toolkit featured artificial intelligence × ab initio for computational chemistry research.

A modern Python interface for interacting with OpenFOAM

generate HPC scheduler systems jobs input scripts and submit these scripts to HPC systems and poke until they finish

☁️ Export Ploomber pipelines to Kubernetes (Argo), Airflow, AWS Batch, SLURM, and Kubeflow.