data-parallel

Paddle Large Scale Classification Tools，supports ArcFace, CosFace, PartialFC, Data Parallel + Model Parallel. Model includes ResNet, ViT, Swin, DeiT, CaiT, FaceViT, MoCo, MAE, ConvMAE, CAE.

bert_seq2seq的DDP版本，支持bert、roberta、nezha、t5、gpt2等模型，支持seq2seq、ner、关系抽取等任务，无需添加额外代码，轻松启动DDP多卡训练。

Multi-GPU parallelized YOLOv5 training and inference pipeline — data-parallel scaling and throughput tuning.

Sample Implementation of the paper "Adaptive Consensus Gradients Aggregation for Scaled Distributed Training".

This is the second assignment of the Big Data Systems course (CS 744) at UW Madison in Fall 2020.

JAX/Flax NNX training infrastructure: device meshes and SPMD sharding, an Orbax checkpoint store, early stopping and callbacks, W&B and MLflow logging.