Cuda-Kernels

This repository contains CUDA implementations of various algorithms, primarily focused on deep learning.

I am currently ranked in the Top 20 (as of this commit) on leetgpu.com, and this repo contains the kernels I’ve implemented.

A Google Colab notebook is also provided for profiling using cudaEvent and Nsight Compute. Happy learning and profiling !

List of Implemented Kernels

Activations

• ReLU

Convolution

• 1D Convolution

Matrix Operations

• GEMM (General Matrix Multiplication)
• Transpose
• Vector Addition

Monte Carlo

• Monte Carlo Simulation

Norms

• Layer Normalization

RGB2Gray

• RGB to Grayscale

Reduction

• Reduction

Loss Functions

• Mean Square Error Loss
• Cross Entropy Loss

Softmax

• Softmax

Examples

• helloparallel.cu — Hello World on GPU