Understanding the Difficulty of Training Transformers
-
Updated
May 31, 2022 - Python
#
initialization
Here are 17 public repositories matching this topic...
For an education purpose, from-scratch, single-file, python-only pose-graph optimization implementation
codegen slam rotation jacobian gtsam gauss-newton pgo initialization pose-graph-optimization cholmod chordal-distance symforce
-
Updated
Nov 6, 2024 - Python
Simple implementation of the LSUV initialization in keras
keras convolutional-networks deeplearning convolutional-neural-networks initialization lsuv lsuv-initialization
-
Updated
Jul 15, 2019 - Python
Simple implementation of the LSUV initialization in PyTorch
cnn pytorch convolutional-networks deeplearning convolutional-neural-networks initialization lsuv lsuv-initialization
-
Updated
Jan 16, 2024 - Python
Class Normalization for Continual Zero-Shot Learning
-
Updated
Feb 5, 2021 - Python
Short description for quick search
tensorflow dropout batch-normalization momentum softmax-regression l2-regularization rmsprop adam-optimizer mini-batch-gradient-descent bias-correction initialization gradient-check exponentially-weighted-averages learning-rate-decay
-
Updated
Jan 31, 2019 - Python
[NeurIPS 2022] Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again by Ajay Jaiswal*, Peihao Wang*, Tianlong Chen, Justin F Rousseau, Ying Ding, Zhangyang Wang
-
Updated
Nov 25, 2022 - Python
Structured Initialization for Attention in Vision Transformers
-
Updated
Apr 2, 2024 - Python
This repository is for numerical experiments with convolutional neural networks, in particular for testing initialization procedures. The training data is the CIFAR10 dataset and the CNN-architecture is the FitNet-1 from ROMERO, Adriana, et al. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014.
-
Updated
Jul 2, 2020 - Python
Warmup initialisation procedure for RNNs
-
Updated
Sep 3, 2023 - Python
This Python program lets you create and use a multi-sided die, just like the ones you’d use in board games. With this class, you can make dice with any number of sides, roll them to get a random number, and set or check the current value showing on the die. The program also includes a simple example to show how it works.
-
Updated
Jul 30, 2024 - Python
Odoo add-on that processes the content of one or more configuration folders.
-
Updated
Apr 2, 2025 - Python
Repo for the course project
-
Updated
May 4, 2022 - Python
Code to reproduce the paper "Deconstructing the Goldilocks Zone of Neural Network Initialization"
-
Updated
May 31, 2024 - Python
Supplementary code for the paper 'Are GATs Out of Balance?' to be published at NeurIPS 2023
-
Updated
Jan 10, 2024 - Python
My solution to an assignment on neural network initializers and optimizers. Contains some of the most popular approaches such as Xavier/He initialization and SGD, Momentum, AdaGrad, AdaDelta and Adam optimizers.
-
Updated
May 7, 2021 - Python
Improve this page
Add a description, image, and links to the initialization topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the initialization topic, visit your repo's landing page and select "manage topics."