Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

License update #726

Merged
merged 91 commits into from
Dec 4, 2022
Merged
Changes from 1 commit
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
3378791
Update utils.py
StellaAthena Dec 2, 2022
7d6af9e
Update text_generation_utils.py
StellaAthena Dec 2, 2022
cf1821e
Update learning_rates.py
StellaAthena Dec 2, 2022
be581d7
Update optimizers.py
StellaAthena Dec 2, 2022
9eb8504
Update samplers.py
StellaAthena Dec 2, 2022
bafeac5
Update blendable_dataset.py
StellaAthena Dec 2, 2022
2d350d0
Update data_utils.py
StellaAthena Dec 2, 2022
d06af2e
Update logging.py
StellaAthena Dec 2, 2022
1ba636a
Update gradient_noise_scale.py
StellaAthena Dec 2, 2022
99fac76
Update activations.py
StellaAthena Dec 2, 2022
1d06433
Update fused_softmax.py
StellaAthena Dec 2, 2022
8e427a2
Update gmlp.py
StellaAthena Dec 2, 2022
38f0627
Update fused_bias_dropout.py
StellaAthena Dec 2, 2022
f110e5e
Update indexed_dataset.py
StellaAthena Dec 2, 2022
05048b6
Update cross_entropy.py
StellaAthena Dec 2, 2022
2228f68
Update random.py
StellaAthena Dec 2, 2022
68c9c30
Update mappings.py
StellaAthena Dec 2, 2022
a14a9c4
Update logging.py
StellaAthena Dec 2, 2022
c718158
Update utils.py
StellaAthena Dec 2, 2022
2699a35
Update training.py
StellaAthena Dec 2, 2022
f3b3d05
Update text_generation_utils.py
StellaAthena Dec 2, 2022
c98583b
Update optimizers.py
StellaAthena Dec 2, 2022
4ecac63
Update learning_rates.py
StellaAthena Dec 2, 2022
25aae1c
Update initialize.py
StellaAthena Dec 2, 2022
c5b3f69
Update checkpointing.py
StellaAthena Dec 2, 2022
6fe08f4
Update blendable_dataset.py
StellaAthena Dec 2, 2022
8e81a00
Update data_utils.py
StellaAthena Dec 2, 2022
e90aac1
Update gpt2_dataset.py
StellaAthena Dec 2, 2022
e5e3b51
Update samplers.py
StellaAthena Dec 2, 2022
a9e0627
Update indexed_dataset.py
StellaAthena Dec 2, 2022
1419984
Update gradient_noise_scale.py
StellaAthena Dec 2, 2022
d0b18f6
Update activations.py
StellaAthena Dec 2, 2022
187e26e
Update fused_bias_dropout.py
StellaAthena Dec 2, 2022
ee31272
Update fused_softmax.py
StellaAthena Dec 2, 2022
f721239
Update gmlp.py
StellaAthena Dec 2, 2022
6269ea4
Update gpt2_model.py
StellaAthena Dec 2, 2022
f8cec5c
Update gpt2_model.py
StellaAthena Dec 2, 2022
af6116a
Update init_functions.py
StellaAthena Dec 2, 2022
f6c0b77
Update norms.py
StellaAthena Dec 2, 2022
965fb45
Update positional_embeddings.py
StellaAthena Dec 2, 2022
cda0145
Update transformer.py
StellaAthena Dec 2, 2022
0d9b120
Update utils.py
StellaAthena Dec 2, 2022
c088beb
Update word_embeddings.py
StellaAthena Dec 2, 2022
2b1d89a
Update cross_entropy.py
StellaAthena Dec 2, 2022
70464bf
Update initialize.py
StellaAthena Dec 2, 2022
c8429c6
Update layers.py
StellaAthena Dec 2, 2022
509d3a5
Update mappings.py
StellaAthena Dec 2, 2022
0d8b529
Update random.py
StellaAthena Dec 2, 2022
be86957
Update utils.py
StellaAthena Dec 2, 2022
e8195e8
Update arguments.py
StellaAthena Dec 2, 2022
c97f9e3
Update deepspeed_args.py
StellaAthena Dec 2, 2022
de32371
Update neox_args.py
StellaAthena Dec 2, 2022
614f789
Update template.py
StellaAthena Dec 2, 2022
1107aa7
Update gpt2_tokenization.py
StellaAthena Dec 2, 2022
824f1b9
Update tokenizer.py
StellaAthena Dec 2, 2022
17bab03
Update train_tokenizer.py
StellaAthena Dec 2, 2022
0753751
Update eval_adapter.py
StellaAthena Dec 2, 2022
582d7ef
Update __init__.py
StellaAthena Dec 2, 2022
1f7f172
Update __init__.py
StellaAthena Dec 2, 2022
ecf4b88
Update test_fused_kernels.py
StellaAthena Dec 2, 2022
e24f8fc
Update test_model_checkpoint.py
StellaAthena Dec 2, 2022
78b1557
Update test_model_generation.py
StellaAthena Dec 2, 2022
2f981c3
Update test_model_instantiation.py
StellaAthena Dec 2, 2022
17026cc
Update test_model_train.py
StellaAthena Dec 2, 2022
9905158
Update test_neoxargs_commandline.py
StellaAthena Dec 2, 2022
1c8d49b
Update test_neoxargs_implementation.py
StellaAthena Dec 2, 2022
0242f7b
Update test_neoxargs_load.py
StellaAthena Dec 2, 2022
a76bb58
Update test_neoxargs_usage.py
StellaAthena Dec 2, 2022
3d714f0
Update test_train_base.yml
StellaAthena Dec 2, 2022
b9c49e3
Update common.py
StellaAthena Dec 2, 2022
7a1830d
Update pytest.ini
StellaAthena Dec 2, 2022
757bb79
Update convert_to_hf.py
StellaAthena Dec 2, 2022
0656b40
Update corpora.py
StellaAthena Dec 2, 2022
aa3df4d
Update inspect_checkpoints.py
StellaAthena Dec 2, 2022
42ad16b
Update merge20b.py
StellaAthena Dec 2, 2022
d9c0bac
Update merge_mp_partitions.py
StellaAthena Dec 2, 2022
d555b90
Update preprocess_data.py
StellaAthena Dec 2, 2022
0998643
Update sync.sh
StellaAthena Dec 2, 2022
5f2f16f
Update sync_cmd.sh
StellaAthena Dec 2, 2022
830d267
Update syncdir.sh
StellaAthena Dec 2, 2022
9c3b069
Update upload.py
StellaAthena Dec 2, 2022
57daa62
Update Dockerfile
StellaAthena Dec 2, 2022
1d89ae3
Update deepy.py
StellaAthena Dec 2, 2022
09cf01a
Update evaluate.py
StellaAthena Dec 2, 2022
c77c454
Update generate.py
StellaAthena Dec 2, 2022
f400d41
Update prepare_data.py
StellaAthena Dec 2, 2022
12045f2
Update train.py
StellaAthena Dec 2, 2022
0aadf62
Update deepy.py
StellaAthena Dec 2, 2022
3717176
Update NeoXArgs docs automatically
invalid-email-address Dec 2, 2022
94c66b3
Merge branch 'main' into license-fix
StellaAthena Dec 3, 2022
a1c275e
Update NeoXArgs docs automatically
invalid-email-address Dec 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update fused_bias_dropout.py
  • Loading branch information
StellaAthena authored Dec 2, 2022
commit 38f0627475f7031d736ef679b87d705e0abf8440
15 changes: 15 additions & 0 deletions megatron/model/fused_bias_dropout.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@

# Copyright (c) 2021, EleutherAI contributors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import torch
import torch.nn.functional as F
from typing import Optional
Expand Down