Skip to content

Debug #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rayzzq wants to merge 30 commits into from
Closed

Debug #59

rayzzq wants to merge 30 commits into from

Conversation

@rayzzq
Copy link

@rayzzq rayzzq commented Jul 1, 2022

No description provided.

marxistz added 30 commits July 1, 2022 02:30
Add new chinese tokenizer (with jieba.cut())
b1d3e8b 
support both Bert, T5 and GPT
Add new chinese tokenizer (with jieba.cut())
d9d60d0 
support both Bert, T5 and GPT
Add new chinese tokenizer (with jieba.cut())
55a0562 
support both Bert, T5 and GPT
Add new chinese tokenizer (with jieba.cut())
e5e5b6c 
support both Bert, T5 and GPT
modify name
f089602
add preprocessor
f46ce72
modify
cd3238b
change requirements.txt
debbd6b 
add new scripts to preprocess chinese corpus
change requirements.txt
9e96add 
add new scripts to preprocess chinese corpus
modify requirements.txt
0748687
expose tokenizer_type outside
29ce2f0
tokenizer.vocab_size
55c4f9f
split docker run & exec
97c7301
print out
6cf4726
print out
9c21f11
print out
4050a3d
print out
1d6bce0
add right vocab
678eb31
add right vocab
0c44b98
debug
1ed8c1b
debug
bedb27e
debug
78bb4b9
debug
2d254bf
debug
5000ce3
debug
e00bd4d
debug
1093d71
debug
74a34c2
debug
374e3db
debug
8e1f628
debug
31fe5aa
@ghost
Copy link

ghost commented Jul 1, 2022

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

❌ rayzzq sign now
You have signed the CLA already but the status is still pending? Let us recheck it.

@rayzzq rayzzq closed this Jul 1, 2022
@rayzzq rayzzq deleted the debug branch July 1, 2022 08:40
saforem2 added a commit to saforem2/Megatron-DeepSpeed that referenced this pull request Oct 11, 2024
@saforem2
Merge pull request deepspeedai#59 from mngom2/spike-skipper
09accde 
Added Sophia Optimizer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeffra jeffra Awaiting requested review from jeffra jeffra is a code owner

@samyam samyam Awaiting requested review from samyam

@tjruwase tjruwase Awaiting requested review from tjruwase tjruwase is a code owner

@ShadenSmith ShadenSmith Awaiting requested review from ShadenSmith

@conglongli conglongli Awaiting requested review from conglongli

@awan-10 awan-10 Awaiting requested review from awan-10

@cli99 cli99 Awaiting requested review from cli99

@eltonzheng eltonzheng Awaiting requested review from eltonzheng

@minjiaz minjiaz Awaiting requested review from minjiaz

@RezaYazdaniAminabadi RezaYazdaniAminabadi Awaiting requested review from RezaYazdaniAminabadi

@duli2012 duli2012 Awaiting requested review from duli2012

@mrwyattii mrwyattii Awaiting requested review from mrwyattii

@yaozhewei yaozhewei Awaiting requested review from yaozhewei

@arashb arashb Awaiting requested review from arashb

@xiaoxiawu-microsoft xiaoxiawu-microsoft Awaiting requested review from xiaoxiawu-microsoft

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@rayzzq