[LLM] Support pre_caches input of llama #6900
Conversation
|
Thanks for your contribution! |
|
root seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
| # "类型#裙*材质#针织*颜色#纯色*风格#复古*风格#文艺*风格#简约*图案#格子*图案#纯色*图案#复古*裙型#背带裙*裙长#连衣裙*裙领型", | ||
| "类型#上衣*材质#牛仔布*颜色#白色*风格#简约*图案#刺绣*衣样式#外套*衣款式#破洞", | ||
| "类型#上衣*风格#嘻哈*图案#卡通*图案#印花*图案#撞色*衣样式#卫衣*衣款式#连帽", |
| @@ -399,13 +404,16 @@ def dybatch_preprocess( | |||
|
|
|||
| inputs = {} | |||
| pad_token_id = tokenizer([tokenizer.pad_token], return_tensors="np")["input_ids"][0][-1] | |||
| pad_token_id = 0 | |||
There was a problem hiding this comment.
这里的 hard-code 还是删掉吧，后面通过 tokenizer config 文件来调整比较好=
| @@ -455,6 +458,7 @@ def forward( | |||
| q_out, k_out, v_out = qkv_transpose_split( | |||
| qkv_out, padding_offset, seq_lens, input_ids, self.num_heads // self.nranks, self.head_dim | |||
| ) | |||
| # import pdb; pdb.set_trace() | |||
| print("cum_offsets", cum_offsets) | ||
| print("padding_offset", padding_offset) | ||
|
|
||
| # if is_decoder: | ||
| # import pdb; pdb.set_trace() |
|
This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动，被标记为stale。 |
|
This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动，被标记为stale。 |
|
This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动，被标记为stale。 |
|
This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动，被标记为stale。 |
PR types
Function optimization
PR changes
Others
Description
Support pre_caches input of llama