运行chinese-text-correction-7b模型，出现报错 #528
Description
Describe the bug
环境搭建完成，运行第一遍的时候可以跑出结果。后续就一直运行不出来，一般会卡在下面这个状态中。下面状态提醒是否不用管？？
Generating outputs: 0%| | 0/1 [00:00<?, ?it/s]The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
后续出来的报错情况：
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
../aten/src/ATen/native/cuda/TensorCompare.cu:110: _assert_async_cuda_kernel: block: [0,0,0], thread: [0,0,0] Assertion probability tensor contains either inf, nan or element < 0 failed.
Generating outputs: 0%| | 0/1 [00:16<?, ?it/s]
Traceback (most recent call last):
File "pycorrector-master/examples/gpt/demo.py", line 30, in
batch_res = m.correct_batch(error_sentences, system_prompt="你是一个中文文本纠错助手。请根据用户提供的原始文本，生成纠正后的文本。")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/pycorrector-master/examples/gpt/../../pycorrector/gpt/gpt_corrector.py", line 73, in correct_batch
corrected_sents = self.predict(
^^^^^^^^^^^^^
File "/opt/anaconda3/envs/pycorrect/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/pycorrector-master/examples/gpt/../../pycorrector/gpt/gpt_model.py", line 575, in predict
outputs = self.model.generate(input_ids, **generation_kwargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/pycorrect/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/pycorrect/lib/python3.12/site-packages/transformers/generation/utils.py", line 2215, in generate
result = self._sample(
^^^^^^^^^^^^^
File "/opt/anaconda3/envs/pycorrect/lib/python3.12/site-packages/transformers/generation/utils.py", line 3195, in _sample
while self._has_unfinished_sequences(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/pycorrect/lib/python3.12/site-packages/transformers/generation/utils.py", line 2413, in _has_unfinished_sequences
elif this_peer_finished:
^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.