Skip to content

Blip2ForConditionalGeneration的generate方法调用了不兼容ascend910b的接口 #1904

@Alemax067

Description

@Alemax067

Describe the bug/ 问题描述 (Mandatory / 必填)

Blip2ForConditionalGeneration的generate方法调用了不兼容ascend910b的接口,具体报错信息见最后截图:

  • Hardware Environment(Ascend/GPU/CPU) / 硬件环境:

Ascend 910b

  • Software Environment / 软件环境 (Mandatory / 必填):
    -- MindSpore version (e.g., 1.7.0.Bxxx) : 2.3.1和2.4.1都一样的问题
    -- Python version (e.g., Python 3.7.5) : 3.9
    -- OS platform and distribution (e.g., Linux Ubuntu 16.04): Euler2(用的是启智的AscendD910B 32G, mindtorch0.3_mindspore2.3.0_torchnpu2.2.0_cann8.0镜像)
    -- GCC/Compiler version (if compiled from source): 7.3.0
    -- MindNLP : 0.4.1(源码编译安装)

  • Excute Mode / 执行模式 (Mandatory / 必填)(PyNative/Graph):

Please delete the mode not involved / 请删除不涉及的模式:
pynative

To Reproduce / 重现步骤 (Mandatory / 必填)
Steps to reproduce the behavior:

  1. git clone https://openi.pcl.ac.cn/lvyufeng/mindnlp.git
  2. 按照issue1902中提到的bug,先将对应文件中的repeat(batch_size, 1)改为tile((batch_size, 1)),然后从源码编译安装mindnlp : bash scripts/build_and_reinstall.sh
  3. 下载Salesforce/blip2-opt-2.7b并移除processor_config.json文件(这里可能涉及另外的bug,不移除无法加载Blip2Processor)
  4. pip install --upgrade tokenizers==0.21.0
  5. pip install mindspore==2.3.1
  6. pip uninstall soundfile mindformers -y
  7. 运行如下代码即可看见报错:
from mindnlp.transformers import Blip2ForConditionalGeneration, Blip2Processor
import mindspore as ms
import mindspore.numpy as np

ms.set_context(device_target='Ascend', device_id=0, pynative_synchronize=True)
processor = Blip2Processor.from_pretrained(<path_to_blip2-opt-2.7b>)
model = Blip2ForConditionalGeneration.from_pretrained(<path_to_blip2-opt-2.7b>)

pixel_values = np.randn((16, 3, 224, 224))
generated_ids = model.generate(pixel_values)
print(generated_ids)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)

Expected behavior / 预期结果 (Mandatory / 必填)
预期正确产生generated_ids,但generated_ids = model.generate(pixel_values)报错

Screenshots/ 日志 / 截图 (Mandatory / 必填)

image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions