internvit 300m pth文件

[awakenlee180]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

请问下有没有internvit300m的pth文件呀 我看classification下面的readme只给了6b的模型文件 如果我要自己训300m的vit要怎么做呀

Reproduction

aa

Environment

aa

Error traceback

[yuecao0119]

你好，

1. 请参考：https://huggingface.co/OpenGVLab/InternViT-300M-448px-V2_5
2. 我们的300M ViT由6B ViT蒸馏而来。

[xzq11111]

请问技术报告里面提到的最后k层参数负余弦相似度loss怎么操作的呢？6B模型和300M模型的embedding dim是不一样的呀，没法计算余弦相似度吧？
“We align the representations of the student model with those of the teacher model by computing the negative cosine similarity loss between the hidden states of the last K transformer layers. The resulting model is named InternViT-300M.”