Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model] PP support for embedding models and update docs #9090

Merged
merged 23 commits into from
Oct 6, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update imports
  • Loading branch information
DarkLight1337 committed Oct 5, 2024
commit 6222fdb2280c77513fead86529db58a649c50799
2 changes: 1 addition & 1 deletion vllm/model_executor/models/gemma2_embedding.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@

from vllm.attention import AttentionMetadata
from vllm.model_executor.layers.pooler import Pooler, PoolingType
from vllm.model_executor.models.gemma2 import Gemma2Model
from vllm.model_executor.pooling_metadata import PoolingMetadata
from vllm.sequence import IntermediateTensors, PoolerOutput

from .gemma2 import Gemma2Model
from .interfaces import SupportsPP


Expand Down
2 changes: 1 addition & 1 deletion vllm/model_executor/models/qwen2_rm.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from typing import Iterable, List, Optional, Tuple, Union

import torch
from qwen2 import Qwen2Model
from torch import nn
from transformers import Qwen2Config

Expand All @@ -17,7 +18,6 @@
from vllm.model_executor.layers.pooler import Pooler, PoolingType
from vllm.model_executor.layers.quantization import QuantizationConfig
from vllm.model_executor.model_loader.weight_utils import default_weight_loader
from vllm.model_executor.models.qwen2 import Qwen2Model
from vllm.model_executor.pooling_metadata import PoolingMetadata
from vllm.sequence import IntermediateTensors, PoolerOutput

Expand Down
Loading