[Misc]: duplicate BaiChuanForCausalLM

[youkaichao]

Anything you want to discuss about vllm.

vllm/vllm/model_executor/models/baichuan.py

Lines 434 to 463 in e7116c0

	class BaichuanForCausalLM(BaiChuanBaseForCausalLM):
	"""Baichuan 13B and Baichuan2 7B/13B."""
	def __init__(
	self,
	config: PretrainedConfig,
	cache_config: Optional[CacheConfig] = None,
	quant_config: Optional[QuantizationConfig] = None,
	lora_config: Optional[LoRAConfig] = None,
	):
	if config.hidden_size == 4096: # baichuan2 7b
	super().__init__(config, "ROPE", cache_config, quant_config,
	lora_config)
	else: # baichuan 13b, baichuan2 13b
	super().__init__(config, "ALIBI", cache_config, quant_config,
	lora_config)
	class BaiChuanForCausalLM(BaiChuanBaseForCausalLM):
	"""Baichuan 7B."""
	def __init__(
	self,
	config: PretrainedConfig,
	cache_config: Optional[CacheConfig] = None,
	quant_config: Optional[QuantizationConfig] = None,
	lora_config: Optional[LoRAConfig] = None,
	):
	super().__init__(config, "ROPE", cache_config, quant_config,
	lora_config)

BaichuanForCausalLM is defined twice, and the second one will overwrite the first one. which one should we keep?

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

Probably keep the first one? Since the 7B case in the first one also works for the second one. (Baichuan 7B config)

Remember that we'll have to update the registry accordingly to redirect HF BaiChuanForCausalLM to vLLM BaichuanForCausalLM.

[youkaichao]

credit to @simon-mo

there’s one that is lower case and the other upper case C in Chuan.

[DarkLight1337]

Oh, I thought you meant merging the classes together.