[Feature]: Error when creating collection: schema does not contain vector field: invalid parameter

[lynnssi]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem? Please describe.

In Milvus 2.4.1, I am able to create collections that do not have a vector field in the schema. When I upgraded to 2.4.4, I encountered the error RPC error: [create_collection], <MilvusException: (code=1100, message=schema does not contain vector field: invalid parameter)>.

I would like to be able to use this functionality of vector-less collections in future versions of Milvus

Example code

from pymilvus import MilvusClient
client_241 = MilvusClient(URL241)
client_244 = MilvusClient(URL244)

from pymilvus import DataType
schema = MilvusClient.create_schema(auto_id=False, enable_dynamic_field=True)
schema.add_field(field_name="key", datatype=DataType.VARCHAR, is_primary=True, max_length=20)
schema.add_field(field_name="text", datatype=DataType.VARCHAR, is_primary=False, max_length=1000)
schema.add_field(field_name="value", datatype=DataType.INT64, is_primary=False)

# Creates collection without error
client_241.create_collection("test_no_vector_field2", schema=schema)

# Fails to create collection
client_244.create_collection("test_no_vector_field", schema=schema)
>> RPC error: [create_collection], <MilvusException: (code=1100, message=schema does not contain vector field: invalid parameter)>

Describe the solution you'd like.

I would like to be able to create, insert into, and query collections that do not have a vector field in future versions of Milvus. This is very useful for the development of search applications.

Describe an alternate solution.

No response

Anything else? (Additional Context)

No response

[sunby]

Hi @lynnssi , may I ask that in what scenario would you need to use a vector database without requiring a vector field?
Milvus is a vector database and it's python sdk always check the vector field. So we add this check in milvus server side recently.

[junjie-landing]

I have the same issue. Reason is simple, I have duplicate metadata that I do not want to flatten/denormalized into the table, which makes it very redundant.

(It's bit like relational db. I know milvus is not meant for that, but I wouldn't want to put meta data in yet another RDBMS, which simplified increase our tech stack complexity.)

BTW, this is a breaking change to our techstack. To avoid reimplementing the workflow, I'm gonna add a dummy vector field and bypass for now.

[xiaofan-luan]

Milvus is not designed for this. Even though milvus will support "data in data out " feature soon, where vector field is not required any more, we still expect data to be embedded into a vector field.

Also, Milvus won't support any operation like Join so store meta into multiple collection should not be a solution

[franciscojavierarceo]

+1 to supporting this.

I'm adding Milvus to Feast (https://github.com/feast-dev/feast) and it'd be great if we could allow creating something without this strict requirement.

For now, I'm just going to insert a dummy column with a vector but it'd be ideal if users could do without this.

[xiaofan-luan]

not sure if milvus is the best option under that use case.

But maybe we can add a config to ignore the check here?

@liliu-z what do you think?

[xiaofan-luan]

what people can do is query with Match and some complicated filters

[franciscojavierarceo]

If a user chooses Milvus even though it may not be the optimal database to use, they should still be able to use it and they probably have good reasons for it.