The collect of the vector knowledge base is deleted

[xtuhcy]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.14

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

✔️ Expected Behavior

Updating an empty file does not cause the entire vector knowledge base to be deleted

❌ Actual Behavior

Updating an empty file causes the entire vector knowledge base to be deleted

[dosubot]

@xtuhcy Hello there! I'm here to help you with any bugs, questions, or contributions while you wait for a human maintainer. Let's tackle this together!

The issue you're encountering, where updating an empty file causes the entire vector knowledge base to be deleted, is likely due to the logic in the QAIndexProcessor class's clean method. When node_ids is not provided, the delete method of the Vector class is called, which deletes the entire vector knowledge base.

Here are the relevant parts of the code:

def clean(self, dataset: Dataset, node_ids: Optional[list[str]], with_keywords: bool = True):
    vector = Vector(dataset)
    if node_ids:
        vector.delete_by_ids(node_ids)
    else:
        vector.delete()

To prevent this, you should ensure that the node_ids parameter is correctly populated when calling the clean method. Additionally, you might want to add extra checks to avoid calling the delete method when dealing with empty files.

For logging and tracking, the document_indexing_update_task.py file already includes logging mechanisms to track the update operations and identify when an empty file update leads to the deletion of the vector knowledge base. These logs can help you diagnose and prevent such issues in the future ^[1].

If you need to implement additional safeguards, consider adding conditions to check for empty files explicitly before proceeding with the update or deletion operations.

To continue talking to Dosu, mention @dosu.

[crazywoola]

Link #5882