Skip to content

Efficiently handle new data #3003

@murphp15

Description

@murphp15

What is the feature request? What problem does it solve?
We want to scrape jira or confluence on a periodic interval.
However we want to make sure that we don't republish all content to the vector store and instead only publish the latest changed data.

Defintion of done
After a first scrape of the datasource is done, every successive scrape should update the minimum amount of rows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestinitiative: VDK for Private AIInitiative including the effort to support Private AI usecases of VMWare with VDK

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions