Skip to content

Refine Data Access using OpeaStore #1916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
joshuayao merged 12 commits into from
Sep 1, 2025
Merged

Refine Data Access using OpeaStore #1916

joshuayao merged 12 commits into from
Sep 1, 2025

Conversation

@joshuayao
Copy link
Collaborator

@joshuayao joshuayao commented Aug 29, 2025 •
edited
Loading

Description

Improve the modularity, flexibility, and maintainability of the data access layer for the Chat History, Feedback Management, and Prompt Registry microservices using OpeaStore. Support MongoDB in this PR.

  1. Porting the data access layer to OpeaStore: Migrating the existing code for data operations (like create, read, update, and delete) from the three microservices to use the OpeaStore. This ensures consistency and leverages the benefits of the shared data layer.
  2. Adding OpeaStore tools: A store factory streamlines the creation of a connection for different databases and tools for resolving database differentiations like ODM.
  3. Decoupling user data structures: Separating the application's data models from OpeaStore data layer implementations.
  4. Adding a search by KV API: A Key-Value (KV) API is a more common way to retrieve data in data access layer. Separating the application's search logic from OpeaStore data layer implementations like "get doc of a specific user".
  5. Updating documentation

Issues

#1578

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)
  • Others (enhancement, documentation, validation, etc.)

@joshuayao
Porting the data store layers of components to OpeaStore.
1117546 
Signed-off-by: Yi Yao <yi.a.yao@intel.com>
@joshuayao joshuayao linked an issue Aug 29, 2025 that may be closed by this pull request
[Feature] Porting ChatHistory, FeedbackManagement, and PromptRegistry to OPEAStore #1578
Closed
@joshuayao joshuayao changed the title Porting the data store layers of components to OpeaStore. [WIP] Porting the data store layers of components to OpeaStore. Aug 29, 2025
@joshuayao joshuayao added the WIP label Aug 29, 2025
@codecov
Copy link

codecov bot commented Sep 1, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 17.89474% with 78 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
comps/cores/storages/stores.py 0.00% 57 Missing ⚠️
comps/cores/storages/mongodb.py 44.44% 20 Missing ⚠️
comps/cores/common/storage.py 50.00% 1 Missing ⚠️
Files with missing lines Coverage Δ
comps/cores/common/storage.py 58.18% <50.00%> (-0.31%) ⬇️
comps/cores/storages/mongodb.py 51.65% <44.44%> (-11.87%) ⬇️
comps/cores/storages/stores.py 0.00% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joshuayao and others added 3 commits September 1, 2025 14:19
@joshuayao
Fix CI issues.
fba4165 
Signed-off-by: Yi Yao <yi.a.yao@intel.com>
@joshuayao
Merge branch 'josh/opea_store' of https://github.com/opea-project/Gen…
a8f0ec1 
…AIComps into josh/opea_store

Signed-off-by: Yi Yao <yi.a.yao@intel.com>
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
b8916c3 
for more information, see https://pre-commit.ci
@joshuayao joshuayao mentioned this pull request Sep 1, 2025
Closed
4 tasks
@joshuayao
Fix CI issues
601c0e1 
Signed-off-by: Yi Yao <yi.a.yao@intel.com>
@joshuayao joshuayao changed the title [WIP] Porting the data store layers of components to OpeaStore. Refine Data Access using OpeaStore Sep 1, 2025
@joshuayao joshuayao removed the WIP label Sep 1, 2025
@joshuayao joshuayao merged commit 82ba539 into main Sep 1, 2025
26 checks passed
@joshuayao joshuayao deleted the josh/opea_store branch September 1, 2025 13:30
jilongW pushed a commit to jilongW/GenAIComps that referenced this pull request Dec 18, 2025
@joshuayao @pre-commit-ci @jlwang1996
Refine Data Access using OpeaStore (opea-project#1916)
6913d91 
* Porting the data store layers of components to OpeaStore.
* Decoupling user-defined data structure from data access layer of MongoDB.
* Adding OpeaStore tools
* Add search by KV API
* Update READMEs

---------

Signed-off-by: Yi Yao <yi.a.yao@intel.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: sunzhonghua2004 <137033036@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ZePan110 ZePan110 ZePan110 approved these changes

@letonghan letonghan letonghan approved these changes

@hteeyeoh hteeyeoh Awaiting requested review from hteeyeoh hteeyeoh is a code owner

@WenjiaoYue WenjiaoYue Awaiting requested review from WenjiaoYue WenjiaoYue is a code owner

@lvliang-intel lvliang-intel Awaiting requested review from lvliang-intel lvliang-intel is a code owner

@ftian1 ftian1 Awaiting requested review from ftian1 ftian1 is a code owner

@Spycsh Spycsh Awaiting requested review from Spycsh Spycsh is a code owner

@lkk12014402 lkk12014402 Awaiting requested review from lkk12014402 lkk12014402 is a code owner

@minmin-intel minmin-intel Awaiting requested review from minmin-intel minmin-intel is a code owner

@yogeshmpandey yogeshmpandey Awaiting requested review from yogeshmpandey yogeshmpandey is a code owner

@chensuyue chensuyue Awaiting requested review from chensuyue chensuyue is a code owner

@rbrugaro rbrugaro Awaiting requested review from rbrugaro rbrugaro is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature] Porting ChatHistory, FeedbackManagement, and PromptRegistry to OPEAStore

3 participants

@joshuayao @ZePan110 @letonghan