-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add Couchbase as an online store #4637
Merged
franciscojavierarceo
merged 11 commits into
feast-dev:master
from
ejscribner:couchbase-online-store
Nov 5, 2024
+827
−50
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
fd01f7b
Initial Couchbase Online Store
ejscribner 5cc42c8
Add Couchbase and Sync Deps
ejscribner f2e6f7d
Test Container using Docker, Add `kv_port` to RepoConfig
ejscribner d3b8141
Rename host to connection_string and expect couchbase://
ejscribner 0b872e5
Warnings and Docstrings
ejscribner a1614a4
Documentation
ejscribner b53130a
Remove Union
ejscribner 0a198cb
Adjust type for `kv_port`
ejscribner 76799f4
RST Docs
ejscribner a7bbf95
Refs in Docs
ejscribner d1ae5d9
Docs fix
ejscribner File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,78 @@ | ||
# Couchbase Online Store | ||
> NOTE: | ||
> This is a community-contributed online store that is in alpha development. It is not officially supported by the Feast project. | ||
|
||
## Description | ||
The [Couchbase](https://www.couchbase.com/) online store provides support for materializing feature values into a Couchbase Operational cluster for serving online features in real-time. | ||
|
||
* Only the latest feature values are persisted | ||
* Features are stored in a document-oriented format | ||
|
||
The data model for using Couchbase as an online store follows a document format: | ||
* Document ID: `{project}:{table_name}:{entity_key_hex}:{feature_name}` | ||
* Document Content: | ||
* `metadata`: | ||
* `event_ts` (ISO formatted timestamp) | ||
* `created_ts` (ISO formatted timestamp) | ||
* `feature_name` (String) | ||
* `value` (Base64 encoded protobuf binary) | ||
|
||
|
||
## Getting started | ||
In order to use this online store, you'll need to run `pip install 'feast[couchbase]'`. You can then get started with the command `feast init REPO_NAME -t couchbase`. | ||
|
||
To get started with Couchbase Capella Operational: | ||
1. [Sign up for a Couchbase Capella account](https://docs.couchbase.com/cloud/get-started/create-account.html#sign-up-free-tier) | ||
2. [Deploy an Operational cluster](https://docs.couchbase.com/cloud/get-started/create-account.html#getting-started) | ||
3. [Create a bucket](https://docs.couchbase.com/cloud/clusters/data-service/manage-buckets.html#add-bucket) | ||
- This can be named anything, but must correspond to the bucket described in the `feature_store.yaml` configuration file. | ||
4. [Create cluster access credentials](https://docs.couchbase.com/cloud/clusters/manage-database-users.html#create-database-credentials) | ||
- These credentials should have full access to the bucket created in step 3. | ||
5. [Configure allowed IP addresses](https://docs.couchbase.com/cloud/clusters/allow-ip-address.html) | ||
- You must allow the IP address of the machine running Feast. | ||
|
||
## Example | ||
{% code title="feature_store.yaml" %} | ||
```yaml | ||
project: my_feature_repo | ||
registry: data/registry.db | ||
provider: local | ||
online_store: | ||
type: couchbase | ||
connection_string: couchbase://127.0.0.1 # Couchbase connection string, copied from 'Connect' page in Couchbase Capella console | ||
user: Administrator # Couchbase username from access credentials | ||
password: password # Couchbase password from access credentials | ||
bucket_name: feast # Couchbase bucket name, defaults to feast | ||
kv_port: 11210 # Couchbase key-value port, defaults to 11210. Required if custom ports are used. | ||
entity_key_serialization_version: 2 | ||
``` | ||
{% endcode %} | ||
|
||
The full set of configuration options is available in `CouchbaseOnlineStoreConfig`. | ||
|
||
|
||
## Functionality Matrix | ||
The set of functionality supported by online stores is described in detail [here](overview.md#functionality). | ||
Below is a matrix indicating which functionality is supported by the Couchbase online store. | ||
|
||
| | Couchbase | | ||
| :-------------------------------------------------------- | :-------- | | ||
| write feature values to the online store | yes | | ||
| read feature values from the online store | yes | | ||
| update infrastructure (e.g. tables) in the online store | yes | | ||
| teardown infrastructure (e.g. tables) in the online store | yes | | ||
| generate a plan of infrastructure changes | no | | ||
| support for on-demand transforms | yes | | ||
| readable by Python SDK | yes | | ||
| readable by Java | no | | ||
| readable by Go | no | | ||
| support for entityless feature views | yes | | ||
| support for concurrent writing to the same key | yes | | ||
| support for ttl (time to live) at retrieval | no | | ||
| support for deleting expired data | no | | ||
| collocated by feature view | yes | | ||
| collocated by feature service | no | | ||
| collocated by entity key | no | | ||
|
||
To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix). | ||
|
21 changes: 21 additions & 0 deletions
21
...python/docs/source/feast.infra.online_stores.contrib.couchbase_online_store.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
feast.infra.online\_stores.contrib.couchbase\_online\_store package | ||
=================================================================== | ||
|
||
Submodules | ||
---------- | ||
|
||
feast.infra.online\_stores.contrib.couchbase\_online\_store.couchbase module | ||
---------------------------------------------------------------------------- | ||
|
||
.. automodule:: feast.infra.online_stores.contrib.couchbase_online_store.couchbase | ||
:members: | ||
:undoc-members: | ||
:show-inheritance: | ||
|
||
Module contents | ||
--------------- | ||
|
||
.. automodule:: feast.infra.online_stores.contrib.couchbase_online_store | ||
:members: | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wanted to confirm which ones to skip here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would just skip what other third party providers do