Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Feature Tables API to Core & Python SDK #1019

Merged
merged 53 commits into from
Oct 2, 2020
Merged

Add Feature Tables API to Core & Python SDK #1019

merged 53 commits into from
Oct 2, 2020

Conversation

mrzzy
Copy link
Collaborator

@mrzzy mrzzy commented Sep 29, 2020

What this PR does / why we need it:
Implements the Feature Table API as defined in the 0.8 RFC

  • Adds Feature Tables API to Core: ApplyFeatureTable(), ListFeatureTables(), GetFeatureTable()
  • Add Feature Tables support to Python SDK: apply(), list_feature_tables(), get_feature_table() (credits to @terryyylim).

Does this PR introduce a user-facing change?:

Add Feature Table apply, list, get to Feast Core and Python SDK.

}

message ListFeatureTablesRequest {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we dropped wildcard search? it was used in JobController to retrieve all feature-sets by store subscription and I guess it could be useful for JobService as well if we won't drop subscription mechanism in stores.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@woop will we support several online stores with different subscriptions in new architecture?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you mean we should support single project, wildcard feature table name matches?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only have a single online store. There is no such thing as a store subscription any more.

Copy link
Member

@woop woop Sep 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we implement a stream sink in the future, then we will need to create some kind of ingestion process that hooks up to a stream based on configuration to update an online store. But Feast doesnt know about these in the form of subscriptions. There is also no call to Feast Core to get this information.

protected static final Set<String> RESERVED_NAMES =
Set.of("created_timestamp", "event_timestamp");

public static void validateSpec(FeatureTableSpec spec) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it check for empty features/entities list?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added check in same file.

@terryyylim
Copy link
Member

/retest

1 similar comment
@terryyylim
Copy link
Member

/retest

private String fieldMapJSON;

@Column(name = "ts_column")
private String tsColumn;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we make this timestamp_column maybe?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace all ts_column/tsColumn with timestamp_column/timestampColumn respectively.

mrzzy and others added 22 commits October 2, 2020 08:53
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
…lds instead of just for feature.

Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
Signed-off-by: Terence <terencelimxp@gmail.com>
@feast-ci-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mrzzy, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@woop
Copy link
Member

woop commented Oct 2, 2020

/lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants