Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Improve quickstart to include push features, batch scoring, odfv #3034

Merged
merged 11 commits into from
Aug 18, 2022
Merged
Prev Previous commit
Next Next commit
add READMEs
Signed-off-by: Danny Chiao <danny@tecton.ai>
  • Loading branch information
adchia committed Aug 18, 2022
commit 50e0d8c0505ddc530ada7c2f1ad3abd0ad662027
2 changes: 1 addition & 1 deletion docs/reference/feature-servers/python-feature-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ There is a CLI command that starts the server: `feast serve`. By default, Feast

## Deploying as a service

One can deploy a feature server by building a docker image that bundles in the project's `feature_store.yaml`. See this [helm chart](https://github.com/feast-dev/feast/blob/master/infra/charts/feast-python-server) for an example on how to run Feast on Kubernetes.
One can deploy a feature server by building a docker image that bundles in the project's `feature_store.yaml`. See this [helm chart](https://github.com/feast-dev/feast/blob/master/infra/charts/feast-feature-server) for an example on how to run Feast on Kubernetes.

A [remote feature server](alpha-aws-lambda-feature-server.md) on AWS Lambda is also available.

Expand Down
19 changes: 19 additions & 0 deletions sdk/python/feast/templates/aws/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Feast Quickstart
A quick view of what's in this repository:

* `data/` contains raw demo parquet data
* `example_repo.py` contains demo feature definitions
* `feature_store.yaml` contains a demo setup configuring where data sources are
* `test_workflow.py` showcases how to run all key Feast commands, including defining, retrieving, and pushing features.

You can run the overall workflow with `python test_workflow.py`.

## To move from this into a more production ready workflow:
1. `feature_store.yaml` points to a local file as a registry. You'll want to setup a remote file (e.g. in S3/GCS) or a
SQL registry. See [registry docs](https://docs.feast.dev/getting-started/concepts/registry) for more details.
2. Setup CI/CD + dev vs staging vs prod environments to automatically update the registry as you change Feast feature definitions. See [docs](https://docs.feast.dev/how-to-guides/running-feast-in-production#1.-automatically-deploying-changes-to-your-feature-definitions).
3. (optional) Regularly scheduled materialization to power low latency feature retrieval (e.g. via Airflow). See [Batch data ingestion](https://docs.feast.dev/getting-started/concepts/data-ingestion#batch-data-ingestion)
for more details.
4. (optional) Deploy feature server instances with `feast serve` to expose endpoints to retrieve online features.
- See [Python feature server](https://docs.feast.dev/reference/feature-servers/python-feature-server) for details.
- Use cases can also directly call the Feast client to fetch features as per [Feature retrieval](https://docs.feast.dev/getting-started/concepts/feature-retrieval)
20 changes: 9 additions & 11 deletions sdk/python/feast/templates/gcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,13 @@ A quick view of what's in this repository:
You can run the overall workflow with `python test_workflow.py`.

## To move from this into a more production ready workflow:
adchia marked this conversation as resolved.
Show resolved Hide resolved
1. First: you should start with a different Feast template, which delegates to a more scalable offline store.
- For example, running `feast init -t gcp`
or `feast init -t aws` or `feast init -t snowflake`.
- You can see your options if you run `feast init --help`.
2. `feature_store.yaml` points to a local file as a registry. You'll want to setup a remote file (e.g. in S3/GCS) or a
1. `feature_store.yaml` points to a local file as a registry. You'll want to setup a remote file (e.g. in S3/GCS) or a
SQL registry. See [registry docs](https://docs.feast.dev/getting-started/concepts/registry) for more details.
3. This example uses a file [offline store](https://docs.feast.dev/getting-started/architecture-and-components/offline-store)
to generate training data. It does not scale. We recommend instead using a data warehouse such as BigQuery,
Snowflake, Redshift. There is experimental support for Spark as well.
4. Setup CI/CD + dev vs staging vs prod environments to automatically update the registry as you change Feast feature definitions. See [docs](https://docs.feast.dev/how-to-guides/running-feast-in-production#1.-automatically-deploying-changes-to-your-feature-definitions).
5. (optional) Regularly scheduled materialization to power low latency feature retrieval (e.g. via Airflow). See [Batch data ingestion](https://docs.feast.dev/getting-started/concepts/data-ingestion#batch-data-ingestion)
for more details.
2. This example uses an already setup BigQuery Feast data warehouse as the [offline store](https://docs.feast.dev/getting-started/architecture-and-components/offline-store)
to generate training data. You'll need to connect your own BigQuery instance to make this work.
3. Setup CI/CD + dev vs staging vs prod environments to automatically update the registry as you change Feast feature definitions. See [docs](https://docs.feast.dev/how-to-guides/running-feast-in-production#1.-automatically-deploying-changes-to-your-feature-definitions).
4. (optional) Regularly scheduled materialization to power low latency feature retrieval (e.g. via Airflow). See [Batch data ingestion](https://docs.feast.dev/getting-started/concepts/data-ingestion#batch-data-ingestion)
for more details.
5. (optional) Deploy feature server instances with `feast serve` to expose endpoints to retrieve online features.
- See [Python feature server](https://docs.feast.dev/reference/feature-servers/python-feature-server) for details.
- Use cases can also directly call the Feast client to fetch features as per [Feature retrieval](https://docs.feast.dev/getting-started/concepts/feature-retrieval)
5 changes: 4 additions & 1 deletion sdk/python/feast/templates/local/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,7 @@ SQL registry. See [registry docs](https://docs.feast.dev/getting-started/concept
Snowflake, Redshift. There is experimental support for Spark as well.
4. Setup CI/CD + dev vs staging vs prod environments to automatically update the registry as you change Feast feature definitions. See [docs](https://docs.feast.dev/how-to-guides/running-feast-in-production#1.-automatically-deploying-changes-to-your-feature-definitions).
5. (optional) Regularly scheduled materialization to power low latency feature retrieval (e.g. via Airflow). See [Batch data ingestion](https://docs.feast.dev/getting-started/concepts/data-ingestion#batch-data-ingestion)
for more details.
for more details.
6. (optional) Deploy feature server instances with `feast serve` to expose endpoints to retrieve online features.
- See [Python feature server](https://docs.feast.dev/reference/feature-servers/python-feature-server) for details.
- Use cases can also directly call the Feast client to fetch features as per [Feature retrieval](https://docs.feast.dev/getting-started/concepts/feature-retrieval)
19 changes: 19 additions & 0 deletions sdk/python/feast/templates/snowflake/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Feast Quickstart
A quick view of what's in this repository:

* `data/` contains raw demo parquet data
* `example_repo.py` contains demo feature definitions
* `feature_store.yaml` contains a demo setup configuring where data sources are
* `test_workflow.py` showcases how to run all key Feast commands, including defining, retrieving, and pushing features.

You can run the overall workflow with `python test_workflow.py`.

## To move from this into a more production ready workflow:
1. `feature_store.yaml` points to a local file as a registry. You'll want to setup a remote file (e.g. in S3/GCS) or a
SQL registry. See [registry docs](https://docs.feast.dev/getting-started/concepts/registry) for more details.
2. Setup CI/CD + dev vs staging vs prod environments to automatically update the registry as you change Feast feature definitions. See [docs](https://docs.feast.dev/how-to-guides/running-feast-in-production#1.-automatically-deploying-changes-to-your-feature-definitions).
3. (optional) Regularly scheduled materialization to power low latency feature retrieval (e.g. via Airflow). See [Batch data ingestion](https://docs.feast.dev/getting-started/concepts/data-ingestion#batch-data-ingestion)
for more details.
4. (optional) Deploy feature server instances with `feast serve` to expose endpoints to retrieve online features.
- See [Python feature server](https://docs.feast.dev/reference/feature-servers/python-feature-server) for details.
- Use cases can also directly call the Feast client to fetch features as per [Feature retrieval](https://docs.feast.dev/getting-started/concepts/feature-retrieval)