Skip to content

Refactor Feast SDK test suite #1496

@woop

Description

@woop

Our current test suite has the following problems

  • Lack of generalization across different sources, stores, providers. Ideally we would have a high level set of e2e tests that use the user API to test different combination of source, stores, providers and other infra config. An example of the problem here is https://github.com/feast-dev/feast/blob/master/sdk/python/tests/test_historical_retrieval.py#L313 where we have a bigquery test and a parquet test.
  • Related to the above problem is the fact that we aren't reusing our setup/teardown for tests. Ideally the setup/teardown would be externalized and reusable across different kinds of tests (testing online retrieval, historical retrieval, materialization). Even better if we can just do the setup/teardown once, and then run a range of tests on the staged infrastructure.
  • Ideally we would have a single "kitchen sink" use case that can be applied to all tests, removing the need for test authors to think about what kind of data they need to use
  • We also need a place to add smaller tests, which are closer to unit tests. An example of this is testing whether entity dataframe inferencing works Infer entity dataframe event timestamp column #1495. Its not scalable to keep adding conditional flags and rerun the whole test suite. Eventually our integration tests will take too long to complete.
  • Lack of parallelism. We can run many of these tests async if they are well isolated.
  • Smaller point: We need to test all the types that we support. Currently we're not testing fringe types with our integration tests.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions