Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Feature/kafkaoptions kwargs #4902

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

zerafachris
Copy link

@zerafachris zerafachris commented Jan 6, 2025

What this PR does / why we need it:

This is an attempt to introduce additional kafka settings to the Kafka stream as raised by #4894

Which issue(s) this PR fixes:

Not applicable.

Misc

Unfortunately, this is a partial solution and am looking for support on this.

I have made most of the necessary changes but unit tests are failing due to:

===================================================================== short test summary info ======================================================================
FAILED sdk/python/tests/unit/test_stream_feature_view.py::test_stream_feature_view_serialization - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/test_stream_feature_view.py::test_stream_feature_view_udfs - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/test_stream_feature_view.py::test_stream_feature_view_initialization_with_optional_fields_omitted - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/local_feast_tests/test_local_feature_store.py::test_apply_stream_feature_view[feature_store_with_local_registry] - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/local_feast_tests/test_local_feature_store.py::test_apply_stream_feature_view_udf[feature_store_with_local_registry] - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/local_feast_tests/test_local_feature_store.py::test_apply_stream_source[feature_store_with_local_registry] - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/test_data_sources.py::test_proto_conversion - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/local_feast_tests/test_local_feature_store.py::test_apply_stream_source_from_repo - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/local_feast_tests/test_local_feature_store.py::test_apply_feature_view_with_inline_stream_source[feature_store_with_local_registry] - typeguard.TypeCheckError: argument "data_source" (feast.core.DataFormat_pb2.StreamFormat) is not an instance of feast.core.DataSource_pb2.DataSource
FAILED sdk/python/tests/unit/local_feast_tests/test_local_feature_store.py::test_apply_feature_view_with_inline_stream_source_from_repo - AssertionError: stdout: b'No project found in the repository. Using project name test5578hi0q50 defined in feature_store.yaml\nApplying changes for project tes...
=============================================== 10 failed, 519 passed, 2 skipped, 558 warnings in 234.66s (0:03:54) ================================================

Any support is greatly appreciated

Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
@zerafachris zerafachris requested a review from a team as a code owner January 6, 2025 21:11
Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
@zerafachris
Copy link
Author

With commit "1ef5d0afb35e3dd2f9f8aaee6da0a28f88982a82" I was able to add the proto definition for kafka_settings as a StreamFormat. Not sure if this is correct of not for dict[str, str]

Signed-off-by: zerafachris PERSONAL <zerafachris@gmail.com>
@zerafachris zerafachris changed the title WIP: feat: Feature/kafkaoptions kwargs feat: Feature/kafkaoptions kwargs Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants