Implement Hive Unit Test Behavior #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
PTAL @wild-endeavor |
chanadian
reviewed
Aug 23, 2019
| @@ -1,4 +1,4 @@ | |||
| from __future__ import absolute_import | |||
| import flytekit.plugins | |||
|
|
|||
| __version__ = '0.1.5' | |||
| __version__ = '0.1.6' | |||
There was a problem hiding this comment.
you probably are doing this, but you can submit this after your other PR so you get both changes the version bump
There was a problem hiding this comment.
Yeah in the future I will bump the version via an explicit PR.
chanadian
reviewed
Aug 23, 2019
| def register(self, identifier, version): | ||
| raise _user_exceptions.FlyteAssertion("You cannot register unit test tasks.") | ||
|
|
||
|
|
||
| class ReturnOutputsTask(UnitTestEngineTask): | ||
| def execute(self, inputs, context=None): |
There was a problem hiding this comment.
It's a refactor so this method is sufficient for transforming the execute function of the super class to behave as we desire.
chanadian
approved these changes
Aug 23, 2019
max-hoffman
pushed a commit
to dolthub/flytekit
that referenced
this pull request
May 11, 2021
Implement Hive Unit Test Behavior
mayitbeegh
added a commit
that referenced
this pull request
Sep 10, 2021
Signed-off-by: Sean Lin <sean@union.ai>
edgar-t
added a commit
to edgar-t/flytekit
that referenced
this pull request
Sep 22, 2022
wild-endeavor
added a commit
that referenced
this pull request
Sep 26, 2023
eapolinario
added a commit
that referenced
this pull request
Feb 8, 2024
* temp Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * stuff Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * temp Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * scaffolding areas mostly identified Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add artifact to upload request Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove an unnecessary line in workflow Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * finish adding examples use cases maybe Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add project/dom to get query Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add from flyte idl Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add project domain to as query Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add condition in parameter to flyte idl Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * test stuff * Remove artifactID from literal oneof, add to metadata (#2) * Triggers (#6) * Minor changes to get time series example working #8 Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * switch channel (#10) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * fix tests ignore - pr into other pr (#1858) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Artf/update idl ux (#1920) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Artf/trigger (#1948) * Add triggers * Remove bind_partition_time and just assume users won't use that. It's just time_partition in the normal call function now. Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove the now deleted artifact spec (#1984) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Literal metadata model update (#2089) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Separate time partition (#2114) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Split service code (#2136) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove empty files Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add noneness check to metadata and add test Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove sandbox test for now Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Artf/cleanup (#2158) * add a test Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * try updates Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> --------- Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Use python 3.9 to run make doc-requirements.txt Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * reasons not msg Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> --------- Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
eapolinario
added a commit
that referenced
this pull request
Feb 8, 2024
* temp Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * stuff Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * temp Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * scaffolding areas mostly identified Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add artifact to upload request Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove an unnecessary line in workflow Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * finish adding examples use cases maybe Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add project/dom to get query Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add from flyte idl Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add project domain to as query Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add condition in parameter to flyte idl Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * test stuff * Remove artifactID from literal oneof, add to metadata (#2) * Triggers (#6) * Minor changes to get time series example working #8 Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * switch channel (#10) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * fix tests ignore - pr into other pr (#1858) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Artf/update idl ux (#1920) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Artf/trigger (#1948) * Add triggers * Remove bind_partition_time and just assume users won't use that. It's just time_partition in the normal call function now. Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove the now deleted artifact spec (#1984) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Literal metadata model update (#2089) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Separate time partition (#2114) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Split service code (#2136) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove empty files Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * add noneness check to metadata and add test Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * remove sandbox test for now Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Artf/cleanup (#2158) * add a test Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * try updates Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> --------- Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Use python 3.9 to run make doc-requirements.txt Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * reasons not msg Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> --------- Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
noahjax
referenced
this pull request
in dominodatalab/flytekit
Mar 18, 2024
mg515
added a commit
to mg515/flytekit
that referenced
this pull request
Apr 9, 2024
Signed-off-by: mg515 <miha.garafolj@gmail.com>
eapolinario
added a commit
that referenced
this pull request
Aug 1, 2024
* add flytekit-hydra Signed-off-by: mg515 <miha.garafolj@gmail.com> * fix small typo readme Signed-off-by: mg515 <miha.garafolj@gmail.com> * ruff ruff Signed-off-by: mg515 <miha.garafolj@gmail.com> * lint more Signed-off-by: mg515 <miha.garafolj@gmail.com> * rename plugin into flytekit-omegaconf Signed-off-by: mg515 <miha.garafolj@gmail.com> * lint sort imports Signed-off-by: mg515 <miha.garafolj@gmail.com> * use flytekit logger Signed-off-by: mg515 <miha.garafolj@gmail.com> * use flytekit logger #2 Signed-off-by: mg515 <miha.garafolj@gmail.com> * fix typing info in is_flatable Signed-off-by: mg515 <miha.garafolj@gmail.com> * use default_factory instead of mutable default value Signed-off-by: mg515 <miha.garafolj@gmail.com> * add python3.11 and python3.12 to setup.py Signed-off-by: mg515 <miha.garafolj@gmail.com> * make fmt Signed-off-by: mg515 <miha.garafolj@gmail.com> * define error message only once Signed-off-by: mg515 <miha.garafolj@gmail.com> * add docstring Signed-off-by: mg515 <miha.garafolj@gmail.com> * remove GenericEnumTransformer and tests Signed-off-by: mg515 <miha.garafolj@gmail.com> * fallback to TypeEngine.get_transformer(node_type) to find suitable transformer Signed-off-by: mg515 <miha.garafolj@gmail.com> * explicit valueerrors instead of asserts Signed-off-by: mg515 <miha.garafolj@gmail.com> * minor style improvements Signed-off-by: mg515 <miha.garafolj@gmail.com> * remove obsolete warnings Signed-off-by: mg515 <miha.garafolj@gmail.com> * import flytekit logger instead of instantiating our own Signed-off-by: mg515 <miha.garafolj@gmail.com> * docstrings in reST format Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor transformer mode Signed-off-by: mg515 <miha.garafolj@gmail.com> * improve docs Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor dictconfig class into smaller methods Signed-off-by: mg515 <miha.garafolj@gmail.com> * add unit tests for dictconfig transformer Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor of parse_type_description() Signed-off-by: mg515 <miha.garafolj@gmail.com> * add omegaconf plugin to pythonbuild.yaml --------- Signed-off-by: mg515 <miha.garafolj@gmail.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
mao3267
pushed a commit
to mao3267/flytekit
that referenced
this pull request
Aug 2, 2024
* add flytekit-hydra Signed-off-by: mg515 <miha.garafolj@gmail.com> * fix small typo readme Signed-off-by: mg515 <miha.garafolj@gmail.com> * ruff ruff Signed-off-by: mg515 <miha.garafolj@gmail.com> * lint more Signed-off-by: mg515 <miha.garafolj@gmail.com> * rename plugin into flytekit-omegaconf Signed-off-by: mg515 <miha.garafolj@gmail.com> * lint sort imports Signed-off-by: mg515 <miha.garafolj@gmail.com> * use flytekit logger Signed-off-by: mg515 <miha.garafolj@gmail.com> * use flytekit logger flyteorg#2 Signed-off-by: mg515 <miha.garafolj@gmail.com> * fix typing info in is_flatable Signed-off-by: mg515 <miha.garafolj@gmail.com> * use default_factory instead of mutable default value Signed-off-by: mg515 <miha.garafolj@gmail.com> * add python3.11 and python3.12 to setup.py Signed-off-by: mg515 <miha.garafolj@gmail.com> * make fmt Signed-off-by: mg515 <miha.garafolj@gmail.com> * define error message only once Signed-off-by: mg515 <miha.garafolj@gmail.com> * add docstring Signed-off-by: mg515 <miha.garafolj@gmail.com> * remove GenericEnumTransformer and tests Signed-off-by: mg515 <miha.garafolj@gmail.com> * fallback to TypeEngine.get_transformer(node_type) to find suitable transformer Signed-off-by: mg515 <miha.garafolj@gmail.com> * explicit valueerrors instead of asserts Signed-off-by: mg515 <miha.garafolj@gmail.com> * minor style improvements Signed-off-by: mg515 <miha.garafolj@gmail.com> * remove obsolete warnings Signed-off-by: mg515 <miha.garafolj@gmail.com> * import flytekit logger instead of instantiating our own Signed-off-by: mg515 <miha.garafolj@gmail.com> * docstrings in reST format Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor transformer mode Signed-off-by: mg515 <miha.garafolj@gmail.com> * improve docs Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor dictconfig class into smaller methods Signed-off-by: mg515 <miha.garafolj@gmail.com> * add unit tests for dictconfig transformer Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor of parse_type_description() Signed-off-by: mg515 <miha.garafolj@gmail.com> * add omegaconf plugin to pythonbuild.yaml --------- Signed-off-by: mg515 <miha.garafolj@gmail.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com>
Future-Outlier
added a commit
that referenced
this pull request
Aug 26, 2024
…class] (#2603) * fix: set dataclass member as optional if default value is provided Signed-off-by: mao3267 <chenvincent610@gmail.com> * lint Signed-off-by: mao3267 <chenvincent610@gmail.com> * feat: handle nested dataclass conversion in JsonParamType Signed-off-by: mao3267 <chenvincent610@gmail.com> * fix: handle errors caused by NoneType default value Signed-off-by: mao3267 <chenvincent610@gmail.com> * test: add nested dataclass unit tests Signed-off-by: mao3267 <chenvincent610@gmail.com> * Sagemaker dict determinism (#2597) * truncate sagemaker agent outputs Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix tests and update agent output Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * lint Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix test Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add idempotence token to workflow Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix type Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix mixin Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * modify output handler Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * make the dictionary deterministic Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * nit Signed-off-by: Samhita Alla <aallasamhita@gmail.com> --------- Signed-off-by: Samhita Alla <aallasamhita@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * refactor(core): Enhance return type extraction logic (#2598) Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Feat: Make exception raised by external command authenticator more actionable (#2594) Signed-off-by: Fabio Grätz <fabiogratz@googlemail.com> Co-authored-by: Fabio Grätz <fabiogratz@googlemail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Fix: Properly re-raise non-grpc exceptions during refreshing of proxy-auth credentials in auth interceptor (#2591) Signed-off-by: Fabio Grätz <fabiogratz@googlemail.com> Co-authored-by: Fabio Grätz <fabiogratz@googlemail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * validate idempotence token length in subsequent tasks (#2604) * validate idempotence token length in subsequent tasks Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * remove redundant param Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add tests Signed-off-by: Samhita Alla <aallasamhita@gmail.com> --------- Signed-off-by: Samhita Alla <aallasamhita@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Add nvidia-l4 gpu accelerator (#2608) Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * eliminate redundant literal conversion for `Iterator[JSON]` type (#2602) * eliminate redundant literal conversion for type Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add test Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * lint Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add isclass check Signed-off-by: Samhita Alla <aallasamhita@gmail.com> --------- Signed-off-by: Samhita Alla <aallasamhita@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * [FlyteSchema] Fix numpy problems (#2619) Signed-off-by: Future-Outlier <eric901201@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * add nim plugin (#2475) * add nim plugin Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * move nim to inference Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * import fix Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix port Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add pod_template method Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add containers Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * update Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * clean up Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * remove cloud import Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix extra config Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * remove decorator Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add tests, update readme Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add env Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add support for lora adapter Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * minor fixes Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add startup probe Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * increase failure threshold Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * remove ngc secret group Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * move plugin to flytekit core Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix docs Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * remove hf group Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * modify podtemplate import Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix import Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix ngc api key Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix tests Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix formatting Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * lint Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * docs fix Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * docs fix Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * update secrets interface Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add secret prefix Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix tests Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add urls Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add urls Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * remove urls Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * minor modifications Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * remove secrets prefix; add failure threshold Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add hard-coded prefix Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * add comment Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * make secrets prefix a required param Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * move nim to flytekit plugin Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * update readme Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * update readme Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * update readme Signed-off-by: Samhita Alla <aallasamhita@gmail.com> --------- Signed-off-by: Samhita Alla <aallasamhita@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * [Elastic/Artifacts] Pass through model card (#2575) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Remove pyarrow as a direct dependency (#2228) Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Boolean flag to show local container logs to the terminal (#2521) Signed-off-by: aditya7302 <aditya7302@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Kevin Su <pingsutw@apache.org> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Enable Ray Fast Register (#2606) Signed-off-by: Jan Fiedler <jan@union.ai> Signed-off-by: mao3267 <chenvincent610@gmail.com> * [Artifacts/Elastic] Skip partitions (#2620) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Install flyteidl from master in plugins tests (#2621) Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Using ParamSpec to show underlying typehinting (#2617) Signed-off-by: JackUrb <jack@datologyai.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Support ArrayNode mapping over Launch Plans (#2480) * set up array node Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * wip array node task wrapper Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * support function like callability Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * temp check in some progress on python func wrapper Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * only support launch plans in new array node class for now Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * add map task array node implementation wrapper Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * ArrayNode only supports LPs for now Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * support local execute for new array node implementation Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * add local execute unit tests for array node Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * set exeucution version in array node spec Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * check input types for local execute Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * remove code that is un-needed for now Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * clean up array node class Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * improve naming Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * clean up Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * utilize enum execution mode to set array node execution path Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * default execution mode to FULL_STATE for new array node class Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * support min_successes for new array node Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * add map task wrapper unit test Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * set min successes for array node map task wrapper Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * update docstrings Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * Install flyteidl from master in plugins tests Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * lint Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * clean up min success/ratio setting Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * lint Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * make array node class callable Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> --------- Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Richer printing for some artifact objects (#2624) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * ci: Add Python 3.9 to build matrix (#2622) Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Future-Outlier <eric901201@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * bump (#2627) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Added alt prefix head to FlyteFile.new_remote (#2601) * Added alt prefix head to FlyteFile.new_remote Signed-off-by: pryce-turner <pryce.turner@gmail.com> * Added get_new_path method to FileAccessProvider, fixed new_remote method of FlyteFile Signed-off-by: pryce-turner <pryce.turner@gmail.com> * Updated tests and added new path creator to FlyteFile/Dir new_remote methods Signed-off-by: pryce-turner <pryce.turner@gmail.com> * Improved docstrings, fixed minor path sep bug, more descriptive naming, better test Signed-off-by: pryce-turner <pryce.turner@gmail.com> * Formatting Signed-off-by: pryce-turner <pryce.turner@gmail.com> --------- Signed-off-by: pryce-turner <pryce.turner@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Feature gate for FlyteMissingReturnValueException (#2623) Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Remove use of multiprocessing from the OAuth client (#2626) * Remove use of multiprocessing from the OAuth client Signed-off-by: Robert Deaton <robert.deaton@freenome.com> * Lint Signed-off-by: Robert Deaton <robert.deaton@freenome.com> --------- Signed-off-by: Robert Deaton <robert.deaton@freenome.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Update codespell in precommit to version 2.3.0 (#2630) Signed-off-by: mao3267 <chenvincent610@gmail.com> * Fix Snowflake Agent Bug (#2605) * fix snowflake agent bug Signed-off-by: Future-Outlier <eric901201@gmail.com> * a work version Signed-off-by: Future-Outlier <eric901201@gmail.com> * Snowflake work version Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix secret encode Signed-off-by: Future-Outlier <eric901201@gmail.com> * all works, I am so happy Signed-off-by: Future-Outlier <eric901201@gmail.com> * improve additional protocol Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix tests Signed-off-by: Future-Outlier <eric901201@gmail.com> * Fix Tests Signed-off-by: Future-Outlier <eric901201@gmail.com> * update agent Signed-off-by: Kevin Su <pingsutw@apache.org> * Add snowflake test Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * sd Signed-off-by: Kevin Su <pingsutw@apache.org> * snowflake loglinks Signed-off-by: Future-Outlier <eric901201@gmail.com> * add metadata Signed-off-by: Future-Outlier <eric901201@gmail.com> * secret Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * remove table Signed-off-by: Future-Outlier <eric901201@gmail.com> * add comment for get private key Signed-off-by: Future-Outlier <eric901201@gmail.com> * update comments: Signed-off-by: Future-Outlier <eric901201@gmail.com> * Fix Tests Signed-off-by: Future-Outlier <eric901201@gmail.com> * update comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * update comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * Better Secrets Signed-off-by: Future-Outlier <eric901201@gmail.com> * use union secret Signed-off-by: Future-Outlier <eric901201@gmail.com> * Update Changes Signed-off-by: Future-Outlier <eric901201@gmail.com> * use if not get_plugin().secret_requires_group() Signed-off-by: Future-Outlier <eric901201@gmail.com> * Use Union SDK Signed-off-by: Future-Outlier <eric901201@gmail.com> * Update Signed-off-by: Future-Outlier <eric901201@gmail.com> * Fix Secrets Signed-off-by: Future-Outlier <eric901201@gmail.com> * Fix Secrets Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove pacakge.json Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * add snowflake-connector-python Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix test_snowflake Signed-off-by: Future-Outlier <eric901201@gmail.com> * Try to fix tests Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix tests Signed-off-by: Future-Outlier <eric901201@gmail.com> * Try Fix snowflake Import Signed-off-by: Future-Outlier <eric901201@gmail.com> * snowflake test passed Signed-off-by: Future-Outlier <eric901201@gmail.com> --------- Signed-off-by: Future-Outlier <eric901201@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Kevin Su <pingsutw@apache.org> Signed-off-by: mao3267 <chenvincent610@gmail.com> * run test_missing_return_value on python 3.10+ (#2637) Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: mao3267 <chenvincent610@gmail.com> * [Elastic] Fix context usage and apply fix to fork method (#2628) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Add flytekit-omegaconf plugin (#2299) * add flytekit-hydra Signed-off-by: mg515 <miha.garafolj@gmail.com> * fix small typo readme Signed-off-by: mg515 <miha.garafolj@gmail.com> * ruff ruff Signed-off-by: mg515 <miha.garafolj@gmail.com> * lint more Signed-off-by: mg515 <miha.garafolj@gmail.com> * rename plugin into flytekit-omegaconf Signed-off-by: mg515 <miha.garafolj@gmail.com> * lint sort imports Signed-off-by: mg515 <miha.garafolj@gmail.com> * use flytekit logger Signed-off-by: mg515 <miha.garafolj@gmail.com> * use flytekit logger #2 Signed-off-by: mg515 <miha.garafolj@gmail.com> * fix typing info in is_flatable Signed-off-by: mg515 <miha.garafolj@gmail.com> * use default_factory instead of mutable default value Signed-off-by: mg515 <miha.garafolj@gmail.com> * add python3.11 and python3.12 to setup.py Signed-off-by: mg515 <miha.garafolj@gmail.com> * make fmt Signed-off-by: mg515 <miha.garafolj@gmail.com> * define error message only once Signed-off-by: mg515 <miha.garafolj@gmail.com> * add docstring Signed-off-by: mg515 <miha.garafolj@gmail.com> * remove GenericEnumTransformer and tests Signed-off-by: mg515 <miha.garafolj@gmail.com> * fallback to TypeEngine.get_transformer(node_type) to find suitable transformer Signed-off-by: mg515 <miha.garafolj@gmail.com> * explicit valueerrors instead of asserts Signed-off-by: mg515 <miha.garafolj@gmail.com> * minor style improvements Signed-off-by: mg515 <miha.garafolj@gmail.com> * remove obsolete warnings Signed-off-by: mg515 <miha.garafolj@gmail.com> * import flytekit logger instead of instantiating our own Signed-off-by: mg515 <miha.garafolj@gmail.com> * docstrings in reST format Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor transformer mode Signed-off-by: mg515 <miha.garafolj@gmail.com> * improve docs Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor dictconfig class into smaller methods Signed-off-by: mg515 <miha.garafolj@gmail.com> * add unit tests for dictconfig transformer Signed-off-by: mg515 <miha.garafolj@gmail.com> * refactor of parse_type_description() Signed-off-by: mg515 <miha.garafolj@gmail.com> * add omegaconf plugin to pythonbuild.yaml --------- Signed-off-by: mg515 <miha.garafolj@gmail.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Adds extra-index-url to default image builder (#2636) Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com> Co-authored-by: Kevin Su <pingsutw@apache.org> Signed-off-by: mao3267 <chenvincent610@gmail.com> * reference_task should inherit from PythonTask (#2643) Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: mao3267 <chenvincent610@gmail.com> * Fix Get Agent Secret Using Key (#2644) Signed-off-by: Future-Outlier <eric901201@gmail.com> Signed-off-by: mao3267 <chenvincent610@gmail.com> * fix: prevent converting Flyte types as custom dataclasses Signed-off-by: mao3267 <chenvincent610@gmail.com> * fix: add None to output type Signed-off-by: mao3267 <chenvincent610@gmail.com> * test: add unit test for nested dataclass inputs Signed-off-by: mao3267 <chenvincent610@gmail.com> * test: add unit tests for nested dataclass, dataclass default value as None, and flyte type exceptions Signed-off-by: mao3267 <chenvincent610@gmail.com> * fix: handle NoneType as default value of list type dataclass members Signed-off-by: mao3267 <chenvincent610@gmail.com> * fix: add comments for `has_nested_dataclass` function Signed-off-by: mao3267 <chenvincent610@gmail.com> * fix: make lint Signed-off-by: mao3267 <chenvincent610@gmail.com> * fix: update tests regarding input through file and pipe Signed-off-by: mao3267 <chenvincent610@gmail.com> * Make JsonParamType convert faster Signed-off-by: Future-Outlier <eric901201@gmail.com> * make has_nested_dataclass func more clean and add tests for dataclass_with_optional_fields Signed-off-by: Future-Outlier <eric901201@gmail.com> * make logic more backward compatible Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix: handle indexing errors in dict/list while checking nested dataclass, add comments Signed-off-by: mao3267 <chenvincent610@gmail.com> --------- Signed-off-by: mao3267 <chenvincent610@gmail.com> Co-authored-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Future-Outlier <eric901201@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hive unit tests are primarily interesting for validating the queries generated. This change surfaces the queries generated as the output of the
@hive_taskunit test.