Support Identifier in generate_console_url #2868
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
thomasjpfan
requested review from
wild-endeavor,
kumare3,
eapolinario,
pingsutw,
cosmicBboy,
samhita-alla and
Future-Outlier
as code owners
dansola
approved these changes
Oct 25, 2024
pingsutw
approved these changes
Oct 25, 2024
eapolinario
approved these changes
Oct 25, 2024
Future-Outlier
pushed a commit
that referenced
this pull request
Oct 28, 2024
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Future-Outlier
added a commit
that referenced
this pull request
Oct 28, 2024
* Pydantic Transformer V2 Signed-off-by: Future-Outlier <eric901201@gmail.com> * add __init__.py Signed-off-by: Future-Outlier <eric901201@gmail.com> * add json schema Signed-off-by: Future-Outlier <eric901201@gmail.com> * convert float to int Signed-off-by: Future-Outlier <eric901201@gmail.com> * change gitsha in test script mode Signed-off-by: Future-Outlier <eric901201@gmail.com> * change gitsha Signed-off-by: Future-Outlier <eric901201@gmail.com> * use strict map=false Signed-off-by: Future-Outlier <eric901201@gmail.com> * Test flytefile console input + attr access Signed-off-by: Future-Outlier <eric901201@gmail.com> * add conditional branch Signed-off-by: Future-Outlier <eric901201@gmail.com> * better rx Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add flytedir generic -> flytedir Signed-off-by: Future-Outlier <eric901201@gmail.com> * support enum Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> * add tests for input from flyte console Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add Tests for dataclass in BaseModel and pydantic.dataclass in BaseModel Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> * update thomas's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * change tree file structure Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Niel's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * > to >= Signed-off-by: Future-Outlier <eric901201@gmail.com> * try monodoc build again Signed-off-by: Future-Outlier <eric901201@gmail.com> * add pydantic README.md Signed-off-by: Future-Outlier <eric901201@gmail.com> * revert -vvv in monodocs Signed-off-by: Future-Outlier <eric901201@gmail.com> * use model_validate_json to turn protobuf struct to python val Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix issue Signed-off-by: Future-Outlier <eric901201@gmail.com> * handle flyte types in dict transformer from protobuf struct input (e.g FlyteConsole) Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add print Signed-off-by: Future-Outlier <eric901201@gmail.com> * expected python type Signed-off-by: Future-Outlier <eric901201@gmail.com> * switch call function order Signed-off-by: Future-Outlier <eric901201@gmail.com> * try msgpack to handle protobug struct Signed-off-by: Future-Outlier <eric901201@gmail.com> * Better Comment in Dict Transformer Signed-off-by: Future-Outlier <eric901201@gmail.com> * Propeller -> FlytePropeller Signed-off-by: Future-Outlier <eric901201@gmail.com> * dict_to_flyte_types Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * add attr for protobuf struct . dict Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add Life Cycle for Flyte Types Signed-off-by: Future-Outlier <eric901201@gmail.com> * better comments for derializing flyteschema and sd Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * add back pv._remote_path = None to flytefile and flytedir Signed-off-by: Future-Outlier <eric901201@gmail.com> * experiment Signed-off-by: Future-Outlier <eric901201@gmail.com> * experiment Signed-off-by: Future-Outlier <eric901201@gmail.com> * experiment Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Yee's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * code dc -> bm Signed-off-by: Future-Outlier <eric901201@gmail.com> * Example dc -> bm and Example all flyte types Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix union dataclass, not yet add comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * add pydantic and dataclass optional test Signed-off-by: Future-Outlier <eric901201@gmail.com> * NoneType=type(None) Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix union transformer none case with Eduardo Signed-off-by: Future-Outlier <eric901201@gmail.com> * add comments for none type transformer + union transformer Signed-off-by: Future-Outlier <eric901201@gmail.com> * add TODO Signed-off-by: Future-Outlier <eric901201@gmail.com> * use deserailize = True Signed-off-by: Future-Outlier <eric901201@gmail.com> * add all deserialize Signed-off-by: Future-Outlier <eric901201@gmail.com> * better comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * better comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * test Signed-off-by: Future-Outlier <eric901201@gmail.com> * Fix flyte directory issue by discussion with Kevin Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> * add tests for providing conext when doing serialization Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * test Signed-off-by: Future-Outlier <eric901201@gmail.com> * move setattr to core Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove Signed-off-by: Future-Outlier <eric901201@gmail.com> * testing Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * flytekit/core/type_engine.py Signed-off-by: Future-Outlier <eric901201@gmail.com> * add tests for Union Signed-off-by: Future-Outlier <eric901201@gmail.com> * Trigger CI Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove nonetype Signed-off-by: Future-Outlier <eric901201@gmail.com> * raw=Fasle as default Signed-off-by: Future-Outlier <eric901201@gmail.com> * pydantic move to core test Signed-off-by: Future-Outlier <eric901201@gmail.com> * move to core Signed-off-by: Future-Outlier <eric901201@gmail.com> * log Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * test Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * move to type_engine Signed-off-by: Future-Outlier <eric901201@gmail.com> * move back to init Signed-off-by: Future-Outlier <eric901201@gmail.com> * update kevin's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> * wip Signed-off-by: Future-Outlier <eric901201@gmail.com> * use decorator Signed-off-by: Future-Outlier <eric901201@gmail.com> * decorator Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix syntax to support python 3.9 Signed-off-by: Future-Outlier <eric901201@gmail.com> * add Eduardo's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * warning Signed-off-by: Future-Outlier <eric901201@gmail.com> * Update Yee's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Show traceback by default (#2862) Signed-off-by: Kevin Su <pingsutw@apache.org> * Support Identifier in generate_console_url (#2868) Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com> * Support overriding node metadata for array node (#2865) * set parent node metadata on arraynode subnode Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * clean up setting passed in metadata for array node Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * add comment Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * test Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> --------- Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * Fix Jupyter Versioning (#2866) * Update versioning Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * Fix unit test Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * Update path if jupyter file actually exist Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * Fix pickled twice problem Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> --------- Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * improved output handling in notebooks (#2869) --------- Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
kumare3
pushed a commit
that referenced
this pull request
Nov 8, 2024
Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
kumare3
pushed a commit
that referenced
this pull request
Nov 8, 2024
* Pydantic Transformer V2 Signed-off-by: Future-Outlier <eric901201@gmail.com> * add __init__.py Signed-off-by: Future-Outlier <eric901201@gmail.com> * add json schema Signed-off-by: Future-Outlier <eric901201@gmail.com> * convert float to int Signed-off-by: Future-Outlier <eric901201@gmail.com> * change gitsha in test script mode Signed-off-by: Future-Outlier <eric901201@gmail.com> * change gitsha Signed-off-by: Future-Outlier <eric901201@gmail.com> * use strict map=false Signed-off-by: Future-Outlier <eric901201@gmail.com> * Test flytefile console input + attr access Signed-off-by: Future-Outlier <eric901201@gmail.com> * add conditional branch Signed-off-by: Future-Outlier <eric901201@gmail.com> * better rx Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add flytedir generic -> flytedir Signed-off-by: Future-Outlier <eric901201@gmail.com> * support enum Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> * add tests for input from flyte console Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add Tests for dataclass in BaseModel and pydantic.dataclass in BaseModel Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> * update thomas's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * change tree file structure Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Niel's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * > to >= Signed-off-by: Future-Outlier <eric901201@gmail.com> * try monodoc build again Signed-off-by: Future-Outlier <eric901201@gmail.com> * add pydantic README.md Signed-off-by: Future-Outlier <eric901201@gmail.com> * revert -vvv in monodocs Signed-off-by: Future-Outlier <eric901201@gmail.com> * use model_validate_json to turn protobuf struct to python val Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix issue Signed-off-by: Future-Outlier <eric901201@gmail.com> * handle flyte types in dict transformer from protobuf struct input (e.g FlyteConsole) Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add print Signed-off-by: Future-Outlier <eric901201@gmail.com> * expected python type Signed-off-by: Future-Outlier <eric901201@gmail.com> * switch call function order Signed-off-by: Future-Outlier <eric901201@gmail.com> * try msgpack to handle protobug struct Signed-off-by: Future-Outlier <eric901201@gmail.com> * Better Comment in Dict Transformer Signed-off-by: Future-Outlier <eric901201@gmail.com> * Propeller -> FlytePropeller Signed-off-by: Future-Outlier <eric901201@gmail.com> * dict_to_flyte_types Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * add attr for protobuf struct . dict Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add Life Cycle for Flyte Types Signed-off-by: Future-Outlier <eric901201@gmail.com> * better comments for derializing flyteschema and sd Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * add back pv._remote_path = None to flytefile and flytedir Signed-off-by: Future-Outlier <eric901201@gmail.com> * experiment Signed-off-by: Future-Outlier <eric901201@gmail.com> * experiment Signed-off-by: Future-Outlier <eric901201@gmail.com> * experiment Signed-off-by: Future-Outlier <eric901201@gmail.com> * Add comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Yee's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * code dc -> bm Signed-off-by: Future-Outlier <eric901201@gmail.com> * Example dc -> bm and Example all flyte types Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix union dataclass, not yet add comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * add pydantic and dataclass optional test Signed-off-by: Future-Outlier <eric901201@gmail.com> * NoneType=type(None) Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix union transformer none case with Eduardo Signed-off-by: Future-Outlier <eric901201@gmail.com> * add comments for none type transformer + union transformer Signed-off-by: Future-Outlier <eric901201@gmail.com> * add TODO Signed-off-by: Future-Outlier <eric901201@gmail.com> * use deserailize = True Signed-off-by: Future-Outlier <eric901201@gmail.com> * add all deserialize Signed-off-by: Future-Outlier <eric901201@gmail.com> * better comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * better comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * test Signed-off-by: Future-Outlier <eric901201@gmail.com> * Fix flyte directory issue by discussion with Kevin Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> * add tests for providing conext when doing serialization Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * test Signed-off-by: Future-Outlier <eric901201@gmail.com> * move setattr to core Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove comments Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove Signed-off-by: Future-Outlier <eric901201@gmail.com> * testing Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * flytekit/core/type_engine.py Signed-off-by: Future-Outlier <eric901201@gmail.com> * add tests for Union Signed-off-by: Future-Outlier <eric901201@gmail.com> * Trigger CI Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove nonetype Signed-off-by: Future-Outlier <eric901201@gmail.com> * raw=Fasle as default Signed-off-by: Future-Outlier <eric901201@gmail.com> * pydantic move to core test Signed-off-by: Future-Outlier <eric901201@gmail.com> * move to core Signed-off-by: Future-Outlier <eric901201@gmail.com> * log Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * test Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * nit Signed-off-by: Future-Outlier <eric901201@gmail.com> * lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * move to type_engine Signed-off-by: Future-Outlier <eric901201@gmail.com> * move back to init Signed-off-by: Future-Outlier <eric901201@gmail.com> * update kevin's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> * wip Signed-off-by: Future-Outlier <eric901201@gmail.com> * use decorator Signed-off-by: Future-Outlier <eric901201@gmail.com> * decorator Signed-off-by: Future-Outlier <eric901201@gmail.com> * fix syntax to support python 3.9 Signed-off-by: Future-Outlier <eric901201@gmail.com> * add Eduardo's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> * warning Signed-off-by: Future-Outlier <eric901201@gmail.com> * Update Yee's advice Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Show traceback by default (#2862) Signed-off-by: Kevin Su <pingsutw@apache.org> * Support Identifier in generate_console_url (#2868) Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com> * Support overriding node metadata for array node (#2865) * set parent node metadata on arraynode subnode Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * clean up setting passed in metadata for array node Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * add comment Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * test Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> --------- Signed-off-by: Paul Dittamo <pvdittamo@gmail.com> * Fix Jupyter Versioning (#2866) * Update versioning Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * Fix unit test Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * Update path if jupyter file actually exist Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * Fix pickled twice problem Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> --------- Signed-off-by: Mecoli1219 <michaellai901026@gmail.com> * improved output handling in notebooks (#2869) --------- Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tracking issue
Follow up to #2804
Why are the changes needed?
I do not think we should have another
generate_url_from_*public API inFlyteRemote. We can addIdentifiersupport intogenerate_console_urlWhat changes were proposed in this pull request?
This PR adds identity support to
generate_console_url.How was this patch tested?
Ran
pyflyte registerand returned the correct URLs.