Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add docker commands in ImageSpec #2676

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open

Add docker commands in ImageSpec #2676

wants to merge 29 commits into from

Commits on Aug 9, 2024

  1. feat: add docker commands in ImageSpec 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267
    mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    662f15e View commit details
    Browse the repository at this point in the history

  2. use private-key (flyteorg#2645) 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @Future-Outlier @mao3267
    Future-Outlier authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    4f670dc View commit details
    Browse the repository at this point in the history

  3. Don't call remote when --help in remote-X (flyteorg#2642) 

    * don't call remote

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* nit

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

---------

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @wild-endeavor @mao3267
    wild-endeavor authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    b4e7ece View commit details
    Browse the repository at this point in the history

  4. Bump grpc receive message size (flyteorg#2640) 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @wild-endeavor @pingsutw @mao3267
    2 people authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    a22792f View commit details
    Browse the repository at this point in the history

  5. Raise an exception when filters' value isn't a list. (flyteorg#2576) 

    * Add an exeception when filters' value isn't a list

* Make the exception more specific

Signed-off-by: Nelson Chen <asd3431090@gmail.com>

* add an unit test for value_in

Signed-off-by: Nelson Chen <asd3431090@gmail.com>

* lint

Signed-off-by: Kevin Su <pingsutw@apache.org>

---------

Signed-off-by: Nelson Chen <asd3431090@gmail.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @arbaobao @pingsutw @mao3267
    2 people authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    fbba823 View commit details
    Browse the repository at this point in the history

  6. Update error message for TypeTransformerFailedError (flyteorg#2648) 

    Signed-off-by: wayner0628 <a901639@gmail.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @wayner0628 @pingsutw @mao3267
    2 people authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    b946674 View commit details
    Browse the repository at this point in the history

  7. [Error Message] Dataclasses Mismatched Type (flyteorg#2650) 

    * Show different of types in dataclass when transforming error

Signed-off-by: Future-Outlier <eric901201@gmail.com>

* add tests for dataclass

Signed-off-by: Future-Outlier <eric901201@gmail.com>

* fix tests

Signed-off-by: Future-Outlier <eric901201@gmail.com>

---------

Signed-off-by: Future-Outlier <eric901201@gmail.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @Future-Outlier @mao3267
    Future-Outlier authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    ffc030e View commit details
    Browse the repository at this point in the history

  8. Added warning for command list and shell true (flyteorg#2653) 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @pryce-turner @mao3267
    pryce-turner authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    ac8563d View commit details
    Browse the repository at this point in the history

  9. In FlyteRemote.upload_file, pass the file object directly rather th… 

    …an the entire bytes buffer (flyteorg#2641)

* pass the local file directly for streaming in FlyteRemote.upload_file

Signed-off-by: Reda Oulbacha <reda@artera.ai>

* ruff format

Signed-off-by: Reda Oulbacha <reda@artera.ai>

* add an integration test

Signed-off-by: Reda Oulbacha <reda@artera.ai>

* remove unnecessary len

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* redo registration in the integration test

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* fix misspel

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* run the integration test serially

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* disable agent

Signed-off-by: Kevin Su <pingsutw@apache.org>

* use os.stat instead of os.seek to determine content_length

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* rewrite tests only uploda a file, use a separate marker

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* parametrize integration test makefile cmd

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* add workflow_dispatch for debugging

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* replace trigger with push

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* remove trailing whitespaces

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* remove agent disabling

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* remove trailing debug CI trigger

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

* clean up botocore imports

Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>

---------

Signed-off-by: Reda Oulbacha <reda@artera.ai>
Signed-off-by: redartera <120470035+redartera@users.noreply.github.com>
Signed-off-by: Kevin Su <pingsutw@apache.org>
Co-authored-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @redartera @pingsutw @mao3267
    2 people authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    b59346a View commit details
    Browse the repository at this point in the history

  10. Modify test_array_node.py to support running in python 3.8 (flyteorg#… 

    …2652)

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @eapolinario @mao3267
    2 people authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f11f397 View commit details
    Browse the repository at this point in the history

  11. Handle common cases of mutable default arguments explicitly (flyteorg… 

    …#2651)

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @eapolinario @mao3267
    2 people authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    901fe4f View commit details
    Browse the repository at this point in the history

  12. Allow a hash method to be present for numpy arrays (flyteorg#2649) 

    Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @demmerichs @mao3267
    demmerichs authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    9e0dbf5 View commit details
    Browse the repository at this point in the history

  13. return exceptions when gathering (flyteorg#2657) 

    * return exceptions when gathering

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* pr comment

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

---------

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @wild-endeavor @mao3267
    wild-endeavor authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    140b58f View commit details
    Browse the repository at this point in the history

  14. Correct FlyteFile docstring (flyteorg#2658) 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @ppiegaze @mao3267
    ppiegaze authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    78a6219 View commit details
    Browse the repository at this point in the history

  15. Remove pip cache after install (flyteorg#2662) 

    Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @thomasjpfan @mao3267
    thomasjpfan authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    8d229d9 View commit details
    Browse the repository at this point in the history

  16. Adds validation to image_spec for list of strings (flyteorg#2655) 

    Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @thomasjpfan @mao3267
    thomasjpfan authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    59b5932 View commit details
    Browse the repository at this point in the history

  17. Make elastic timeout configurable for HorovovJob. (flyteorg#2631) 

    Signed-off-by: Chen Zhu <chzhu@linkedin.com>
Signed-off-by: Chen Zhu <zhuchen1033@gmail.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @supercharleszhu @mao3267
    supercharleszhu authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    be3b145 View commit details
    Browse the repository at this point in the history

  18. Fix overriding of loader_args task resolver in papermill plugin (flyt… 

    …eorg#2660)

* Add repro test case

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

* Restore loader_args in papermill plugin

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

* Add unit tests

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>

---------

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @eapolinario @mao3267
    2 people authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    9ea6b23 View commit details
    Browse the repository at this point in the history

  19. Catch all exceptions when rendering python dependencies (flyteorg#2664) 

    Signed-off-by: Thomas J. Fan <thomasjpfan@gmail.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @thomasjpfan @mao3267
    thomasjpfan authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    e83c240 View commit details
    Browse the repository at this point in the history

  20. Don't check the retrun statement for reference_launch_plan (flyteorg#… 

    …2665)

Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @pingsutw @mao3267
    pingsutw authored and mao3267 committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    b3dbc08 View commit details
    Browse the repository at this point in the history

  21. Bump flyteidl to 1.13.1 (flyteorg#2666) 

    Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267 @eapolinario
    mao3267 and eapolinario committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    72319a2 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2024

  1. Merge branch 'master' of https://github.com/mao3267/flytekit into add… 

    …-docker-commands
    @mao3267
    mao3267 committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    5a72f98 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. fix: raise exception for docker commands in envd builder 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267
    mao3267 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    8446503 View commit details
    Browse the repository at this point in the history

  2. test: add unit tests for docker commands 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267
    mao3267 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    0a85ccc View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. fix: modify error message 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267
    mao3267 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    f18c283 View commit details
    Browse the repository at this point in the history

  2. test: raise exception for docker commands in envd builder 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267
    mao3267 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    f073069 View commit details
    Browse the repository at this point in the history

  3. test: ImageSpec with docker commands 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267
    mao3267 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    f2c3679 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of https://github.com/mao3267/flytekit into add… 

    …-docker-commands
    @mao3267
    mao3267 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    767ef0f View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2024

  1. fix: add docker commands to parameters_str_list 

    Signed-off-by: mao3267 <chenvincent610@gmail.com>
    @mao3267
    mao3267 committed Aug 17, 2024
    Configuration menu
    Copy the full SHA
    cca874b View commit details
    Browse the repository at this point in the history