Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SDK - Components - Verify the object type when serializing primitive arguments #2272

Merged
merged 2 commits into from
Oct 1, 2019
Merged

SDK - Components - Verify the object type when serializing primitive arguments #2272

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9316586
SDK - Components - Verify the object type when serializing primitive …
Ark-kun Sep 30, 2019
ff2b464
Allow serializing integer when float is required.
Ark-kun Sep 30, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 34 additions & 4 deletions sdk/python/kfp/components/_data_passing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,36 @@
])


def _serialize_str(str_value: str) -> str:
if not isinstance(str_value, str):
raise TypeError('Value "{}" has type "{}" instead of str.'.format(str(str_value), str(type(str_value))))
return str_value


def _serialize_int(int_value: int) -> str:
if isinstance(int_value, str):
return int_value
if not isinstance(int_value, int):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can combine this two if by saying if not isinstance(int_value, int) and not isinstance(int_value, str)?

Copy link

@numerology numerology Oct 1, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar for float/bool serializer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will work in this case, but it might not work for general serializer structure:

def serialize_some_type(value: SomeType):
    if isinstance(value, str):
        return value
    if not isinstance(value, SomeType):
        raise TypeError('Value "{}" has type "{}" instead of SomeType.'.format(str(value), str(type(value))))
    return serialize(value)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find the existing code structure a little bit more readable with three clear cases.

raise TypeError('Value "{}" has type "{}" instead of int.'.format(str(int_value), str(type(int_value))))
return str(int_value)


def _serialize_float(float_value: float) -> str:
if isinstance(float_value, str):
return float_value
if not isinstance(float_value, (float, int)):
raise TypeError('Value "{}" has type "{}" instead of float.'.format(str(float_value), str(type(float_value))))
return str(float_value)


def _serialize_bool(bool_value: bool) -> str:
if isinstance(bool_value, str):
return bool_value
if not isinstance(bool_value, bool):
raise TypeError('Value "{}" has type "{}" instead of bool.'.format(str(bool_value), str(type(bool_value))))
return str(bool_value)


def _deserialize_bool(s) -> bool:
from distutils.util import strtobool
return strtobool(s) == 1
Expand Down Expand Up @@ -75,10 +105,10 @@ def _deserialize_base64_pickle(s):


_converters = [
Converter([str], ['String', 'str'], str, 'str', None),
Converter([int], ['Integer', 'int'], str, 'int', None),
Converter([float], ['Float', 'float'], str, 'float', None),
Converter([bool], ['Boolean', 'bool'], str, _bool_deserializer_code, _bool_deserializer_definitions),
Converter([str], ['String', 'str'], _serialize_str, 'str', None),
Converter([int], ['Integer', 'int'], _serialize_int, 'int', None),
Converter([float], ['Float', 'float'], _serialize_float, 'float', None),
Converter([bool], ['Boolean', 'bool'], _serialize_bool, _bool_deserializer_code, _bool_deserializer_definitions),
Converter([list], ['JsonArray', 'List', 'list'], _serialize_json, 'json.loads', 'import json'), # ! JSON map keys are always strings. Python converts all keys to strings without warnings
Converter([dict], ['JsonObject', 'Dictionary', 'Dict', 'dict'], _serialize_json, 'json.loads', 'import json'), # ! JSON map keys are always strings. Python converts all keys to strings without warnings
Converter([], ['Json'], _serialize_json, 'json.loads', 'import json'),
Expand Down