Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IP address field type. #1485

Merged
merged 26 commits into from
Sep 16, 2020
Merged

Conversation

mgetka
Copy link
Contributor

@mgetka mgetka commented Jan 3, 2020

This pull request adds IPv4/IPv6 fields serialization and deserialization w/ tests included. To avoid confusion I am excluding the possibility to specify IP addresses as an integer or byte values.

src/marshmallow/fields.py Outdated Show resolved Hide resolved
Copy link
Member

@lafrech lafrech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a parameter to only accept IPv4 or IPv6 rather than accepting any?

Should we also provide IPv4 and IPv6 fields as a shortcut?

Also, I don't see the point of the _serialize method. Apparently you copied it from an already existing field (UUID) so it's nothing new. But this seems a bit dodgy to me. Let's discuss this in #1489.

src/marshmallow/fields.py Outdated Show resolved Hide resolved
@mgetka
Copy link
Contributor Author

mgetka commented Jan 13, 2020

Also, I don't see the point of the _serialize method. Apparently you copied it from an already existing field (UUID) so it's nothing new. But this seems a bit dodgy to me.

That is correct, I've overridden the _serialize based on insights collected from UUID field implementation. After inspecting the implementation of String base class, I can no longer see any purpose of _serialize method.

src/marshmallow/fields.py Outdated Show resolved Hide resolved
@mgetka
Copy link
Contributor Author

mgetka commented Jan 13, 2020

Also please note that exploded notation is specific for IPv6 addresses, but the argument is also present in IPv4 constructor. This feature is sourced from ipaddress python standard library, where exploded notation can be applied to both IPv4 and IPv6 for consistency.

@sloria
Copy link
Member

sloria commented Jan 24, 2020

Would it make sense to implement these as validators? That would allow combining with other validators.

@mgetka
Copy link
Contributor Author

mgetka commented Jan 29, 2020

I have to admit that maybe I was too inspired by the implementation of the UUID field, and implemented the functionality without more complete understanding of fields module structure. As I can see, fields inheriting from String, are in general deserialized from strings to strings with additional input value format validation. We can see it in the Url and Email fields cases. Exception from this rule is the UUID field, which deserializes from string into UUID object representation. Therefore, it is not clear for me why UUID inherits from String and not Field, like in the case of DateTime or TimeDelta which also deserializes to object representation.

ipaddress library utilities provides with IP address object representations with validation occurring on object construction. In my opinion creation of the object representation for the purpose of validation, and then discarding it would be an overkill. Also I can see clear contract between validators based implementation of Url and Email fields and String base class features. But IP addresses are not the case - they are more like DateTime or TimeDelta fields. Therefore, IP field now inherits from Field and not String.

Also, I can see, that _validated method is utilized for chained validation in inheriting fields classes.
For IP fields it is not the case, since IPv4 and IPv6 Fields implements its own independent validation procedures. I have moved validation/object creation into _deserialize methods.

Returning to the case of the UUID field. Maybe it should be discussed whether it should inherit from String or rather Field to get a more transparent structure of the module?

I am also wondering whether IP specific validators are welcome? Based on ipaddress functionalities it is straightforward to implement validators like: PrivateIPAddress, GlobalIPAddress or one that checks whether IP address belongs to specified subnetwork.

Copy link
Member

@lafrech lafrech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Just a question/comment about a test. I wouldn't mind another opinion here.

Your remarks about UUID field make sense. Reworking it could be done in another PR, and perhaps in marshmallow 4 if it is a breaking change.

IP specific validation welcome. Should those be field constructor arguments instead like exploded?

tests/test_deserialization.py Outdated Show resolved Hide resolved
src/marshmallow/fields.py Outdated Show resolved Hide resolved
@sloria sloria requested review from lafrech and deckar01 August 6, 2020 14:36
@sloria
Copy link
Member

sloria commented Aug 6, 2020

@lafrech @deckar01 Can one of you do a final CR pass on this? Looks like it's close to--if not completely--ready to merge.

Copy link
Member

@lafrech lafrech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'm just concerned about the code duplication. Could _deserialize be factorized, using class attributes for the object class, like what is done for the error message? (The typing would be a little less strict, but oh well.)

@mgetka
Copy link
Contributor Author

mgetka commented Aug 10, 2020

LGTM. I'm just concerned about the code duplication. Could _deserialize be factorized, using class attributes for the object class, like what is done for the error message? (The typing would be a little less strict, but oh well.)

I've tried to define class variable typed as typing.Callable[[str], typing.Union[ipaddress.IPv4Address, ipaddress.IPv6Address]] which holds ipaddress.ip_address for generic IP field, and ipaddress.IPv4Address and ipaddress.IPv6Address classes for IPv4 and IPv6 fields respectively, but there are issues with ipaddress.ip_address becoming a method. The issue can be suppressed by decorating the ipaddress.ip_address with staticmethod like

DESERIALIZATION_CALLABLE = staticmethod(ipaddress.ip_address)

but I'm not entirely comfortable with such construct. Also, mypy doesn't like it as well since there is unresolved issue #3482 which is of relevance for class methods and static methods defined this way.

@mgetka mgetka requested a review from lafrech August 12, 2020 12:46
@mgetka mgetka requested a review from lafrech August 25, 2020 10:52
Copy link
Member

@lafrech lafrech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay and thanks for your work on this.

@lafrech
Copy link
Member

lafrech commented Aug 29, 2020

@sloria, @deckar01, any objection to merging this?.

@sloria
Copy link
Member

sloria commented Sep 10, 2020

@lafrech I didn't to a fine-toothed review of this, but it looked good last time I looked. I defer to you on merging this.

@lafrech lafrech merged commit a857318 into marshmallow-code:dev Sep 16, 2020
@lafrech
Copy link
Member

lafrech commented Sep 16, 2020

Merged and released.

Thanks again @mgetka.

@mgetka mgetka deleted the ip-address-field branch September 21, 2020 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants