Create IDGenerator transformer #680
Conversation
| @@ -67,7 +67,9 @@ def _validate_helper(validator_function, args, steps): | |||
|
|
|||
| def _is_valid_transformer(transformer_name): | |||
| """Determine if transformer should be tested or not.""" | |||
| invalid_names = ['IdentityTransformer', 'Dummy', 'OrderedLabelEncoder', 'CustomLabelEncoder'] | |||
| invalid_names = [ | |||
There was a problem hiding this comment.
I don't know if we want this. I think if yes, we need an appropriate dataset with ID columns.
There was a problem hiding this comment.
How does the AnonymizedFaker get tested? Wouldn't that have similar data?
There was a problem hiding this comment.
It does not here because it has INPUT_SDTYPE='pii'. Following the discussion in the EngMeeting to not support id sdtype on RDT, should I set INPUT_SDTYPE='pii' to the IDGenerator rather than INPUT_SDTYPE='id' (as mentioned in the issue) @amontanez24?
There was a problem hiding this comment.
I think we said text actually. Right @npatki
Codecov ReportPatch coverage:
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more. Additional details and impacted files@@ Coverage Diff @@
## master #680 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 17 17
Lines 1660 1684 +24
=========================================
+ Hits 1660 1684 +24
☔ View full report in Codecov by Sentry. |
| @@ -67,7 +67,9 @@ def _validate_helper(validator_function, args, steps): | |||
|
|
|||
| def _is_valid_transformer(transformer_name): | |||
| """Determine if transformer should be tested or not.""" | |||
| invalid_names = ['IdentityTransformer', 'Dummy', 'OrderedLabelEncoder', 'CustomLabelEncoder'] | |||
| invalid_names = [ | |||
There was a problem hiding this comment.
How does the AnonymizedFaker get tested? Wouldn't that have similar data?
|
Thanks for your review @frances-h. I addressed the comments in 33d4deb. I'm just not sure which |
rdt/transformers/pii/anonymizer.py
Outdated
| @@ -132,7 +132,9 @@ def get_supported_sdtypes(cls): | |||
| list: | |||
| Accepted input sdtypes of the transformer. | |||
| """ | |||
| unsupported_sdtypes = {'numerical', 'datetime', 'categorical', 'boolean', 'text', None} | |||
| unsupported_sdtypes = { | |||
| 'numerical', 'datetime', 'categorical', 'boolean', 'text', None, 'id' | |||
There was a problem hiding this comment.
Do we need to add id here? My understanding from the discussion was that RDT doesn't support the id sdtype.
There was a problem hiding this comment.
Yes good catch thank you, I forgot about it, done in f3638cc
Resolve #675