Problem Description
Once we have created the RegexFormatAdherence (#897) and DateimeFormatAdherence (#898) metrics, we should update the DataValidity property to include these two metrics in its calculation.
With this change:
- Datetime columns will be checked twice for Data Validity: once for min/max boundaries and another time for datetime format.
- ID columns might also be checked twice for Data Validity: once for primary key uniqueness (if it's a primary key) and another time for regex format.
Expected behavior
Update DataValidty property to include regex adherence and datetime adherence.
- These checks should only happen if the
regex_format and datetime_format are supplied in the metadata for those columns. We should update the progress bar to account for these extra checks that will be done.
- The details for this property should include the new metric.
Score Calculation
This property should add the following metrics to calculate the score:
-
RegexFormatAdherence
-
DateimeFormatAdherence
>>> report.get_details(property_name='Data Validity', table_name='users')
Table Column Metric Score
users user_id KeyUniqueness 1.0
users user_id RegexFormatAdherence 1.0
users age BoundaryAdherence 1.0
users dob BoundaryAdherence 1.0
users dob DatetimeFormatAdherence 1.0
...
Problem Description
Once we have created the RegexFormatAdherence (#897) and DateimeFormatAdherence (#898) metrics, we should update the
DataValidityproperty to include these two metrics in its calculation.With this change:
Expected behavior
Update
DataValidtyproperty to include regex adherence and datetime adherence.regex_formatanddatetime_formatare supplied in the metadata for those columns. We should update the progress bar to account for these extra checks that will be done.Score Calculation
This property should add the following metrics to calculate the score:
RegexFormatAdherenceDateimeFormatAdherence