Add a check for overlapping PII values between real and synthetic data

[npatki]

Problem Description

If a value is listed as PII, we expect that that the synthetic data does not re-use the same values as the real data. It should be inventing new values instead.

We should write a function that explicitly measures this.

Expected behavior

Write a function called get_pii_overlap that computes the overlap of PII values between the real and synthetic data. (I.e. values that appear in both datasets).

from sdv.evaluation.utils import get_pii_overlap

overlap_amt = get_pii_overlap(
    real_data=real_data,
    synthetic_data=synthetic_data,
    table_name=table_name,
    pii_column_name=column_name,
    verbose=True
)

Number of common data points: 0 (0.0%)
✅ The synthetic data does not contain any PII values from the real data

Parameters:

• (required) real_data: A dictionary mapping a table name to a pandas DataFrame containing real data
• (required) synthetic_data: A dictionary mapping a table name to a pandas DataFrame containing synthetic dat
• (required) table_name: The name of the table that contains the PII column to check
• (required) pii_column_name: The name of the column that contains PII values to check
• verbose: Whether to print out the interpretation of the results

Returns: The # of overlapping PII values

Prints: If verbose, it should print: "Number of common data points: [number] ([percent])" The percent is defined as (# overlapping values)/(# of unique values across all the real and synthetic data). After this, it should print out the interpretation:

• If there are 0 overlapping values, then print: "✅ The synthetic data does not contain any PII values from the real data"
• If there are only a few overlapping values (<2%), then print: "⚠️ The synthetic data contains a few PII values from the real data. This might be due to random chance."
• If there are more than a few overlapping values (>2%), then print: "❌ The synthetic data contains a significant number of the same PII values of as the real data. This might be due to a small number of possible PII values, a large sample of synthetic data, or a misconfiguration in your synthesizer."