Add a check for overlapping combinations of data values between the real and synthetic data

[npatki]

Problem Description

Even if a column isn't explicitly PII, a combination of different values might be enough to de-identify the original data. For example, date of birth, zipcode, and gender.

SDV should offer a function that checks whether combinations of columns overlap between the real and synthetic data.

Expected behavior

Write a function called get_combination_overlap that computes the overlap of combinations of column values between the real and synthetic data. (I.e. combinations that appear exactly the same in both datasets).

from sdv.evaluation.utils import get_combination_overlap

overlap_amt = get_combination_overlap(
    real_data=real_data,
    synthetic_data=synthetic_data,
    table_name=table_name,
    column_names=['date_of_birth', 'zipcode', 'gender'],
    verbose=True
)

Number of common combinations: 0 (0.0%)
✅ The synthetic data does not contain any of the same combinations from the real data

Parameters:

• (required) real_data: A dictionary mapping a table name to a pandas DataFrame containing real data
• (required) synthetic_data: A dictionary mapping a table name to a pandas DataFrame containing synthetic dat
• (required) table_name: The name of the table that contains the columns to check
• (required) column_names: A list of strings represent the column names to check. Combinations of these columns will be checked.
• verbose: Whether to print out the interpretation of the results

Returns: The # of overlapping combinations

Prints: If verbose, it should print: "Number of common combinations: [number] ([percent])" The percent is defined as (# unique overlapping combinations)/(# of unique combinations across both real and synthetic data). After this, it should print out the interpretation:

• If there are 0 overlapping combinations, then print: "✅ The synthetic data does not contain any of the same combinations from the real data"
• If there are only a few overlapping combinations (<2%), then print: "⚠️ The synthetic data contains a few of the same combinations as the real data. This might be due to random chance."
• If there are more than a few overlapping combinations (>2%), then print: "❌ The synthetic data contains a significant number of the same combinations as the real data. This might be due to a small number of possible combinations, a large sample of synthetic data, or a misconfiguration in your synthesizer."