Is it possible to exclude some rows based on a column value?

[HaroldAlingGSD]

If we want to make all records in the users table anonymous, except for our *@company.ext users, can we exclude those from being processed?

[ppomes]

Hi @HaroldAlingGSD,

The current design of myanon lacks mechanisms to exclude records according to some criteria.

However, you should be able to do filtering using mysqldump. The approach is to first create a fully anonymized dump, then replace the records that should not be anonymized.Something like:

# 1. First, complete anonymized dump
mysqldump mydb | myanon -f myanon.cfg > mydb_anon.sql

# 2. Then, dump users with @company.ext (using REPLACE statements) at the end to previous dump.
mysqldump mydb users \
  --no-create-info \
  --replace \
  --where="email LIKE '%@company.ext'" \
  --complete-insert >> mydb_anon.sql

Does this solve your needs? Let me know!

Regards,
Pierre

[ppomes]

Actually, I just realized that if you only need to anonymize the email field in the users table, the Python support might work for your case!

Since myanon's Python support works field-by-field, you could create a conditional anonymization function that checks the email value:

# In your Python script for myanon
def anonymize_email_conditional(email):
    if email.endswith('@company.ext'):
        return email  # Keep as-is
    else:
        # Anonymize other emails using the same HMAC approach as myanon
        import hashlib, hmac
        secret = myanon_utils.get_secret()
        return hmac.new(secret.encode(), email.encode(), hashlib.sha256).hexdigest()[:len(email)]

Then in your myanon config:

tables = {
    `users` = {
        `email` = pydef 'anonymize_email_conditional'
    }
}

This would process emails one by one and only anonymize those not ending with '@company.ext'.

Limitation: This only works if email is the only field you need to conditionally anonymize. If you need to exclude entire records (multiple fields), then the mysqldump two-step approach I mentioned earlier is still the way to go.

[ppomes]

Any news @HaroldAlingGSD ?

[HaroldAlingGSD]

Hey @ppomes, we are still just orientating on the possibilities to hash privacy related data; I'll try to set up a test soon but your Python solution looks really promising!

[ppomes]

Hi @HaroldAlingGSD,

Any news? Do you need help?

Pierre

[HaroldAlingGSD]

I am not familiar with Python, so forgive my amateur questions.

myanon.conf:

secret = 'my_secret_key'
pypath = '/app'
pyscript = 'processor.py'

tables = {
  `user` = {
    `firstname`  = texthash 8
    `lastname`  = texthash 8
    `email` = pydef 'anonymize_email_conditional'
  }
}

processor.py:

import myanon_utils

# In your Python script for myanon
def anonymize_email_conditional(email):
    if email.endswith('@company.ext'):
        return email  # Keep as-is
    else:
        # Anonymize other emails using the same HMAC approach as myanon
        import hashlib, hmac
        secret = myanon_utils.get_secret()
        return hmac.new(secret.encode(), email.encode(), hashlib.sha256).hexdigest()[:len(email)]

I'm using the myanon docker image so my python environment should be ok?

/app # myanon -v
myanon 0.8
/app # ls
__pycache__   in.sql        myanon.conf   processor.py

/app # myanon -f myanon.conf < in.sql
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
START TRANSACTION;
SET time_zone = "+00:00";

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8mb4 */;


CREATE TABLE `user` (
  `id` int(11) NOT NULL,
  `email` varchar(255) NOT NULL,
  `firstname` varchar(255) NOT NULL,
  `lastname` varchar(255) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

INSERT INTO `user` (`id`, `email`, `firstname`, `lastname`) VALUES
(1, ModuleNotFoundError: No module named 'processor.py'; 'processor' is not a package
Segmentation fault (core dumped)

/app # python3 processor.py
Traceback (most recent call last):
  File "/app/processor.py", line 1, in <module>
    import myanon_utils
ModuleNotFoundError: No module named 'myanon_utils'

It must be something simple I'm missing?

[ppomes]

The issue is that pyscript should be a Python module name, not a filename. Change:

pyscript = 'processor.py'

to:

pyscript = 'processor'

Python interprets processor.py as "import py from package processor", which causes the ModuleNotFoundError.

With that fix, your config and script should work as expected.

---

Regarding the segfault: that's a bug on my side — when the Python import fails, myanon crashes during cleanup instead of exiting gracefully. I've pushed a fix for that (Py_DECREF on a NULL pointer).

Thanks for the report!