Skip to content

mdshack/dbanon

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

dbanon

Release Status codecov

A run-anywhere, dependency-less database anonymizer.

Installation

Download the latest release from GitHub.

Usage

dbanon reads from stdin and writes to stdout.

mysqldump mydb | dbanon -config=myconfig.yml | gzip > mydb.sql.gz

The -config flag can use bundled configurations or point to the path of a custom configuration file.

Rules

Static

These are basic faking rules that don't accept any arguments or know/care about the current value of a cell.

first_name
last_name
full_name
company_name
username
password
ipv4
url
linkedin_url
md5
uuid
json
query_params
datetime
state
city
postcode
street
country_code
money
money_decimal
email
telephone

Dynamic

These are more advanced rules that accept arguments and can be impacted by the current value of the cell.

dynamic.email

arguments

  • ignoreEmail - Any emails with this suffix will not be faked.

example

rule = dyanmic.email(@test.com)
current cell value = blam@test.com
after dbanon = blam@test.com

rule = dyanmic.email(@test.com)
current cell value = test@blam.com
after dbanon = random@fake.email

Configuration

Magento 2

dbanon bundles a default Magento 2 configuration file. However you almost certainly won't use it directly.

At minimum, you'll first need to run the map-eav subcommand. This translates EAV attribute codes to their respective attribute ids.

You must feed it a mysqldump of eav_entity_type and eav_attribute (in that order).

mysqldump mydb eav_entity_type eav_attribute | dbanon -config=magento2 map-eav > ~/magento2-mapped.yml

map-eav will replace the attribute codes in the config file with attribute ids and print an updated config to stdout.

Next you'd run dbanon with the config generated by map-eav.

mysqldump mydb | dbanon -config=~/magento2-mapped.yml | gzip > mydb.sql.gz

Most Magento 2 databases, however, will have additional data that needs to be anonymized beyond the default bundled file.

For this you'll first want to create a new configuration file based off the bundled configuration. Instructions on customizing the configuration file are included in the "Custom Configuration" section.

Custom Configuration

Specify the path to your config file via the -config flag

mysqldump mydb | dbanon -config=myconfig.yml | gzip > mydb.sql.gz

See the etc directory for examples.

Columns are specified as key / value pairs. The value string winds up getting passed to this function, which gets random values from dmgk/faker.

It is also possible to pass direct Faker function calls for supported "raw providers"

Logging

dbanon records messages about anything notable (e.g. invalid configuration) to the file dbanon.log in the directory from which you run it.

-log-file

The -log-file flag can be used to have dbanon log to a different location.

mysqldump mydb | dbanon -config=myconfig.yml -log-file=var/dbanon.log

-log-level

The -log-level flag can be used to control the verbosity of logs. Supported values can be found here.

mysqldump mydb | dbanon -config=myconfig.yml -log-level=debug | gzip > mydb.sql.gz

The default log level is info.

-silent

Logging can be disabled entirely by passing the -silent flag to dbanon

mysqldump mydb | dbanon -config=myconfig.yml -silent | gzip > mydb.sql.gz

Profiling

dbanon will generate a CPU profile to the file dbanon.prof when passed the -profile flag.

mysqldump mydb | dbanon -profile -config=myconfig.yml >/dev/null

Limitations

  • Currently only supports MySQL

Updating

dbanon will self-update when passed the -update flag

dbanon -update

Development

Publishing releases

Releases are published with GoReleaser.

  • Create and push a tag (e.g. git tag -a v0.6.2 -m "v0.6.2 && git push origin v0.6.2)
  • Build the go-bindata (e.g. go-bindata -pkg bindata -o bindata/bindata.go etc/*)
  • Run goreleaser (e.g. goreleaser release --rm-dist)

May need to create a new GitHub token if the current one expires

https://goreleaser.com/scm/github/

About

A run anywhere database anonymizer

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 98.7%
  • Makefile 1.3%