Don't wipe some tables when running GET

[jakeworrall]

When pulling a database from production to a staging/dev environment you may want some tables to remain in place, such as admin user tables for example.

An admin user table would usually be excluded from the backup to protect sensitive information.

It would be nice to have a way of not wiping tables like this when importing a database so that new admin user accounts do not have to be created every time the database is migrated.

[tgerulaitis]

The get command doesn't do any sanitisation - it imports the database exactly as it is on S3. All of sanitisation is done on the put command when uploading the database to S3. The idea is that we don't want to push the data up (especially sensitive data, like customer information, which we don't want to leave the production servers at all!) if we don't need it.

You can adjust what tables put excludes though! It uses Magerun2's --strip option to allow specifying individual tables or groups of tables, with @developer being the default. In fact the reason it is removing admin users is because Magerun 2 started doing that with @developer group a while ago.

I don't know if it's worth updating the --strip option in Magedbm2 to keep admin users by default. It is a risk (you risk data leaks any time any sensitive data leaves the production server), so I would personally air on the side of caution and strip it out by default. You can choose to override that though, by simply adding the following option to your put command:

--strip="@trade @stripped @search"

[jakeworrall]

Hi @tgerulaitis,

Sorry I do not think I was very clear.

I meant that when a database is imported using the get command, an option to not replace some tables with the version in the backup would be nice.

My understanding of how this works now is if a table is stripped when a database is uploaded using put, this table will be empty after the get command has been run to import the database.

The feature that I am suggesting would be an option to not replace some tables when importing a database.

For example, if my admin user on prod is 'jake-prod@test.com' and on staging is 'jake-staging@test.com', I would like to be able to not replace the admin_user on staging when I import the prod database so that I can still login with my 'jake-staging@test.com' account.

[tgerulaitis]

@jakeworrall ,

Ah, yes, sorry, you did explain it well the first time, I just misread it and assumed you were talking about keeping things in the backup itself.

That is a good feature suggestion! Not the easiest to implement, as you'd have to replace the current solution of "push the entire backup into mysql and let it deal with it" with something that reads in the SQL statements and understands what to do with them, but it's doable.

[Quintenps]

Good suggestion +1

[kennyaxial]

I am looking at implementing something like this. But from what I understand, it would be simpler to add a flag in the put function to not include the drop/create of the stripped tables.

We would only need to add a condition on the if on https://github.com/Space48/magedbm2/blob/master/src/Service/Database/Shell.php#L121

        if (count($stripTables) > 0) {
            // Create the structure-only dump for tables that we don't want the data from.
            $commands[] = $this->createDumpProcess()
                ->argument('--no-data')
                ->argument('--add-drop-table')
                ->argument('--default-character-set=utf8')
                ->argument(escapeshellarg($databaseName))
                ->arguments(array_map('escapeshellarg', $stripTables))
                ->output(escapeshellarg($structureOutputFile));
        } else {
            // Empty structure file to make the rest of the code more consistent.
            file_put_contents($structureOutputFile, '');
        }

My suggestion for the flag would probably be --exclude-stripped-tables.

I can probably manage to send a PR soon, if I can get a dev setup going.