Skip to content

Set 'utf8mb4' when exporting without '--default-character-set' option #199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
},
"require-dev": {
"wp-cli/entity-command": "^1.3 || ^2",
"wp-cli/wp-cli-tests": "^3.0.11"
"wp-cli/wp-cli-tests": "^3.0.17"
},
"config": {
"process-timeout": 7200,
Expand Down
37 changes: 37 additions & 0 deletions features/db-import.feature
Original file line number Diff line number Diff line change
Expand Up @@ -151,3 +151,40 @@ Feature: Import a WordPress database
"""
Debug (db): Running shell command: /usr/bin/env mysql --no-defaults --no-auto-rehash
"""

@require-wp-4.2
Scenario: Import db that has emoji in post
Given a WP install

When I run `wp post create --post_title="🍣"`
And I run `wp post list`
Then the return code should be 0
And STDOUT should contain:
"""
🍣
"""

When I try `wp db export wp_cli_test.sql --debug`
Then the return code should be 0
And the wp_cli_test.sql file should exist
And STDERR should contain:
"""
Detected character set of the posts table: utf8mb4
"""
And STDERR should contain:
"""
Setting missing default character set to utf8mb4
"""

When I run `wp db import --dbuser=wp_cli_test --dbpass=password1`
Then STDOUT should be:
"""
Success: Imported from 'wp_cli_test.sql'.
"""

When I run `wp post list`
Then the return code should be 0
And STDOUT should contain:
"""
🍣
"""
14 changes: 7 additions & 7 deletions features/db-search.feature
Original file line number Diff line number Diff line change
Expand Up @@ -917,42 +917,42 @@ Feature: Search through the database
Given a WP install

When I run `SHELL_PIPE=0 wp db search example.com`
Then STDOUT should contain:
Then STDOUT should strictly contain:
"""
wp_options:option_value
1:http://example.com
"""

When I run `SHELL_PIPE=0 wp db search example.com --table_column_color=%r --id_color=%g --match_color=%b`
Then STDOUT should contain:
Then STDOUT should strictly contain:
"""
wp_options:option_value
1:http://example.com
"""

When I run `SHELL_PIPE=0 wp db search example.com --table_column_color=%r`
Then STDOUT should contain:
Then STDOUT should strictly contain:
"""
wp_options:option_value
1:http://example.com
"""

When I run `SHELL_PIPE=0 wp db search example.com --id_color=%g`
Then STDOUT should contain:
Then STDOUT should strictly contain:
"""
wp_options:option_value
1:http://example.com
"""

When I run `SHELL_PIPE=0 wp db search example.com --match_color=%b`
Then STDOUT should contain:
Then STDOUT should strictly contain:
"""
wp_options:option_value
1:http://example.com
"""

When I run `SHELL_PIPE=0 wp db search example.com --before_context=0 --after_context=0`
Then STDOUT should contain:
Then STDOUT should strictly contain:
"""
wp_options:option_value
1:example.com
Expand All @@ -967,7 +967,7 @@ Feature: Search through the database
"""
example.com
"""
And STDOUT should not contain:
And STDOUT should strictly not contain:
"""

"""
Expand Down
74 changes: 72 additions & 2 deletions src/DB_Command.php
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,20 @@
*/
class DB_Command extends WP_CLI_Command {

/**
* Legacy UTF-8 encoding for MySQL.
*
* @var string
*/
const ENCODING_UTF8 = 'utf8';

/**
* Standards-compliant UTF-8 encoding for MySQL.
*
* @var string
*/
const ENCODING_UTF8MB4 = 'utf8mb4';

/**
* A list of incompatible SQL modes.
*
Expand Down Expand Up @@ -554,6 +568,23 @@ public function export( $args, $assoc_args ) {

$support_column_statistics = exec( $mysqldump_binary . ' --help | grep "column-statistics"' );

/*
* In case that `--default-character-set` is not given and `DB_CHARSET` is `utf8`,
* we try to deduce what the actual character set for the posts table of the
* current database is and use `utf8mb4` as a `default-character-set` if that
* seems like the safer default, to ensure emojis are encoded correctly.
*/
if (
! isset( $assoc_args['default-character-set'] )
&&
( defined( 'DB_CHARSET' ) && self::ENCODING_UTF8 === constant( 'DB_CHARSET' ) )
&&
self::ENCODING_UTF8MB4 === $this->get_posts_table_charset( $assoc_args )
) {
WP_CLI::debug( 'Setting missing default character set to ' . self::ENCODING_UTF8MB4, 'db' );
$assoc_args['default-character-set'] = self::ENCODING_UTF8MB4;
}

$initial_command = sprintf( "{$mysqldump_binary}%s ", $this->get_defaults_flag_string( $assoc_args ) );
WP_CLI::debug( "Running initial shell command: {$initial_command}", 'db' );

Expand Down Expand Up @@ -607,6 +638,45 @@ public function export( $args, $assoc_args ) {
}
}

/**
* Get the current character set of the posts table.
*
* @param array Associative array of associative arguments.
* @return string Posts table character set.
*/
private function get_posts_table_charset( $assoc_args ) {
$query = 'SELECT CCSA.character_set_name '
. 'FROM information_schema.`TABLES` T, '
. 'information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` CCSA '
. 'WHERE CCSA.collation_name = T.table_collation '
. 'AND T.table_schema = "' . DB_NAME . '" '
. 'AND T.table_name LIKE "%\_posts";';

list( $stdout, $stderr, $exit_code ) = self::run(
sprintf(
'/usr/bin/env mysql%s --no-auto-rehash --batch --skip-column-names',
$this->get_defaults_flag_string( $assoc_args )
),
[ 'execute' => $query ],
false
);

if ( $exit_code ) {
WP_CLI::warning(
'Failed to get current character set of the posts table.'
. ( ! empty( $stderr ) ? " Reason: {$stderr}" : '' )
);

return self::ENCODING_UTF8MB4;
}

$stdout = trim( $stdout );

WP_CLI::debug( "Detected character set of the posts table: {$stdout}.", 'db' );

return $stdout;
}

/**
* Imports a database from a file or from STDIN.
*
Expand Down Expand Up @@ -1235,7 +1305,7 @@ public function search( $args, $assoc_args ) {
}

$encoding = null;
if ( 0 === strpos( $wpdb->charset, 'utf8' ) ) {
if ( 0 === strpos( $wpdb->charset, self::ENCODING_UTF8 ) ) {
$encoding = 'UTF-8';
}

Expand Down Expand Up @@ -1607,7 +1677,7 @@ private static function is_text_col( $type ) {
* @return string|array An escaped string if given a string, or an array of escaped strings if given an array of strings.
*/
private static function esc_sql_ident( $idents ) {
$backtick = function ( $v ) {
$backtick = static function ( $v ) {
// Escape any backticks in the identifier by doubling.
return '`' . str_replace( '`', '``', $v ) . '`';
};
Expand Down