Env: Only perform expensive install work when required

packages/env/CHANGELOG.md

@@ -6,17 +6,18 @@
 
 -   The `config` and `mappings` options in `.wp-env.json` are now merged with any overrides instead of being overwritten.
 -   The first listed theme is no longer activated when running wp-env start, since this overwrote whatever theme the user manually activated.
+-   `wp-env start` no longer stops the WordPress instance if it was already started unless it needs to configure WordPress.
+-   `wp-env start` no longer updates remote sources after first install if the configuration is the same. Use `wp-env start --update` to update sources.
 
 ### New Feature
 
 -   You may now specify specific configuration for different environments using `env.tests` or `env.development` in `.wp-env.json`.
-
-### Bug Fixes
-
--   `wp-env start` performance is now 2x faster after first install.
+-   `wp-env start` is significantly faster after first install.
 
 ## 1.6.0-rc.0 (2020-06-24)
 
+### Bug Fixes
+
 -   `wp-env destroy` now removes dangling docker volumes and networks associated with the WordPress environment.
 
 ## 1.4.0 (2020-05-28)

packages/env/README.md

@@ -172,20 +172,22 @@ $ wp-env start
 
 `wp-env` creates generated files in the `wp-env` home directory. By default, this is `~/.wp-env`. The exception is Linux, where files are placed at `~/wp-env` [for compatibility with Snap Packages](https://github.com/WordPress/gutenberg/issues/20180#issuecomment-587046325). The `wp-env` home directory contains a subdirectory for each project named `/$md5_of_project_path`. To change the `wp-env` home directory, set the `WP_ENV_HOME` environment variable. For example, running `WP_ENV_HOME="something" wp-env start` will download the project files to the directory `./something/$md5_of_project_path` (relative to the current directory).
 
-### `wp-env start [ref]`
+### `wp-env start`
+
+The start command installs and initalizes the WordPress environment, which includes downloading any specified remote sources. By default, `wp-env` will not update or re-configure the environment except when the configuration file changes. Tell `wp-env` to update sources and apply the configuration options again with `wp-env start --update`. This will not overrwrite any existing content.
 
 ```sh
 wp-env start
 
-Starts WordPress for development on port 8888 (​http://localhost:8888​)
-(override with WP_ENV_PORT) and tests on port 8889 (​http://localhost:8889​)
-(override with WP_ENV_TESTS_PORT). The current working directory must be a
-WordPress installation, a plugin, a theme, or contain a .wp-env.json file.
-
+Starts WordPress for development on port 8888 (override with WP_ENV_PORT) and
+tests on port 8889 (override with WP_ENV_TESTS_PORT). The current working
+directory must be a WordPress installation, a plugin, a theme, or contain a
+.wp-env.json file. After first insall, use the '--update' flag to download updates
+to mapped sources and to re-apply WordPress configuration options.
 
-Positionals:
-  ref  A `https://github.com/WordPress/WordPress` git repo branch or commit for
-       choosing a specific version.                 [string] [default: "master"]
+Options:
+  --update   Download source updates and apply WordPress configuration.
+                                                      [boolean] [default: false]
 ```
 
 ### `wp-env stop`
@@ -286,14 +288,14 @@ You can customize the WordPress installation, plugins and themes that the develo
 
 `.wp-env.json` supports six fields for options applicable to both the tests and development instances.
 
-| Field        | Type           | Default                                | Description                                                                                                               |
-| ------------ | -------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
-| `"core"`     | `string\|null` | `null`                                 | The WordPress installation to use. If `null` is specified, `wp-env` will use the latest production release of WordPress.  |
-| `"plugins"`  | `string[]`     | `[]`                                   | A list of plugins to install and activate in the environment.                                                             |
-| `"themes"`   | `string[]`     | `[]`                                   | A list of themes to install in the environment. The first theme in the list will be activated.                            |
+| Field        | Type           | Default                                | Description                                                                                                                |
+| ------------ | -------------- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
+| `"core"`     | `string\|null` | `null`                                 | The WordPress installation to use. If `null` is specified, `wp-env` will use the latest production release of WordPress.   |
+| `"plugins"`  | `string[]`     | `[]`                                   | A list of plugins to install and activate in the environment.                                                              |
+| `"themes"`   | `string[]`     | `[]`                                   | A list of themes to install in the environment. The first theme in the list will be activated.                             |
 | `"port"`     | `integer`      | `8888` (`8889` for the tests instance) | The primary port number to use for the installation. You'll access the instance through the port: 'http://localhost:8888'. |
-| `"config"`   | `Object`       | See below.                             | Mapping of wp-config.php constants to their desired values.                                                               |
-| `"mappings"` | `Object`       | `"{}"`                                 | Mapping of WordPress directories to local directories to be mounted in the WordPress instance.                            |
+| `"config"`   | `Object`       | See below.                             | Mapping of wp-config.php constants to their desired values.                                                                |
+| `"mappings"` | `Object`       | `"{}"`                                 | Mapping of WordPress directories to local directories to be mounted in the WordPress instance.                             |
 
 _Note: the port number environment variables (`WP_ENV_PORT` and `WP_ENV_TESTS_PORT`) take precedent over the .wp-env.json values._
 

packages/env/lib/cache.js

@@ -0,0 +1,89 @@
+/**
+ * External dependencies
+ */
+const path = require( 'path' );
+const fs = require( 'fs' ).promises;
+
+/**
+ * Options for cache parsing.
+ *
+ * @typedef WPEnvCacheOptions
+ * @property {string}  workDirectoryPath Path to the work directory located in ~/.wp-env.
+ */
+
+const CACHE_FILE_NAME = 'wp-env-cache.json';
+
+/**
+ * This function can be used to compare a possible new cache value against an
+ * existing cache value. For example, we can use this to check if the configuration
+ * has changed in a new run of wp-env start.
+ *
+ * @param {string}            key     A unique identifier for the cache.
+ * @param {any}               value   The value to check against the existing cache.
+ * @param {WPEnvCacheOptions} options Parsing options
+ *
+ * @return {boolean} If true, the value is different from the cache which exists.
+ */
+async function didCacheChange( key, value, options ) {
+	const existingValue = await getCache( key, options );
+	return value !== existingValue;
+}
+
+/**
+ * This function persists the given cache value to the cache file. It creates the
+ * file if it does not exist yet, and overwrites the existing cache value for the
+ * given key if it already exists.
+ *
+ * @param {string}            key     A unique identifier for the cache.
+ * @param {any}               value   The value to persist.
+ * @param {WPEnvCacheOptions} options Parsing options
+ */
+async function setCache( key, value, options ) {
+	const existingCache = await getCacheFile( options );
+	existingCache[ key ] = value;
+
+	await fs.writeFile(
+		getPathToCacheFile( options.workDirectoryPath ),
+		JSON.stringify( existingCache )
+	);
+}
+
+/**
+ * This function retrieves the cache associated with the given key from the file.
+ * Returns undefined if the key does not exist or if the cache file has not been
+ * created yet.
+ *
+ * @param {string}            key     The unique identifier for the cache value.
+ * @param {WPEnvCacheOptions} options Parsing options
+ *
+ * @return {any?} The cache value. Undefined if it has not been set or if the cache
+ *                file has not been created.
+ */
+async function getCache( key, options ) {
+	const cache = await getCacheFile( options );
+	return cache[ key ];
+}
+
+/**
+ * Returns the data stored in the cache file as a JS object. Instead of throwing
+ * an error, simply returns an empty object if the file cannot be retrieved.
+ *
+ * @param {WPEnvCacheOptions} options Parsing options
+ *
+ * @return {Object} The data from the cache file. Empty if the file does not exist.
+ */
+async function getCacheFile( { workDirectoryPath } ) {
+	const filename = getPathToCacheFile( workDirectoryPath );
+	try {
+		const rawCache = await fs.readFile( filename );
+		return JSON.parse( rawCache );
+	} catch {
+		return {};
+	}
+}
+
+function getPathToCacheFile( workDirectoryPath ) {
+	return path.resolve( workDirectoryPath, CACHE_FILE_NAME );
+}
+
+module.exports = { didCacheChange, setCache, getCache, getCacheFile };

packages/env/lib/cli.js

@@ -90,9 +90,16 @@ module.exports = function cli() {
 			) }} (override with WP_ENV_PORT) and tests on port {bold.underline ${ terminalLink(
 				'8889',
 				'http://localhost:8889'
-			) }} (override with WP_ENV_TESTS_PORT). The current working directory must be a WordPress installation, a plugin, a theme, or contain a .wp-env.json file.`
+			) }} (override with WP_ENV_TESTS_PORT). The current working directory must be a WordPress installation, a plugin, a theme, or contain a .wp-env.json file. After first insall, use the '--update' flag to download updates to mapped sources and to re-apply WordPress configuration options.`
 		),
-		() => {},
+		( args ) => {
+			args.option( 'update', {
+				type: 'boolean',
+				describe:
+					'Download source updates and apply WordPress configuration.',
+				default: false,
+			} );
+		},
 		withSpinner( env.start )
 	);
 	yargs.command(

packages/env/lib/commands/start.js

@@ -26,32 +26,24 @@ const {
 	configureWordPress,
 	setupWordPressDirectories,
 } = require( '../wordpress' );
+const { didCacheChange, setCache } = require( '../cache' );
+const md5 = require( '../md5' );
 
 /**
  * @typedef {import('../config').Config} Config
  */
+const CONFIG_CACHE_KEY = 'config_checksum';
 
 /**
  * Starts the development server.
  *
  * @param {Object}  options
  * @param {Object}  options.spinner A CLI spinner which indicates progress.
  * @param {boolean} options.debug   True if debug mode is enabled.
+ * @param {boolean} options.update  If true, update sources.
  */
-module.exports = async function start( { spinner, debug } ) {
-	/**
-	 * If the Docker image is already running and the `wp-env` files have been
-	 * deleted, the start command will not complete successfully. Stopping
-	 * the container before continuing allows the docker entrypoint script,
-	 * which restores the files, to run again when we start the containers.
-	 *
-	 * Additionally, this serves as a way to restart the container entirely
-	 * should the need arise.
-	 *
-	 * @see https://github.com/WordPress/gutenberg/pull/20253#issuecomment-587228440
-	 */
-	await stop( { spinner, debug } );
-
+module.exports = async function start( { spinner, debug, update } ) {
+	spinner.text = 'Reading configuration.';
 	await checkForLegacyInstall( spinner );
 
 	const config = await initConfig( { spinner, debug } );
@@ -64,18 +56,42 @@ module.exports = async function start( { spinner, debug } ) {
 		spinner.start();
 	}
 
-	spinner.text = 'Downloading WordPress.';
+	// Check if the hash of the config has changed. If so, run configuration.
+	const configHash = md5( config );
+	const workDirectoryPath = config.workDirectoryPath;
+	const shouldConfigureWp =
+		update ||
+		( await didCacheChange( CONFIG_CACHE_KEY, configHash, {
+			workDirectoryPath,
+		} ) );
+
+	/**
+	 * If the Docker image is already running and the `wp-env` files have been
+	 * deleted, the start command will not complete successfully. Stopping
+	 * the container before continuing allows the docker entrypoint script,
+	 * which restores the files, to run again when we start the containers.
+	 *
+	 * Additionally, this serves as a way to restart the container entirely
+	 * should the need arise.
+	 *
+	 * @see https://github.com/WordPress/gutenberg/pull/20253#issuecomment-587228440
+	 */
+	if ( shouldConfigureWp ) {
+		await stop( { spinner, debug } );
+		spinner.text = 'Downloading sources.';
+	}
 
 	await Promise.all( [
-		// Preemptively start the database while we wait for sources to download.
 		dockerCompose.upOne( 'mysql', {
 			config: config.dockerComposeConfigPath,
 			log: config.debug,
 		} ),
-		downloadSources( config, spinner ),
+		shouldConfigureWp && downloadSources( config, spinner ),
 	] );
 
-	await setupWordPressDirectories( config );
+	if ( shouldConfigureWp ) {
+		await setupWordPressDirectories( config );
+	}
 
 	spinner.text = 'Starting WordPress.';
 
@@ -84,37 +100,47 @@ module.exports = async function start( { spinner, debug } ) {
 		log: config.debug,
 	} );
 
-	if ( config.coreSource === null ) {
-		// Don't chown wp-content when it exists on the user's local filesystem.
+	// Only run WordPress install/configuration when config has changed.
+	if ( shouldConfigureWp ) {
+		spinner.text = 'Configuring WordPress.';
+
+		if ( config.coreSource === null ) {
+			// Don't chown wp-content when it exists on the user's local filesystem.
+			await Promise.all( [
+				makeContentDirectoriesWritable( 'development', config ),
+				makeContentDirectoriesWritable( 'tests', config ),
+			] );
+		}
+
+		try {
+			await checkDatabaseConnection( config );
+		} catch ( error ) {
+			// Wait 30 seconds for MySQL to accept connections.
+			await retry( () => checkDatabaseConnection( config ), {
+				times: 30,
+				delay: 1000,
+			} );
+
+			// It takes 3-4 seconds for MySQL to be ready after it starts accepting connections.
+			await sleep( 4000 );
+		}
+
+		// Retry WordPress installation in case MySQL *still* wasn't ready.
 		await Promise.all( [
-			makeContentDirectoriesWritable( 'development', config ),
-			makeContentDirectoriesWritable( 'tests', config ),
+			retry( () => configureWordPress( 'development', config, spinner ), {
+				times: 2,
+			} ),
+			retry( () => configureWordPress( 'tests', config, spinner ), {
+				times: 2,
+			} ),
 		] );
-	}
 
-	try {
-		await checkDatabaseConnection( config );
-	} catch ( error ) {
-		// Wait 30 seconds for MySQL to accept connections.
-		await retry( () => checkDatabaseConnection( config ), {
-			times: 30,
-			delay: 1000,
+		// Set the cache key once everything has been configured.
+		await setCache( CONFIG_CACHE_KEY, configHash, {
+			workDirectoryPath,
 		} );
-
-		// It takes 3-4 seconds for MySQL to be ready after it starts accepting connections.
-		await sleep( 4000 );
 	}
 
-	// Retry WordPress installation in case MySQL *still* wasn't ready.
-	await Promise.all( [
-		retry( () => configureWordPress( 'development', config, spinner ), {
-			times: 2,
-		} ),
-		retry( () => configureWordPress( 'tests', config, spinner ), {
-			times: 2,
-		} ),
-	] );
-
 	spinner.text = 'WordPress started.';
 };
 

packages/env/lib/config/config.js

@@ -5,7 +5,6 @@
 const fs = require( 'fs' ).promises;
 const path = require( 'path' );
 const os = require( 'os' );
-const crypto = require( 'crypto' );
 
 /**
  * Internal dependencies
@@ -14,6 +13,7 @@ const detectDirectoryType = require( './detect-directory-type' );
 const { validateConfig, ValidationError } = require( './validate-config' );
 const readRawConfigFile = require( './read-raw-config-file' );
 const parseConfig = require( './parse-config' );
+const md5 = require( '../md5' );
 
 /**
  * wp-env configuration.
@@ -332,13 +332,3 @@ async function getHomeDirectory() {
 			: '.wp-env'
 	);
 }
-
-/**
- * Hashes the given string using the MD5 algorithm.
- *
- * @param {string} data The string to hash.
- * @return {string} An MD5 hash string.
- */
-function md5( data ) {
-	return crypto.createHash( 'md5' ).update( data ).digest( 'hex' );
-}

packages/env/lib/md5.js

@@ -0,0 +1,17 @@
+/**
+ * External dependencies
+ */
+const crypto = require( 'crypto' );
+
+/**
+ * Hashes the given string using the MD5 algorithm.
+ *
+ * @param {any} data The data to hash. If not a string, converted with JSON.stringify.
+ * @return {string} An MD5 hash string.
+ */
+module.exports = function md5( data ) {
+	const convertedData =
+		typeof data === 'string' ? data : JSON.stringify( data );
+
+	return crypto.createHash( 'md5' ).update( convertedData ).digest( 'hex' );
+};