Docker environment for Jetpack Development

Unified environment for developing Jetpack using Docker containers providing following goodies:

• An Ubuntu base operating system.
• Latest stable version of WordPress.
• Jetpack source code will be available as plugin from parent directory.
• PHPUnit setup.
• Xdebug setup.
• WP-CLI installed.
• MailDev to catch all the emails leaving WordPress so that you can observe them from browser.
• Handy NPM/Yarn shorthand commands like yarn docker:up and yarn docker:phpunit to simplify the usage.

To get started

All commands mentioned in this document should be run from the base Jetpack directory. Not from the docker directory!

Prerequisites

• Docker
• NodeJS
• Yarn — please make sure your version is higher than v1.3: yarn --version
• Optionally Ngrok client and account or some other service for creating a local HTTP tunnel. It’s fine to stay on the free pricing tier with Ngrok.

Install prerequisites; you will need to open up Docker to install its dependencies.

Start by cloning the Jetpack repository:

git clone git@github.com:Automattic/jetpack.git && cd jetpack

Optionally, copy settings file to modify it:

cp docker/default.env docker/.env

Anything you put in .env overrides values in default.env. You should modify all the password fields for security, for example.

Finally, spin up the containers:

yarn docker:up

Non-installed WordPress is running at http://localhost now.

You should establish a tunnel to your localhost with Ngrok or other similar service to be able to connect Jetpack. You cannot connect Jetpack when running WordPress via http://localhost. Read more from "Using Ngrok with Jetpack" section below.

You are now ready to login to your new WordPress install and connect Jetpack, congratulations!

You should follow Jetpack’s development documentation for installing Jetpack’s dependencies and building files. Docker setup does not build these for you.

Good to know

WordPress’ WP_SITEURL and WP_HOME constants are configured to be dynamic in ./docker/wordpress/wp-config.php so you shouldn’t need to change these even if you access the site via different domains.

Environment Variables, .env Files, and Ports

You can control some of the behavior of Jetpack's Docker configuration with environment variables. Note, though, that there are two types of environments:

1. The host environment in which the yarn docker:* (docker-compose) commands run when creating/managing the containers.
2. The containers' environments.

Host Environment

You can set the following variables on a per-command basis (PORT_WORDPRESS=8000 yarn docker:up) or, preferably, in a ./.env file in Jetpack's root directory.

• PORT_WORDPRESS: (default=80) The port on your host machine connected to the WordPress container's HTTP server.
• PORT_MYSQL: (default=3306) The port on your host machine connected to the MySQL container's MySQL server.
• PORT_MAILDEV: (default=1080) The port on your host machine connected to the MailDev container's MailDev HTTP server.
• PORT_SMTP: (default=25) The port on your host machine connected to the MailDev container's SMTP server.
• PORT_SFTP: (default=1022) The port on your host machine connected to the SFTP container's SFTP server.

Container Environments

Configurable settings are documented in the ./docker/default.env file. Customizations should go into a ./docker/.env file you create, though, not in the ./docker/default.env file.

Working with containers

Quick install WordPress

You can just quickly install WordPress and activate Jetpack via command line. Ensure you have your domain modified in .env file, spin up the containers and then run:

yarn docker:install

This will give you a single site with user/pass wordpress (unless you changed these from ./docker/.env file). You will still have to connect Jetpack to WordPress.com manually.

To convert installed single site into a multisite, run:

yarn docker:multisite-convert

To remove WordPress installation and start over, run:

yarn docker:uninstall

Start containers

yarn docker:up

Start three containers (WordPress, MySQL and MailDev) defined in docker-composer.yml. Wrapper for docker-composer up.

This command will rebuild the WordPress container if you made any changes to docker-composer.yml. It won’t build the images again on its own if you changed any of the other files like Dockerfile, run.sh (the entry-point file) or the provisioned files for configuring Apache and PHP. See "rebuilding images".

For running the containers in the background, use:

yarn docker:up -- -d

Stop containers

yarn docker:stop

Stops all containers. Wrapper for docker-composer stop.

yarn docker:down

Will stop all of the containers created by this docker-compose configuration and remove them, too. It won’t remove the images. Just the containers that have just been stopped.

Rebuild images

yarn docker:build-image

You need to rebuild the WordPress image with this command if you modified Dockerfile, docker-composer.yml or the provisioned files we use for configuring Apache and PHP.

Running unit tests

yarn docker:phpunit

This will run unit tests for Jetpack. You can pass arguments to phpunit like so:

yarn docker:phpunit --filter=Protect

This command runs the tests as a multi site install

yarn docker:phpunit:multisite --filter=Protect

Starting over

To remove all docker images, all mysql data, and all docker-related files from your local machine run:

yarn docker:clean

Note: this command does not work in Windows.

Using WP CLI

You can run WP CLI commands inside WordPress container:

yarn docker:wp COMMAND

For example run cron event list:

yarn docker:wp cron event list

shell is a handy wp-cli command you can use like so:

yarn docker:wp shell

Shell allows you to evaluate PHP code while having your installed WordPress loaded, so you could do things like:

wp> get_bloginfo( 'name' );
=> string(6) "WP-CLI"

Note that each wp shell session counts as a single request, causing unexpected situations with WP cache. You might want to run wp_cache_flush() between requests you expect to get cached by WordPress.

MySQL database

Connecting to your MySQL database from outside the container, use:

• Host: 127.0.0.1
• Port: 3306
• User: wordpress
• Pass: wordpress
• Database: wordpress

You can also see your database files via local filesystem at ./docker/data/mysql

SFTP access

You can access WordPress and Jetpack files via SFTP server container.

• Host: localhost
• Port: 1022
• User: wordpress
• Pass: wordpress
• WordPress path: /var/www/html

You can tunnel to this container using Ngrok or other similar service.

Tunnelling makes testing Jetpack Backup & Scan possible. Read more from "Using Ngrok with Jetpack" section below.

Must Use Plugins directory

You can add your own PHP code to ./docker/mu-plugins directory and they will be loaded by WordPress, in alphabetical order, before normal plugins, meaning API hooks added in an mu-plugin apply to all other plugins even if they run hooked-functions in the global namespace. Read more about must use plugins.

You can add your custom Jetpack constants (such as JETPACK__SANDBOX_DOMAIN) to a file under this folder. Automattic engineers can use this to sandbox their environment:

define( 'JETPACK__SANDBOX_DOMAIN', '{your sandbox}.wordpress.com' );

Using Ngrok with Jetpack

To be able to connect Jetpack you will need a domain - you can use Ngrok.com to assign one.

If you use one-off domains, you'll have to re-install WordPress and re-connect Jetpack each time you close Ngrok (thus losing your randomly assigned domain). That's perfectly fine for quick testing or lightweight development. You can use other similar services as well.

If you're developing Jetpack often you'll want to reserve a domain you can keep using.

If you are an Automattician, sign up on Ngrok.com using your a8c Google account; you'll be automattically added to the Automattic team. That will enable you to re-use domains, reserve your custom domains and reserve TCP ports.

Go to this page to reserve a permanent domain.

Once you’ve done that, follow these steps to download and set up ngrok. However, instead of step four, edit your config file as explained below:

authtoken: YOUR_AUTH_TOKEN # This should already be here
region: eu # only needed for subdomains in Europe (eu), Asia/Pacific (ap) or Australia (au)
tunnels:
  jetpack:
    subdomain: YOUR_RESERVED_SUBDOMAIN # without the .ngrok.io
    addr: 80
    proto: http

You can start your ngrok tunnel like so:

./ngrok start jetpack

These two commands are all you need to run to get Docker running when you start your computer:

./ngrok start jetpack
yarn docker:up -d

Docker Ngrok

Alternative to the above configuration file is running ngrok in the container with docker-compose file. That starts ngrok inside a container and you don't have to install it or configure as a standalone software on your machine.

1. Configure environment

Add these variables to your docker/.env file:

This configures example.us.ngrok.io reserved domain that is available on my basic plan. Possible values for NGROK_REGION are: (United States, default), eu (Europe), ap (Asia/Pacific) or au (Australia). Read more about ngrok regions

NGROK_AUTH=<your auth key>
NGROK_SUBDOMAIN=example
NGROK_REGION=us

2. Start docker with Ngrok

Start container with yarn docker:ngrok-up -d Stop container with yarn docker:ngrok-down -d

All the other docker-compose commands can be invoked via yarn docker:ngrok COMMAND

Configuration file

If you need more granular control over the Ngrok tunnel, you could create a configuration file. See default configuration file location from Ngrok Docs or use -config=your_config_file.yml argument with ngrok to use your configuration file.

Ngrok SFTP Tunnel with Jetpack

A sample config for adding an sftp tunnel to your Ngrok setup would look like this:

authtoken: YOUR_AUTH_TOKEN
tunnels:
  jetpack:
    subdomain: YOUR_PERMANENT_SUBDOMAIN
    addr: 80
    proto: http
  jetpack-sftp:
    addr: 1022
    proto: tcp
    remote_addr: 0.tcp.ngrok.io:YOUR_RESERVED_PORT

See more configuration options from Ngrok documentation.

You can now start both tunnels:

ngrok start jetpack jetpack-sftp

You can inspect traffic between your WordPress/Jetpack container and WordPress.com using the inspector.

Configuring Jetpack Backup & Scan with Ngrok tunnel

You should now be able to configure Jetpack Backup & Scan credentials point to your Docker container:

• Credential Type: SSH/SFTP
• Server Address: 0.tcp.ngrok.io
• Port Number: YOUR_RESERVED_PORT
• Server username: wordpress
• Server password: wordpress
• WordPress installation path: /var/www/html

Custom plugins & themes in the container

Jetpack Docker environment can be wonderful for developing your own plugins and themes, too.

Since everything under mu-plugins and wordpress/wp-content is git-ignored, you'll want to keep those folders outside Jetpack repository folder and link them as volumes to your Docker instance.

1. First ensure your containers are stopped (yarn docker:stop).
2. Create a docker-compose file. You can place it anywhere in your computer:

   version: '3.3'
   services:
     wordpress:
       volumes:
         - ~/my-plugin:/var/www/html/wp-content/plugins/my-plugin

   What comes before : is the path to your own plugin or theme, in your system. What comes after : is the path inside the Docker container. You can replace plugins/my-plugin with the path to your own plugin or theme.
3. Start containers and include your custom volumes by running:

   yarn docker:compose -f ~/docker-compose.my-volumes.yml up

You can pass multiple configuration files by adding more -f/--file arguments. Docker Compose combines them into a single configuration.

Debugging

Accessing logs

Logs are stored in your filesystem under ./docker/logs directory.

PHP error log

To tail -f the PHP error log, run:

yarn docker:tail

Debugging emails

Emails don’t leave your WordPress and are caught by MailDev SMTP server container instead.

To debug emails via web-interface, open http://localhost:1080

Debugging PHP with Xdebug

The WordPress image is leveraged with Xdebug present as a PHP Extension.

You’ll likely need to install a browser extension like the following:

• The easiest Xdebug for Mozilla Firefox
• Xdebug Helper for Google Chrome

Remote debugging with Atom editor

You’ll need to install the php-debug package for Atom. Features of this package include:

• Add Breakpoints
• Step through debugging (Over, In, Out)
• Stack and Context views
• Add Watchpoints to inspect current values of variables

Configuring Atom editor

1. Install php-debug package for your Atom editor.

2. Configure php-debug:

   1. To listen on all addresses (Server Address: 0.0.0.0)

   2. To map your current Jetpack directory to the docker filesystem path (Path Maps to /var/www/html/wp-content/plugins/jetpack;/local-path-in-your-computer/jetpack)

      

3. Make sure you installed the Chrome extension on your browser and configure it to send the IDE Key xdebug-atom

   • In the case of the Xdebug Helper extension, you get to set this by right-clicking (secondary click) on the extensions’ icon and clicking Options:

     

   • Set the IDE key field to Other, enter xdebug-atom in the text field, and press Save.

     

4. Going back to Atom, proceed to toggle debugging on from the Package Menu item:

   

   • Expect to see the debugger console window opening:

   

   • This window will read Listening on address port 0.0.0.0:9000 until you go to the WordPress site and refresh to make a new request. Then this window will read: Connected for a short time until the request ends. Note that it will also remain as such if you had added a breakpoint and the code flow has stopped:

   

5. You should be able to set breakpoints now:

   

Remote debugging with PhpStorm editor

Below are instructions for starting a debug session in PhpStorm that will listen to activity on your Jetpack docker.

1. Configure your browser extension to use 'PHPSTORM' for its session ID.

2. Open your Jetpack project in PhpStorm and chose 'Run -> Edit Configurations' from the main menu.

3. Click the '+' icon, and chose 'PHP Remote Debug' to create a new debug configuration.

4. Name your debug configuration whatever you like.

5. Check the 'Filter debug connection by IDE key', and enter 'PHPSTORM' for 'IDE Key ( Session ID )'.

6. Click the '...' on the 'Server' line to configure your remote server.

7. In the server configuration window, click the '+' icon to create a new server configuration. Name it whatever you like.

8. In the server configuration window, set your host to the URL you use to run Jetpack locally. ( Eg, localhost, or 0.0.0.0, or example.ngrok.io )

9. In the server configuration window, check the 'Use path mappings' checkbox.

10. In the server configuration window, map the main Jetpack folder to '/var/www/html/wp-content/plugins/jetpack' and map '/docker/wordpress' to '/var/www'

11. In the server configuration window, click 'Apply' then 'Ok'.

12. Back in the main configuration window, click 'Apply' then 'Ok'.

13. You can now start a debug session by clicking 'Run -> Debug' in the main menu