How We Included WordPress in Our Google Cloud Migration
Categories
Infrastructure & Operations
The approach and code snippets in this Google Cloud migration article reflect our use of macOS. If you are using Windows, Linux, etc., ensure that you source the correct configuration for your system.
When we decided to move our corporate website, xmatters.com, to Google Cloud Platform (GCP) more than a year ago, we also wanted to set up a pipeline for deployment. Specifically, our goals were to:
- Have a repeatable, controlled deployment process
- Automatically deploy to different environments
- Improve stability and performance
- Use engineering best practices (for us, this meant a corporate site where we could demo and test on a dev and staging server that was the same as production)
Both the Google Cloud migration and our goals presented a number of challenges. So if you’re looking to make a similar website move to GCP, you might save some time (and frustration!) by learning about our approach – and by liberally borrowing from the technical solutions we used to achieve our migration goals.
Dancing With Docker
The problem:
Using the default WordPress Docker image, we initially had a very complicated Docker file using gcsfuse for storage and running cloudsql proxy locally… and that was all really slow. We didn’t want to use a docker-compose.yml file because it was overkill – for example, GCP already handles our SQL/DB needs on the web. So, we simply installed MySQL to our machines for local development.
The solution:
Using GCP required us to add our credentials to the wp-config.php file on WordPress. Problematically, the WordPress Docker image creates the wp-config file but doesn’t have a very straightforward way to modify it without using a docker-compose.yml file…which we didn’t want to use. Ultimately, the solution was to modify the base Docker image file. That was a bit hacky and less-than-pretty, but it’s far less bloated and more effective than the alternative.
Technical details:
Our Docker image is based on the default WordPress image with some minor tweaks: we load from the standard WordPress:{version}-php:{version} image but add the entrypoint manually so we can enter our GOOGLE_APPLICATION_CREDENTIALS into wp-config.php:
sed -i "39i putenv('GOOGLE_APPLICATION_CREDENTIALS=/credentials/creds.json');" wp-config.php
We use entrypoint.sh from the base Docker WordPress image, adding our gcloud authentication so we can access our cloud SQL/DB.
All of our custom code is in a custom WordPress theme, so we check this into Git and handle the plugin installation with Composer and wpackagist.
#!/bin/bash
#!/bin/bash
set -euo pipefail
# usage: file_env VAR [DEFAULT]
# ie: file_env 'XYZ_DB_PASSWORD' 'example'
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of
# "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature)
file_env() {
local var="$1"
local fileVar="${var}_FILE"
local def="${2:-}"
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then
echo >&2 "error: both $var and $fileVar are set (but are exclusive)"
exit 1
fi
local val="$def"
if [ "${!var:-}" ]; then
val="${!var}"
elif [ "${!fileVar:-}" ]; then
val="$(< "${!fileVar}")"
fi
export "$var"="$val"
unset "$fileVar"
}
if [[ "$1" == apache2* ]] || [ "$1" == php-fpm ]; then
if ! [ -e index.php -a -e wp-includes/version.php ]; then
echo >&2 "WordPress not found in $PWD - copying now..."
if [ "$(ls -A)" ]; then
echo >&2 "WARNING: $PWD is not empty - press Ctrl+C now if this is an error!"
( set -x; ls -A; sleep 10 )
fi
tar cf - --one-file-system -C /usr/src/wordpress . | tar xf -
echo >&2 "Complete! WordPress has been successfully copied to $PWD"
if [ ! -e .htaccess ]; then
# NOTE: The "Indexes" option is disabled in the php:apache base image
cat > .htaccess <<-'EOF'
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress
EOF
chown www-data:www-data .htaccess
fi
fi
# TODO handle WordPress upgrades magically in the same way, but only if wp-includes/version.php's $wp_version is less than /usr/src/wordpress/wp-includes/version.php's $wp_version
# allow any of these "Authentication Unique Keys and Salts." to be specified via
# environment variables with a "WORDPRESS_" prefix (ie, "WORDPRESS_AUTH_KEY")
uniqueEnvs=(
AUTH_KEY
SECURE_AUTH_KEY
LOGGED_IN_KEY
NONCE_KEY
AUTH_SALT
SECURE_AUTH_SALT
LOGGED_IN_SALT
NONCE_SALT
)
envs=(
WORDPRESS_DB_HOST
WORDPRESS_DB_USER
WORDPRESS_DB_PASSWORD
WORDPRESS_DB_NAME
"${uniqueEnvs[@]/#/WORDPRESS_}"
WORDPRESS_TABLE_PREFIX
WORDPRESS_DEBUG
)
haveConfig=
for e in "${envs[@]}"; do
file_env "$e"
if [ -z "$haveConfig" ] && [ -n "${!e}" ]; then
haveConfig=1
fi
done
# linking backwards-compatibility
if [ -n "${!MYSQL_ENV_MYSQL_*}" ]; then
haveConfig=1
# host defaults to "mysql" below if unspecified
: "${WORDPRESS_DB_USER:=${MYSQL_ENV_MYSQL_USER:-root}}"
if [ "$WORDPRESS_DB_USER" = 'root' ]; then
: "${WORDPRESS_DB_PASSWORD:=${MYSQL_ENV_MYSQL_ROOT_PASSWORD:-}}"
else
: "${WORDPRESS_DB_PASSWORD:=${MYSQL_ENV_MYSQL_PASSWORD:-}}"
fi
: "${WORDPRESS_DB_NAME:=${MYSQL_ENV_MYSQL_DATABASE:-}}"
fi
# only touch "wp-config.php" if we have environment-supplied configuration values
if [ "$haveConfig" ]; then
: "${WORDPRESS_DB_HOST:=mysql}"
: "${WORDPRESS_DB_USER:=root}"
: "${WORDPRESS_DB_PASSWORD:=}"
: "${WORDPRESS_DB_NAME:=wordpress}"
# version 4.4.1 decided to switch to windows line endings, that breaks our seds and awks
# https://github.com/docker-library/wordpress/issues/116
# https://github.com/WordPress/WordPress/commit/1acedc542fba2482bab88ec70d4bea4b997a92e4
sed -ri -e 's/\r$//' wp-config*
if [ ! -e wp-config.php ]; then
awk '/^\/\*.*stop editing.*\*\/$/ && c == 0 { c = 1; system("cat") } { print }' wp-config-sample.php > wp-config.php <<'EOPHP'
// If we're behind a proxy server and using HTTPS, we need to alert WordPress of that fact
// see also http://codex.wordpress.org/Administration_Over_SSL#Using_a_Reverse_Proxy
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
$_SERVER['HTTPS'] = 'on';
}
EOPHP
chown www-data:www-data wp-config.php
fi
# see http://stackoverflow.com/a/2705678/433558
sed_escape_lhs() {
echo "$@" | sed -e 's/[]\/$*.^|[]/\\&/g'
}
sed_escape_rhs() {
echo "$@" | sed -e 's/[\/&]/\\&/g'
}
php_escape() {
php -r 'var_export(('$2') $argv[1]);' -- "$1"
}
set_config() {
key="$1"
value="$2"
var_type="${3:-string}"
start="(['\"])$(sed_escape_lhs "$key")\2\s*,"
end="\);"
if [ "${key:0:1}" = '$' ]; then
start="^(\s*)$(sed_escape_lhs "$key")\s*="
end=";"
fi
sed -ri -e "s/($start\s*).*($end)$/\1$(sed_escape_rhs "$(php_escape "$value" "$var_type")")\3/" wp-config.php
}
set_config 'DB_HOST' "$WORDPRESS_DB_HOST"
set_config 'DB_USER' "$WORDPRESS_DB_USER"
set_config 'DB_PASSWORD' "$WORDPRESS_DB_PASSWORD"
set_config 'DB_NAME' "$WORDPRESS_DB_NAME"
for unique in "${uniqueEnvs[@]}"; do
uniqVar="WORDPRESS_$unique"
if [ -n "${!uniqVar}" ]; then
set_config "$unique" "${!uniqVar}"
else
# if not specified, let's generate a random value
currentVal="$(sed -rn -e "s/define\((([\'\"])$unique\2\s*,\s*)(['\"])(.*)\3\);/\4/p" wp-config.php)"
if [ "$currentVal" = 'put your unique phrase here' ]; then
set_config "$unique" "$(head -c1m /dev/urandom | sha1sum | cut -d' ' -f1)"
fi
fi
done
if [ "$WORDPRESS_TABLE_PREFIX" ]; then
set_config '$table_prefix' "$WORDPRESS_TABLE_PREFIX"
fi
if [ "$WORDPRESS_DEBUG" ]; then
set_config 'WP_DEBUG' 1 boolean
fi
TERM=dumb php -- <<'EOPHP'
<?php
// database might not exist, so let's try creating it (just to be safe)
$stderr = fopen('php://stderr', 'w');
// https://codex.wordpress.org/Editing_wp-config.php#MySQL_Alternate_Port
// "hostname:port"
// https://codex.wordpress.org/Editing_wp-config.php#MySQL_Sockets_or_Pipes
// "hostname:unix-socket-path"
list($host, $socket) = explode(':', getenv('WORDPRESS_DB_HOST'), 2);
$port = 0;
if (is_numeric($socket)) {
$port = (int) $socket;
$socket = null;
}
$user = getenv('WORDPRESS_DB_USER');
$pass = getenv('WORDPRESS_DB_PASSWORD');
$dbName = getenv('WORDPRESS_DB_NAME');
$maxTries = 10;
do {
$mysql = new mysqli($host, $user, $pass, '', $port, $socket);
if ($mysql->connect_error) {
fwrite($stderr, "\n" . 'MySQL Connection Error: (' . $mysql->connect_errno . ') ' . $mysql->connect_error . "\n");
--$maxTries;
if ($maxTries <= 0) {
exit(1);
}
sleep(3);
}
} while ($mysql->connect_error);
if (!$mysql->query('CREATE DATABASE IF NOT EXISTS `' . $mysql->real_escape_string($dbName) . '`')) {
fwrite($stderr, "\n" . 'MySQL "CREATE DATABASE" Error: ' . $mysql->error . "\n");
$mysql->close();
exit(1);
}
$mysql->close();
EOPHP
fi
# now that we're definitely done writing configuration, let's clear out the relevant environment variables (so that stray "phpinfo()" calls don't leak secrets from our code)
for e in "${envs[@]}"; do
unset "$e"
done
fi
sed -i "39i putenv('GOOGLE_APPLICATION_CREDENTIALS=/credentials/creds.json');" wp-config.php
touch /var/www/html/health.html
exec "$@"
We have a DOCKERFILE that doesn’t download the latest WP version – we do that manually so no breaks occur:
FROM wordpress:5.4.1-php7.3
RUN apt-get update && \
apt-get -y upgrade && \
apt-get -y install sudo && \
rm -rf /var/lib/apt/lists/*
EXPOSE 80
EXPOSE 443
RUN a2enmod ssl && \
a2enmod headers && \
a2ensite default-ssl
RUN touch /var/log/apache2/php_err.log && \
chown www-data:www-data /var/log/apache2/php_err.log
COPY ./php/php_error.ini /usr/local/etc/php/conf.d/php_error.ini
COPY ./src/.htaccess /usr/src/wordpress/.htaccess
COPY ./src/robots.txt /usr/src/wordpress/robots.txt
COPY ./build/wp-content/ /usr/src/wordpress/wp-content/
COPY entrypoint.sh /usr/local/bin/
RUN chmod 777 /usr/local/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
CMD ["apache2-foreground"]
Next, we add a Makefile to create the project, push to gcloud, and run composer-update to install plugins.
version := $(shell git rev-parse --short HEAD)
gcr_proj := xmatters-eng-mgmt
build_number := $$(if [ "$${BUILD_NUMBER}" == "" ]; then BUILD_NUMBER=1; fi; echo $${BUILD_NUMBER})
git_branch := $$(if [ "$${GIT_BRANCH}" == "" ]; then GIT_BRANCH=$$(echo $$(git rev-parse --abbrev-ref HEAD)); fi; echo $${GIT_BRANCH} | sed 's/^origin\/\(remote\/\)\?//g')
git_branch_sanatized := $$(if [ "$${GIT_BRANCH}" == "" ]; then GIT_BRANCH=$$(echo $$(git rev-parse --abbrev-ref HEAD)); fi; echo $${GIT_BRANCH,,} | sed 's/[-\/]/_/g' | sed 's/^origin_\(remote_\)\?//g')
image_name := gcr.io/${gcr_proj}/mktgwebsite
full_image := ${image_name}:${version}
local_port := 8080
create_properties_file:
echo "VERSION=${version}" > gcpBuildVersion.properties
echo "GIT_HASH=${version}" >> gcpBuildVersion.properties
echo "BUILD=${build_number}" >> gcpBuildVersion.properties
echo "BRANCH=${git_branch}" >> gcpBuildVersion.properties
echo "TAG=${version}" >> gcpBuildVersion.properties
docker-image: create_properties_file
docker build --squash -t "${full_image}" .
docker-run: build docker-image
docker run --rm --name wordpress \
-e WORDPRESS_DB_NAME=your_db_name \
-e WORDPRESS_DB_HOST=docker.for.mac.localhost \
-e WORDPRESS_DB_USER=root \
-e WORDPRESS_DB_PASSWORD= \
-e WORDPRESS_TABLE_PREFIX=wp_ \
-v ${PWD}/build/wp-content:/var/www/html/wp-content \
-v ${PWD}/../credentials:/credentials \
-p 80:80 -p 443:443 "${full_image}"
docker-push: docker-image
docker push "${image_name}:${version}"
composer-update: grunt-build
docker pull composer:1.6 && docker run --rm -v `pwd`:`pwd` -w `pwd` composer:1.6 update -vvv
grunt-build: create-build-image
docker run --rm -v `pwd`:`pwd` -w `pwd` xm_corp_site/build /bin/bash -c "grunt build:release"
create-build-image:
docker build -t xm_corp_site/build ./build-env/
clean:
rm -rf build/*
build: composer-update
build/:
mkdir -p build
We make a Composer file for handling our plugins. Fortunately, most of the plugins our site uses were available on wpackagist… for the ones that aren’t, we have them checked into Git. Yes, manually updating plugins sux. 🙁
We have a Gruntfile.js file to handle creating the build/ directory after Docker starts. We run our new build:release command which recreates the build folder completely, including all the plugins, and we have a build:dev command for local development. Build:dev doesn’t remove the plugins (which take a while to reinstall). We use Sass and JS, and compile and minify them using Grunt to keep the code base clean and centralized.
'use strict';
module.exports = function(grunt) {
var buildThemePath ='build/wp-content/themes/theme/';
var GruntConfig = {
clean:{
release: ['build/wp-content/*'],
dev: ['build/wp-content/themes/*']
},
uglify: {
options: {
mangle: true,
compress: true,
warnings: false,
preserveComments: false
},
release: {
files: {
'build/wp-content/themes/theme/js/app.js': ['src/themes/theme/js/app.js'],
}
}
},
copy: {
main: {
files: [{
expand: true,
cwd: 'src/themes/theme',
src: ['**/*', '!.sass-cache/*', '!sass', '!sass/**', '!js/**', '!js_src/**'],
dest: buildThemePath ,
nonull: true
},{
expand: true,
cwd: 'src/themes/theme/js/',
src: ['**/*'],
dest: buildThemePath + 'js/',
nonull: true
},{
expand: true,
cwd: 'src/plugins',
src: ['**/*'],
dest: 'build/wp-content/plugins/' ,
nonull: true
}]
}
},
autoprefixer: {
options: {
browsers: ['Last 2 versions', '> 0.5%', 'ie 10', 'firefox 54', 'safari 10'],
safe: true,
diff: false
},
multiple_files: {
expand: true,
flatten: true,
src: 'src/themes/theme/css/*.css',
dest: 'src/themes/theme/css/'
}
},
sass: {
options: {
sourceMap: true,
outputStyle: 'expanded',
sourceComments: false
},
dist: {
files: {
'src/themes/theme/css/styles.css': 'src/themes/theme/sass/styles.scss',
'build/wp-content/themes/theme/css/styles.css': 'src/themes/theme/sass/styles.scss',
},
}
},
cssmin: {
target: {
files: {
'build/wp-content/themes/theme/css/styles.css' : 'src/themes/theme/css/styles.css'
}
}
},
concat: {
options: {
seperator: ";"
},
dev: {
src: ['src/themes/theme/js_src/_*.js', 'src/themes/theme/js_src/app.js'],
dest: 'src/themes/theme/js/app.js',
nonull: true
},
release: {
src: ['src/themes/theme/js_src/_*.js', 'src/themes/theme/js_src/app.js'],
dest: 'src/themes/theme/js/app.js',
nonull: true
}
},
};
grunt.initConfig(GruntConfig);
grunt.loadNpmTasks('grunt-sass');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-contrib-copy');
grunt.loadNpmTasks('grunt-contrib-uglify');
grunt.loadNpmTasks('grunt-autoprefixer');
grunt.loadNpmTasks('grunt-contrib-cssmin');
//tasks
grunt.registerTask('build:dev', ['clean:dev', 'copy', 'concat:dev', 'sass', 'uglify']);
grunt.registerTask('build:release', ['clean:release', 'copy', 'concat:release', 'sass', 'autoprefixer', 'cssmin', 'uglify']);
grunt.registerTask('build', ['build:dev']);
grunt.registerTask('default', ['build:dev']);
};
We make a couple of jenkinsfiles that control our pipeline deployment. I won’t go into the details of this but in a nutshell, we set up YAML templates and connect with Stash.
GCP connection
We connect to the Google Cloud SQL instance using cloud SQL proxy.
CDN image hosting
Lastly, to host our images on a Content Delivery Network (CDN), we use the WP-Stateless plugin to connect to our GCP bucket.
Migrate-ness!
All that upfront setup and configuration work has resulted in some great long-term gains:
- Database is in Cloud SQL and connects through cloudsql proxy container
- Cloud SQL does lots: automatic backups, replication, one-button failover
- We move our content to a CDN using WP-Stateless plugin
- All our Plugins are moved to being installed with Composer
For our unique setup, we created a pipeline that’s leaner than the basic Docker/YML set up, uses GCP for both the DB and the CDN, and automates the installation of plugins, scripts, and styles, and is also easy to use for local development. While your setup is very likely different than ours, you might face some of the same challenges we did. We hope that learning some of the technical details about our Google Cloud migration makes yours just a bit smoother!