xMatters has released an official statement and workshops to help customers navigate through COVID-19

Intelligent Communications Blog

BACK TO BLOG

How We Included WordPress in Our Google Cloud Migration

Brianna Thomas ON Jul 29, 2020

The approach and code snippets in this Google Cloud migration article reflect our use of macOS. If you are using Windows, Linux, etc., ensure that you source the correct configuration for your system.

When we decided to move our corporate website, xmatters.com, to Google Cloud Platform (GCP) more than a year ago, we also wanted to set up a pipeline for deployment. Specifically, our goals were to:

  • Have a repeatable, controlled deployment process
  • Automatically deploy to different environments
  • Improve stability and performance
  • Use engineering best practices (for us, this meant a corporate site where we could demo and test on a dev and staging server that was the same as production)

Both the Google Cloud migration and our goals presented a number of challenges. So if you’re looking to make a similar website move to GCP, you might save some time (and frustration!) by learning about our approach – and by liberally borrowing from the technical solutions we used to achieve our migration goals.

Dancing With Docker

The problem:
Using the default WordPress Docker image, we initially had a very complicated Docker file using gcsfuse for storage and running cloudsql proxy locally… and that was all really slow. We didn’t want to use a docker-compose.yml file because it was overkill – for example, GCP already handles our SQL/DB needs on the web. So, we simply installed MySQL to our machines for local development.

The solution:
Using GCP required us to add our credentials to the wp-config.php file on WordPress. Problematically, the WordPress Docker image creates the wp-config file but doesn’t have a very straightforward way to modify it without using a docker-compose.yml file…which we didn’t want to use. Ultimately, the solution was to modify the base Docker image file. That was a bit hacky and less-than-pretty, but it’s far less bloated and more effective than the alternative.

Technical details:
Our Docker image is based on the default WordPress image with some minor tweaks: we load from the standard WordPress:{version}-php:{version} image but add the entrypoint manually so we can enter our GOOGLE_APPLICATION_CREDENTIALS into wp-config.php:

sed -i "39i  putenv('GOOGLE_APPLICATION_CREDENTIALS=/credentials/creds.json');" wp-config.php

We use entrypoint.sh from the base Docker WordPress image, adding our gcloud authentication so we can access our cloud SQL/DB.

All of our custom code is in a custom WordPress theme, so we check this into Git and handle the plugin installation with Composer and wpackagist.

#!/bin/bash 
#!/bin/bash 
set -euo pipefail 
 
# usage: file_env VAR [DEFAULT] 
#    ie: file_env 'XYZ_DB_PASSWORD' 'example' 
# (will allow for "$XYZ_DB_PASSWORD_FILE" to fill in the value of 
#  "$XYZ_DB_PASSWORD" from a file, especially for Docker's secrets feature) 
file_env() { 
local var="$1" 
local fileVar="${var}_FILE" 
local def="${2:-}" 
if [ "${!var:-}" ] && [ "${!fileVar:-}" ]; then 
  echo >&2 "error: both $var and $fileVar are set (but are exclusive)" 
  exit 1 
fi 
local val="$def" 
if [ "${!var:-}" ]; then 
  val="${!var}" 
elif [ "${!fileVar:-}" ]; then 
  val="$(< "${!fileVar}")" 
fi 
export "$var"="$val" 
unset "$fileVar" 
} 
 
if [[ "$1" == apache2* ]] || [ "$1" == php-fpm ]; then 
if ! [ -e index.php -a -e wp-includes/version.php ]; then 
  echo >&2 "WordPress not found in $PWD - copying now..." 
  if [ "$(ls -A)" ]; then 
   echo >&2 "WARNING: $PWD is not empty - press Ctrl+C now if this is an error!" 
   ( set -x; ls -A; sleep 10 ) 
  fi 
  tar cf - --one-file-system -C /usr/src/wordpress . | tar xf - 
  echo >&2 "Complete! WordPress has been successfully copied to $PWD" 
  if [ ! -e .htaccess ]; then 
   # NOTE: The "Indexes" option is disabled in the php:apache base image 
   cat > .htaccess <<-'EOF' 
    # BEGIN WordPress 
    <IfModule mod_rewrite.c> 
    RewriteEngine On 
    RewriteBase / 
    RewriteRule ^index\.php$ - [L] 
    RewriteCond %{REQUEST_FILENAME} !-f 
    RewriteCond %{REQUEST_FILENAME} !-d 
    RewriteRule . /index.php [L] 
    </IfModule> 
    # END WordPress 
   EOF 
   chown www-data:www-data .htaccess 
  fi 
fi 
 
# TODO handle WordPress upgrades magically in the same way, but only if wp-includes/version.php's $wp_version is less than /usr/src/wordpress/wp-includes/version.php's $wp_version 
 
# allow any of these "Authentication Unique Keys and Salts." to be specified via 
# environment variables with a "WORDPRESS_" prefix (ie, "WORDPRESS_AUTH_KEY") 
uniqueEnvs=( 
  AUTH_KEY 
  SECURE_AUTH_KEY 
  LOGGED_IN_KEY 
  NONCE_KEY 
  AUTH_SALT 
  SECURE_AUTH_SALT 
  LOGGED_IN_SALT 
  NONCE_SALT 
) 
envs=( 
  WORDPRESS_DB_HOST 
  WORDPRESS_DB_USER 
  WORDPRESS_DB_PASSWORD 
  WORDPRESS_DB_NAME 
  "${uniqueEnvs[@]/#/WORDPRESS_}" 
  WORDPRESS_TABLE_PREFIX 
  WORDPRESS_DEBUG 
) 
haveConfig= 
for e in "${envs[@]}"; do 
  file_env "$e" 
  if [ -z "$haveConfig" ] && [ -n "${!e}" ]; then 
   haveConfig=1 
  fi 
done 
 
# linking backwards-compatibility 
if [ -n "${!MYSQL_ENV_MYSQL_*}" ]; then 
  haveConfig=1 
  # host defaults to "mysql" below if unspecified 
  : "${WORDPRESS_DB_USER:=${MYSQL_ENV_MYSQL_USER:-root}}" 
  if [ "$WORDPRESS_DB_USER" = 'root' ]; then 
   : "${WORDPRESS_DB_PASSWORD:=${MYSQL_ENV_MYSQL_ROOT_PASSWORD:-}}" 
  else 
   : "${WORDPRESS_DB_PASSWORD:=${MYSQL_ENV_MYSQL_PASSWORD:-}}" 
  fi 
  : "${WORDPRESS_DB_NAME:=${MYSQL_ENV_MYSQL_DATABASE:-}}" 
fi 
 
# only touch "wp-config.php" if we have environment-supplied configuration values 
if [ "$haveConfig" ]; then 
  : "${WORDPRESS_DB_HOST:=mysql}" 
  : "${WORDPRESS_DB_USER:=root}" 
  : "${WORDPRESS_DB_PASSWORD:=}" 
  : "${WORDPRESS_DB_NAME:=wordpress}" 
 
  # version 4.4.1 decided to switch to windows line endings, that breaks our seds and awks 
  # https://github.com/docker-library/wordpress/issues/116 
  # https://github.com/WordPress/WordPress/commit/1acedc542fba2482bab88ec70d4bea4b997a92e4 
  sed -ri -e 's/\r$//' wp-config* 
 
  if [ ! -e wp-config.php ]; then 
   awk '/^\/\*.*stop editing.*\*\/$/ && c == 0 { c = 1; system("cat") } { print }' wp-config-sample.php > wp-config.php <<'EOPHP' 
// If we're behind a proxy server and using HTTPS, we need to alert WordPress of that fact 
// see also http://codex.wordpress.org/Administration_Over_SSL#Using_a_Reverse_Proxy 
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') { 
$_SERVER['HTTPS'] = 'on'; 
} 
EOPHP 
   chown www-data:www-data wp-config.php 
  fi 
 
  # see http://stackoverflow.com/a/2705678/433558 
  sed_escape_lhs() { 
   echo "$@" | sed -e 's/[]\/$*.^|[]/\\&/g' 
  } 
  sed_escape_rhs() { 
   echo "$@" | sed -e 's/[\/&]/\\&/g' 
  } 
  php_escape() { 
   php -r 'var_export(('$2') $argv[1]);' -- "$1" 
  } 
  set_config() { 
   key="$1" 
   value="$2" 
   var_type="${3:-string}" 
   start="(['\"])$(sed_escape_lhs "$key")\2\s*," 
   end="\);" 
   if [ "${key:0:1}" = '$' ]; then 
    start="^(\s*)$(sed_escape_lhs "$key")\s*=" 
    end=";" 
   fi 
   sed -ri -e "s/($start\s*).*($end)$/\1$(sed_escape_rhs "$(php_escape "$value" "$var_type")")\3/" wp-config.php 
  } 
 
  set_config 'DB_HOST' "$WORDPRESS_DB_HOST" 
  set_config 'DB_USER' "$WORDPRESS_DB_USER" 
  set_config 'DB_PASSWORD' "$WORDPRESS_DB_PASSWORD" 
  set_config 'DB_NAME' "$WORDPRESS_DB_NAME" 
 
  for unique in "${uniqueEnvs[@]}"; do 
   uniqVar="WORDPRESS_$unique" 
   if [ -n "${!uniqVar}" ]; then 
    set_config "$unique" "${!uniqVar}" 
   else 
    # if not specified, let's generate a random value 
    currentVal="$(sed -rn -e "s/define\((([\'\"])$unique\2\s*,\s*)(['\"])(.*)\3\);/\4/p" wp-config.php)" 
    if [ "$currentVal" = 'put your unique phrase here' ]; then 
     set_config "$unique" "$(head -c1m /dev/urandom | sha1sum | cut -d' ' -f1)" 
    fi 
   fi 
  done 
 
  if [ "$WORDPRESS_TABLE_PREFIX" ]; then 
   set_config '$table_prefix' "$WORDPRESS_TABLE_PREFIX" 
  fi 
 
  if [ "$WORDPRESS_DEBUG" ]; then 
   set_config 'WP_DEBUG' 1 boolean 
  fi 
  TERM=dumb php -- <<'EOPHP' 
<?php 
// database might not exist, so let's try creating it (just to be safe) 
$stderr = fopen('php://stderr', 'w'); 
// https://codex.wordpress.org/Editing_wp-config.php#MySQL_Alternate_Port 
//   "hostname:port" 
// https://codex.wordpress.org/Editing_wp-config.php#MySQL_Sockets_or_Pipes 
//   "hostname:unix-socket-path" 
list($host, $socket) = explode(':', getenv('WORDPRESS_DB_HOST'), 2); 
$port = 0; 
if (is_numeric($socket)) { 
$port = (int) $socket; 
$socket = null; 
} 
$user = getenv('WORDPRESS_DB_USER'); 
$pass = getenv('WORDPRESS_DB_PASSWORD'); 
$dbName = getenv('WORDPRESS_DB_NAME'); 
$maxTries = 10; 
do { 
$mysql = new mysqli($host, $user, $pass, '', $port, $socket); 
if ($mysql->connect_error) { 
  fwrite($stderr, "\n" . 'MySQL Connection Error: (' . $mysql->connect_errno . ') ' . $mysql->connect_error . "\n"); 
  --$maxTries; 
  if ($maxTries <= 0) { 
   exit(1); 
  } 
  sleep(3); 
} 
} while ($mysql->connect_error); 
if (!$mysql->query('CREATE DATABASE IF NOT EXISTS `' . $mysql->real_escape_string($dbName) . '`')) { 
fwrite($stderr, "\n" . 'MySQL "CREATE DATABASE" Error: ' . $mysql->error . "\n"); 
$mysql->close(); 
exit(1); 
} 
$mysql->close(); 
EOPHP 
fi 
 
# now that we're definitely done writing configuration, let's clear out the relevant environment variables (so that stray "phpinfo()" calls don't leak secrets from our code) 
for e in "${envs[@]}"; do 
  unset "$e" 
done 
fi 
 
sed -i "39i  putenv('GOOGLE_APPLICATION_CREDENTIALS=/credentials/creds.json');" wp-config.php 
 
touch /var/www/html/health.html 
 
exec "$@" 

 

We have a DOCKERFILE that doesn’t download the latest WP version – we do that manually so no breaks occur:

FROM wordpress:5.4.1-php7.3 
 
RUN   apt-get update && \ 
      apt-get -y upgrade && \ 
      apt-get -y install sudo && \ 
      rm -rf /var/lib/apt/lists/* 
 
EXPOSE 80 
EXPOSE 443 
 
RUN a2enmod ssl && \ 
      a2enmod headers && \ 
       a2ensite default-ssl 
RUN touch /var/log/apache2/php_err.log && \ 
    chown www-data:www-data /var/log/apache2/php_err.log 
 
COPY ./php/php_error.ini /usr/local/etc/php/conf.d/php_error.ini 
COPY ./src/.htaccess /usr/src/wordpress/.htaccess 
COPY ./src/robots.txt /usr/src/wordpress/robots.txt 
COPY ./build/wp-content/ /usr/src/wordpress/wp-content/ 
COPY entrypoint.sh /usr/local/bin/ 
RUN chmod 777 /usr/local/bin/entrypoint.sh 
 
ENTRYPOINT ["entrypoint.sh"] 
 
CMD ["apache2-foreground"] 

Next, we add a Makefile to create the project, push to gcloud, and run composer-update to install plugins.

version := $(shell git rev-parse --short HEAD) 
gcr_proj := xmatters-eng-mgmt 
build_number := $$(if [ "$${BUILD_NUMBER}" == "" ]; then BUILD_NUMBER=1; fi; echo $${BUILD_NUMBER}) 
git_branch := $$(if [ "$${GIT_BRANCH}" == "" ]; then GIT_BRANCH=$$(echo $$(git rev-parse --abbrev-ref HEAD)); fi; echo $${GIT_BRANCH} | sed 's/^origin\/\(remote\/\)\?//g') 
git_branch_sanatized := $$(if [ "$${GIT_BRANCH}" == "" ]; then GIT_BRANCH=$$(echo $$(git rev-parse --abbrev-ref HEAD)); fi; echo $${GIT_BRANCH,,} | sed 's/[-\/]/_/g' | sed 's/^origin_\(remote_\)\?//g') 
image_name := gcr.io/${gcr_proj}/mktgwebsite 
full_image := ${image_name}:${version} 
local_port := 8080 
 
 
create_properties_file: 
   echo "VERSION=${version}" > gcpBuildVersion.properties 
   echo "GIT_HASH=${version}" >> gcpBuildVersion.properties 
   echo "BUILD=${build_number}" >> gcpBuildVersion.properties 
   echo "BRANCH=${git_branch}" >> gcpBuildVersion.properties 
   echo "TAG=${version}" >> gcpBuildVersion.properties 
 
docker-image: create_properties_file 
   docker build --squash -t "${full_image}" . 
 
docker-run: build docker-image 
                     docker run --rm  --name wordpress \ 
                     -e WORDPRESS_DB_NAME=your_db_name \ 
                     -e WORDPRESS_DB_HOST=docker.for.mac.localhost \ 
                     -e WORDPRESS_DB_USER=root \ 
                     -e WORDPRESS_DB_PASSWORD= \ 
                     -e WORDPRESS_TABLE_PREFIX=wp_ \ 
                     -v ${PWD}/build/wp-content:/var/www/html/wp-content \ 
                     -v ${PWD}/../credentials:/credentials \ 
                     -p 80:80 -p 443:443  "${full_image}" 
 
docker-push: docker-image 
   docker push "${image_name}:${version}" 
 
composer-update: grunt-build 
   docker pull composer:1.6 && docker run --rm -v `pwd`:`pwd` -w `pwd`  composer:1.6 update -vvv 
 
grunt-build: create-build-image 
   docker run --rm -v `pwd`:`pwd` -w `pwd`  xm_corp_site/build /bin/bash -c "grunt build:release" 
 
create-build-image: 
   docker build -t xm_corp_site/build ./build-env/ 
 
clean: 
   rm -rf build/* 
 
build:  composer-update 
 
build/: 
   mkdir -p build 

We make a Composer file for handling our plugins. Fortunately, most of the plugins our site uses were available on wpackagist… for the ones that aren’t, we have them checked into Git. Yes, manually updating plugins sux. 🙁

We have a Gruntfile.js file to handle creating the build/ directory after Docker starts. We run our new build:release command which recreates the build folder completely, including all the plugins, and we have a build:dev command for local development. Build:dev doesn’t remove the plugins (which take a while to reinstall). We use Sass and JS, and compile and minify them using Grunt to keep the code base clean and centralized.

'use strict'; 
 
module.exports = function(grunt) { 
    var buildThemePath ='build/wp-content/themes/theme/'; 
   var GruntConfig = { 
      clean:{ 
            release: ['build/wp-content/*'], 
            dev: ['build/wp-content/themes/*'] 
      }, 
      uglify: { 
         options: { 
            mangle: true, 
            compress: true, 
            warnings: false, 
            preserveComments: false 
         }, 
         release: { 
            files: { 
               'build/wp-content/themes/theme/js/app.js': ['src/themes/theme/js/app.js'], 
            } 
         } 
      }, 
      copy: { 
         main: { 
            files: [{ 
               expand: true, 
               cwd: 'src/themes/theme', 
               src: ['**/*', '!.sass-cache/*', '!sass', '!sass/**', '!js/**', '!js_src/**'], 
               dest: buildThemePath , 
               nonull: true 
            },{ 
               expand: true, 
               cwd: 'src/themes/theme/js/', 
               src: ['**/*'], 
               dest: buildThemePath + 'js/', 
               nonull: true 
            },{ 
                    expand: true, 
                    cwd: 'src/plugins', 
                    src: ['**/*'], 
                    dest: 'build/wp-content/plugins/' , 
                    nonull: true 
                }] 
 
         } 
 
      }, 
      autoprefixer: { 
         options: { 
            browsers: ['Last 2 versions', '> 0.5%', 'ie 10', 'firefox 54', 'safari 10'], 
            safe: true, 
                diff: false 
         }, 
         multiple_files: { 
               expand: true, 
               flatten: true, 
               src: 'src/themes/theme/css/*.css', 
               dest: 'src/themes/theme/css/' 
         } 
      }, 
 
      sass: { 
         options: { 
            sourceMap: true, 
            outputStyle: 'expanded', 
            sourceComments: false 
         }, 
         dist: { 
            files: { 
               'src/themes/theme/css/styles.css': 'src/themes/theme/sass/styles.scss', 
               'build/wp-content/themes/theme/css/styles.css': 'src/themes/theme/sass/styles.scss', 
            }, 
         } 
      }, 
        cssmin: { 
            target: { 
                files: { 
                    'build/wp-content/themes/theme/css/styles.css' : 'src/themes/theme/css/styles.css' 
                } 
            } 
        }, 
      concat: { 
         options: { 
            seperator: ";" 
         }, 
         dev: { 
            src: ['src/themes/theme/js_src/_*.js', 'src/themes/theme/js_src/app.js'], 
            dest: 'src/themes/theme/js/app.js', 
            nonull: true 
         }, 
         release: { 
            src: ['src/themes/theme/js_src/_*.js', 'src/themes/theme/js_src/app.js'], 
            dest: 'src/themes/theme/js/app.js', 
            nonull: true 
         } 
      }, 
   }; 
 
   grunt.initConfig(GruntConfig); 
 
   grunt.loadNpmTasks('grunt-sass'); 
   grunt.loadNpmTasks('grunt-contrib-concat'); 
   grunt.loadNpmTasks('grunt-contrib-clean'); 
   grunt.loadNpmTasks('grunt-contrib-copy'); 
   grunt.loadNpmTasks('grunt-contrib-uglify'); 
   grunt.loadNpmTasks('grunt-autoprefixer'); 
    grunt.loadNpmTasks('grunt-contrib-cssmin'); 
 
   //tasks 
   grunt.registerTask('build:dev', ['clean:dev', 'copy', 'concat:dev', 'sass', 'uglify']); 
   grunt.registerTask('build:release', ['clean:release', 'copy', 'concat:release', 'sass', 'autoprefixer', 'cssmin', 'uglify']); 
   grunt.registerTask('build', ['build:dev']); 
   grunt.registerTask('default', ['build:dev']); 
}; 

We make a couple of jenkinsfiles that control our pipeline deployment. I won’t go into the details of this but in a nutshell, we set up YAML templates and connect with Stash.

GCP connection

We connect to the Google Cloud SQL instance using cloud SQL proxy.

CDN image hosting

Lastly, to host our images on a Content Delivery Network (CDN), we use the WP-Stateless plugin to connect to our GCP bucket.

Migrate-ness!

All that upfront setup and configuration work has resulted in some great long-term gains:

  • Database is in Cloud SQL and connects through cloudsql proxy container
  • Cloud SQL does lots: automatic backups, replication, one-button failover
  • We move our content to a CDN using WP-Stateless plugin
  • All our Plugins are moved to being installed with Composer

For our unique setup, we created a pipeline that’s leaner than the basic Docker/YML set up, uses GCP for both the DB and the CDN, and automates the installation of plugins, scripts, and styles, and is also easy to use for local development. While your setup is very likely different than ours, you might face some of the same challenges we did. We hope that learning some of the technical details about our Google Cloud migration makes yours just a bit smoother!

Try xMatters Free