Professional Documents
Culture Documents
By Scott Farrell
Copyright 2017 By Scott Farrell
All rights reserved. No part of this publication may be
reproduced, distributed, or transmitted in any form or by
any means, including photocopying, recording, or other
electronic or mechanical methods, without the prior written
permission of the publisher, except in the case of brief
quotations embodied in critical reviews and certain other
noncommercial uses permitted by copyright law. For
permission requests, write to the publisher, addressed
Attention: Permissions Coordinator, at the address below.
scott@wpdone.com.au
www.wpdone.com.au
Ordering Information:
Quantity sales. Special discounts are available on quantity
purchases by corporations, associations, and others. For
details, contact the publisher at the address above.
Printed in Australia
First Edition
Table of Contents
Introduction 4
Chapter 1. The Need 6
Chapter 2. Performance Analysis 12
Chapter 3. Fixing Macro Performance Issues 25
Chapter 4. Fixing Micro Performance Issues 34
Chapter 5. WooCommerce Optimization 57
Chapter 6. Scaling WordPress 58
Chapter 7. The Ultimate WordPress Hosting Stack 60
Chapter 8. Whats Next? 65
Chapter 9. Hosting Recipes 66
Chapter 10. Php-fpm Memory Usage 68
Chapter 11. Case Study 69
Introduction
I wrote this book for a number of reasons. First of all, making a WordPress
website fast is a big, complex subject. There is a lot of confusion, mixed
messages, and deception out there. I also really love working with WordPress
hosting as well as helping customers and agencies.
There is a bunch of really poor hosting out there. How do I explain why certain
hosting is poor? By going into some detail about what good hosting looks like.
Plus, you need to be able to critically analyze the hosting that you are using to
work out whether its appropriate.
I also see many people upgrading from shared hosting to VPS hosting, but for
most people, this is an out of the frying pan and into the fire situation. VPS
hosting results in more complexity, time, risk, and problems.
Some areas of complexity are what brand of hosting youre using, what type of
hosting youre using, the software, the hardware, the versions of php, which
web server you use, and how the MySql database affects the speed. There are
also plugins, themes, caching, and CDNs to consider.
The complexity also interacts with itself. Now, there are plenty of Band-Aid
approacheslike slapping on pluginsbut people often use the wrong tools to
test WordPress website performance.
Yes, another obvious reason for writing this book is to generate business. It
shows that Im serious about hosting and can lend some valuable insight to any
hosting decisions you might have.
About My Business
I run www.wpdone.com.au. It is WordPress hosting done right, done fast, and
done securely.
Wpdone works with professional agencies in Australia to provide premium
hosting without the sysadmin headaches. We use a revenue-sharing model that
helps to deliver premium services to your clients while allowing you to put
more money in your pocket than you do now. You spend less time yourself, less
staff time, and fewer support phone calls from your clientswe just send you
a direct debit each month.
We do a better job than the big names in hosting. This book will help you better
understand your needs, what your current provider is delivering, and how you
can do better.
We do our hosting here in Australia to better support your business and your
clients businesses.
We also operate a Virtual CTO program to help those with hosting scenarios
that dont exactly fit our main hosting. If youre offshore or want to stay with
your provider but want access to our leadership, deep skills, and intellectual
property, we can help: https://www.wpdone.com.au/virtual-cto/.
Chapter 1. The Need
Why should you consider the performance of your WordPress site?
When PCs were slow, people were used to waiting for websites.
However, PCs are getting faster every day, so users notice a slow website
when they see it. Having a slow website is the same as having old or faded
paint on your storefront.
A website that was fast enough a few years ago probably isnt fast enough by
todays standards.
Google SEO
Okay, I know that some people might want to argue this one.
Google SEO is to some extent a secret and not completely knowable. Its also
difficult to understand the extent to which the performance of your website
impacts your SEO rankings.
However, few will argue that website response speed is an indicator for
Google SEO.
Some of you dont pay for SEM/AdWords or an SEO consultant (Ahhpaying
monthly for SEO magicanother pet peeve of mine). For those of you who do,
a little more attention paid to performance can sometimes save money as a
result of better SEO ranking.
Conversion Rates
There are a few documents floating around about how your websites
performance relates to conversion rates.
If youre selling on your website, or even just collecting leads, the
performance on your site will have a direct impact on your conversion rate.
Say your website is loading in seven seconds; over a year, youll have lost
15%35% of your eCommerce sales. That is a simple business decision for an
eCommerce website to move to faster hosting.
Delight Customers
This is a straight marketing concept. I remember hearing about it when studying
for my masters in management. I think it was Drucker, but I could be wrong.
The concept is to exceed your customers expectations. Dont merely have a
good enough website; make sure it delights everyone who uses it.
Google ZMOT (zero moment of truth) says that customers have already made
up their minds before they enter your real store. They are checking out you and
competitors online, so make sure your first impression is good. Read more
about ZMOT here: https://www.thinkwithgoogle.com/collections/zero-
moment-truth.html.
Before you jump headlong into fixing your WordPress performance, take a
moment to take stock of where youre at.
I see loads of folks spending their weekends adding this or that plugin. Why a
weekend? Because adding stuff to your site causes other stuff to break, so then
you need a support call. Then you need to learn that tech a bit to get your
WordPress site working again (even if its just a plugin), but you also have to
go out to dinner with friends and see your childs play. Before you know it, you
didnt get the mowing done and its Sunday evening, but your website is a bit
fasterjust not as much as you expected.
That weekend would have been better spent with a little more analysis and
planning rather than semi-random plugins and changes.
This whole performance and analysis concept is complicated, and Ill try to
stop you from going down the rabbit hole. You need to take a broad look at
whats going onas well as at your optionsbefore you dive into the
minutiae.
Im breaking the rest of the ebook into a few sections.
- Analysis Steps 3 Major Things You Need to Review:
- Server Response
- On-page Performance
- Server Tech Level
- Macro Changes Changes Outside of WordPress and the Server
- Micro Changes Changes Within WordPress or Its Server
Basically, you need to make sure you have some idea of whats going on with
your site. Then you need to consider the bigger questions. Once you have the
basics down, we can dive into the details of the tech.
Preamble of Terms
In order to keep the conversation straight and the analysis easy to understand,
Ill define a few terms to help us move forward.
Server response: This is the time it takes the server to return the WordPress
html page. This is a measure of the technology on the server and its capability
to generate pages.
On-page performance: This concerns how many assets are on the page.
Assets can be images, javascripts, sliders, css, etc. The more assets you have,
the slower the page will load.
On-page performance can affect a few different things. Most notably, it causes
the browser to make more requests to the web server. It also causes more
thinking time on the browser, so the slower the browser/tablet/phone, the
slower the page will appear.
Server tech level: This includes things like your php version, caching design,
php handler, CPU, and memory. Basically, this concerns the hardware and
software running your site.
The plugin will also graph the results over time so that you can observe the
consistency of your WordPress website performance.
Ever notice that your site seems to perform fine when you check but other
people later comment that its slow? As though gremlins are secretly getting in
to make it slower?
Its often difficult to know how your WordPress website is performing and
how real end users are experiencing the performance of your WordPress
website.
This plugin will display graphs of the performance of the server (so youll be
able to see what those pesky gremlins are doing). It also tracks the
performance of the webpage load speed, which includes all the assets: jpg,
png,css,js, etc.
Youll be able to view graphs for different periods of time, to see how your
server has been performing, and to see how real-world users are observing the
performance of your WordPress site.
Youll also be able to observe when the server is sluggish and when it is fast.
Many of you may be using pingdom, gmetrix, yslow, etc. These tools are good,
but I do have a few issues with them:
- Firstly, they mix up the results from server response time and on-page
performance.
- Secondly, they are an artificial test while you are sitting there looking at the
site, at a single point in time, while you know its going well. You need results
over a period of time in order to observe the site when you havent pre-clicked
and pre-cached the site.
Benchmarking
Its important to know where you are in terms of your websites current
performanceand to have some way of knowing how well your changes are
helping.
My plugin can help with that. It records the real performance of your
WordPress website for 90 days. That way, if you add Plugin X today, you can
compare its performance to last week very easily.
You can also compare your results with others and even compare to other
hosting companieswithout having to actually move your site.
We need to work out how the hosting server you have stacks up against the
competition. We can try all the tricks in the world, but if the server is slow,
with poor technology, we cant really fix it. You cant make a purse out of a
sows ear.
In some instances, you can upgrade your server, or if its a VPS, you can
upgrade some of the software stack. However, its more often than not time to
rethink your hosting needs.
If youre using my plugin, look for a few things.
First, check out the Hosting Check page, where you will want to review the
column headed Server response time. That shows how well the server is
responding to the WordPress page requests.
Ideally, you want to be under one second; thats reasonably easy with modern
hardware and software.
The second thing to look at in my plugin is to see if over the course of a day,
there isnt too much variation in the web server response. Or, when you look at
the tally area (the horizontal bars), not too many of them are orange or red.
If youd like, here is a client of ours: https://artilux.com.au. They have a great
site (and if you struggle with mosquitos, they can help you out). Here are some
of their graph results. How does that look for consistentand under one
second?
Webpagetest.org is also great. Do a test from Australia and you can look at the
overall result, but the main thing youre interested in for this analysis is the
TTFB (time to first byte). Here are my recommended settings.
Here are the results. There are other important numbers, but for now, were
just interested in the TTFB.
There are a whole bunch of factors that make up TTFB, and these will be
addressed later. For now, were just interested in the time being under one
second. Over two seconds is really, really bad. Ive seen poor hosting with as
much as 20 seconds TTFB.
Daily Process Log Report
This is a great report in WHM/Cpanel. Search for daily in the WHM menu.
The report is broken into two parts. The top part ranks the highest CPU users
of your VPS. This is great to know as it gives you some areas to focus on. It
also helps because as you speed up your server, these numbers should drop.
The lower portion of the report is about processesit is a bit more fine-
grained. However, from here, you can see individual high CPU usages. This
will sometimes show slow long-running processes. It might show up that
youre using CGI instead of fpm (more on that later). Anything over 80% is a
target for review.
Disk Performance
This one is a little tricky. Most users should skip this section, but heavy VPS
users should really focus on this. Trust meits important to you.
Ive written a blog article on it here:
https://www.wpdone.com.au/tool-to-test-vps-disk-performance-are-you-
getting-what-you-paid-for/
Disk performance is a good overall indicator of performance and over-selling
as well as the level of tech.
Even though a good hosting config should never even touch the disk, its a clear
indicator of the strength of your server.
It is harder to test disk performance if youre on shared web hosting, so
perhaps skip this one if you are.
You should only be using hosting with SSD these days.
Database Performance
WordPress Plugins
As part of the checkup, you need to review your plugin use.
Up to around 20 plugins is reasonable; once you get past 50 plugins, youre
dragging your own performance down. Ive seen sites with more than 100
plugins activated, and its a mess!
Sometimes youve paid someone to build your site for you, and it came with a
bunch of plugins, which means you cant do much about it.
P3-profiler
The P3-profiler plugin is great. It can give you a report on how much each
plugin is dragging your site down as well as the total of how much the plugins
are eating up in CPU usage.
https://wordpress.org/plugins/p3-profiler/
Server Technology
This is more of a list of what is or isnt on your server and how the software
stack on the server is arranged.
If you use my plugin, its incredibly easy. Simply click on the sysinfo tab.
The output is something like this (Yes, its ugly; I need to get a designer to tidy
up the pages.):
On-page Performance
This is related to how many assets you have on the page and how they are
managed.
Assets can be images, javascript, css, sliders, and other fancy gadgets. The
more you have, the slower everything is.
There are a few knock-on effects of on-page gadgetry:
- The browser has to request more objects from the web server.
- The browser has to spend more bandwidth and time downloading objects.
- All the requests need to be queued through only four connections (assuming
http/1.1).
- This can be exacerbated by the distance from the server (see the discussion
of country later).
- The browser then has to think about and render more images, formatting,
fonts, etc.
The best way to review this is my plugin or by using yslow, pagespeed, and
webpagetest.org reports:
Pagespeed testing can be done here:
https://developers.google.com/speed/pagespeed/insights/
Some people focus heavily on these reports, but its only one part of the
overall picture.
The results from the test are a tad artificial and inaccurate.
The real answer is above the fold page completion, meaning that the first
screen the user sees completes quickly. A page being complete in under five to
seven seconds is the goal.
Measuring above the fold completion is what webpagetest.org measures. I
think webpagetest.org is even more accurate than my plugin. They record
videos of pages loading and stop the clock once the images stop changing. The
only inaccuracy they have is if you have a moving slider or something similar
that skews the results.
Here is my client (https://artilux.com.au) again and their on-page
performance. They have some orange/red bars. Youre going to have a few
slow pages, slow users with slow equipment or poor Internet, or users from
overseas. However, the graphs show that a majority of users are happy.
Chapter 3. Fixing Macro Performance Issues
The aim of this chapter is to consider the issues that are outside your current
solution. Theres no use getting under the hood of your FJ Holden if youre
trying to win a drag raceits probably cheaper and easier to just buy a new
car.
Why is this important? Back when I was talking about the Internet and
PCs/tablets getting faster, I should have mentioned that the distance between
continents isnt getting any smaller. Data takes time to traverse distance. The
more distance, the longer it takes.
If youve got your WordPress site hosted by a bunch of Americans, then its
takes about 250ms to get to the U.S. and back (a quarter of a second). Given
that were trying to get under one second, thats a big deal. However, it gets
worseyou could pay this cross-continent charge more than once!
As everything in the cloud speeds up, particularly software, the percentage of
the time were waiting for cross continent hosting is becoming a larger
percentage. Back when a reasonable time was three seconds for a page, adding
0.25 seconds (even four to five times) wasnt too big of a deal. But at todays
speeds, 0.25 seconds x4 is a whole secondwhich is how fast we want the
WordPress page delivered.
The next cross-continent issue is DNS. If your DNS is also hosted in the U.S.,
thats another one or two round trips that your customers have to take before
they even reach your site.
Now, this gets even worse if you have a .com site and your customers are in
Australia. Even if you have your hosting in Sydney, you still have to pay the
0.25-second cross-continent tax anyway. This is because the first part of the
DNS lookup is to ask where do I look up the DNS, which for .com is in the
U.S.
There are also issues with HTTPS.
You might think, My site doesnt sell stuff, so I dont need HTTPS. Google
already has a weak signal for https for SEO. More is to come; eventually all
sites will be HTTPS or not on Google.
Firstly, there is some certificate-checking that goes on. This can be solved with
OCS stapling. This is far too technical for this book, but you can see my blog
post on this subject:
https://www.wpdone.com.au/the-hidden-https-slowdown-no-one-knows-
about-adding-1-second-to-wordpress-page-load-times-and-how-to-fix-it/
Secondly, HTTPS almost always requires an extra trip to the web server
before the page request. Again, the details are too technical, but you can see it
in webpagetest.org results.
Here is https://apple.com from Sydney. Granted, this is a worst-case scenario,
with a 301 redirect to the www page, but reallywho types www anymore?
- The first line wastes 1.1 seconds just to second the redirect.
- Line 2 is the certificate check.
- Line 3 is the attempt to get the real page, which still has lots of colors
showing more delays. A total of 1.6 seconds has passed before the page even
attempts to start downloading.
Here is a zoomed-in shot of the first line. You can see time being wasted in
DNS lookup, initial connect, and then a double whammy for the SSL (https)
chat. Thats 4x the intercontinental Internet tax. Overall, the 1.6 seconds is
roughly 7x the wrong continent time tax.
My plugin can also help you with country analysis. You can see your overall
performance or just the performance of Aussie visitors.
Now, Ill address the elephant in the roomCDNs. Many people know about
CDNs, and CDNs promise to make the country problem go away. However,
thats not completely true. A CDN really only helps in everything besides the
first few lines I showed above. CDNs dont help there and sometimes actually
make it worse. A CDN can help the https issue, and cloudflare also does
sometimes. However, its a mixed bagsometimes it helps, and sometimes it
doesnt.
There is occasionally a double whammy when using a CDN (like cloudflare)
and HTTPS. Given that the CDN doesnt cache the main WordPress html page,
it cant speed it up. The CDN must request the page from the origin server
(your real WordPress server). If the CDN also uses https to fetch the page from
WordPress, then the CDN will feel the tax of https connections and certificate
checks, etc. This will show up as an increase in TTFB (time to first byte).
Therefore, youll have https slowdown for both the browser CDN and CDN
origin server, which is the double whammy.
One test you can do is to ping your website. Roughly, under 50ms means you
are in the same continent (I think Perth is roughly 50ms from Sydney). The USA
is about 200250ms away.
Having your website hosted in Australia is a big deal. Fix it now!
Oversubscribed
3. Managed WordPress
These are specialist WordPress hosting providers. This is where youll get
better skills and gain focus on keeping WordPress humming along smoothly.
We have a bigger investment in skills and tools that are focused solely on
WordPress hosting.
Here are the situations where you need to consider changing your hosting
provider:
- If your server response is not in the green zone (under one second)
- If there is too much variation in the server response. You cant fix this with
plugins, meaning that you are likely in an oversold situation.
- If your support is poorhow can plugins help that?
- If you suspect that you might be on the wrong style of plan
- Youve reconsidered and think you have your hosting based in the wrong
country
Server Speed
You should be aiming for under one second TTFB (or server response time,
which is roughly the same measure by a different name). This does not mean
loading the whole page, just the initial php/html page.
Hosting Strategy
One of the bigger impactful factors on performance is the hosting strategy that
you are employing: shared vs. VPS vs. managed WordPress.
Shared web hosting is cheap and slow; expect that and live with it. It will be
difficult to get your pages reliably fast enough throughout the day. It can be
done, but major issues like php version often cannot be corrected under shared
hosting.
PHP Version
One of the biggest performance impacts is the version of php you are using.
You should already know your php version from the analysis steps.
Upgrading to php 5.6 or php 7.0 (or HHVM) will result in a good performance
boost.
VPS
Php 5.3/5.4 php 5.5 is about two times faster, php 5.5 5.6 is about three
times faster, and 5.6 0> 7.0 is about four times faster.
WHM / cpanel can easily get you to php 5.6. Have a look in MultiPHP
Manager. You can change one site at a time or all the sites at once. Php 5.6 is
a very safe change as everything works with it.
WHM / cpanel can use php7 under easy Apache 4. However, until WHM 60 is
released (any day now), Id suggest you stay at php 5.6. There will be more on
that later in php handlers.
Php 7 is reasonably safe for WordPress now, but its important that your core
WordPress is updated, as well as all the plugins. Some sites might still break
under php7so be careful.
HHVM was super fast and exciting, but its being knocked off its high horse by
php 7. (They deliver roughly the same performance, but php7 is more
mainstream.) I dont think people are deploying it any more, so I wont spend
much time covering that.
Shared Web Hosting
Sometimes you can log a support request and theyll upgrade you but usually
not. Ive seen people deploy a new website on netregistry, and they had php
5.3 (which is a few years old, has poor security, and is super slow).
PHP Handler
This is something not many people know about, and it is fairly complex. A php
handler works with the web server and controls how php is executed.
VPS
Older versions of php5.5 and belowwere designed to run a php program
and exit. Php 5.6 and php 7.0 are designed to run a php program and stay
running. Both these versions optimize the code the longer they run, so its a
huge advantage on php 5.6 and even more on php 7 to have the php processes
run for a long time.
Back in the good ol days, we had Apache and basic software, and simpler
security and hosting were much easier.
Then we moved to multiple users on the same serverjust a single web server
but we wanted to separate out users to keep them safe from each other.
Anyway, to cut a long story short, in order to run php (and in turn WordPress),
we had to run separate php tasks for each user.
SuPHP was the default for whm/cpanel for a long time. It is secure, but it starts
a new php process for each page request and then shuts it down after. As
youve already learned, new php versions need to stay running. So while suphp
is secure, its very slow. Plus, if a whole bunch of WordPress pages are
requested at once, more php tasks start, they consume too much memory, and
the server crashesbut Ill get into that more later.
CGI is very similar to suphpso well consider them to be the same.
The good ol days php handler is mod_php. This keeps php running for long
periods of time, so its fast. However, if you have more than one WordPress
site, it lacks the appropriate security, so this isnt really an option.
The super new and fast php handler is php-fpm. And there is php 5.6 fpm and
php 7.0 fpm, which isnt really supported by whm/cpanel. (WHM v60 is due
out any day, which will change things.) So this is mostly managed by hand if
you want the ultimate performance php handler. Isnt owning a VPS just grand?
Youll need EasyApache 4 under whm to get php7.0, but you have to run it
under CGI, which is why I am recommending php 5.6 under EasyApache 3.
EasyApache 3 has a nice php handler fcgid. This keeps php running and
manages the number of running processes. I recommend the number of
processes for fcgid to be between two and the number of cores in your VPS
server.
So under whm 58 and below, EasyApache 3, fcgid, and php 5.6 are the best
supported and fastest you can get. Staying supported by the cpanel support
team should be a priority for you for either managed or self-managed VPS.
Whm 60, when its released, will likely be the best combination with
EasyApache 4, php7, and php-fpm. Its still a little experimental, the multi-php
screens are a bit too verbose, and there is a need to do mouse clicks for each
domain hosted. Its a bit of a pain to currently manage.
Shared Web Hosting
On shared hosting, there is no way to adjust the php handler; support calls will
never help you.
Nginx
Ive seen a lot of misplaced advice about upgrading from Apache to nginx.
At the base level, the web server (Apache or nginx) is doing two basic tasks
for your WordPress site:
- Serve static assets like images, javascript, and css files
- Call the php handler to deliver the dynamic WordPress pages
For serving static assets, there isnt much thinking required by the web server,
and both Apache and nginx perform the same.
For calling php, nginx has fewer options. Given that nginx is newer, it doesnt
have as many older and lower performing options. Nginx almost always uses
php-fpm, which is an excellent option, as mentioned in the discussion above.
If you compare both Apache and nginx using php-fpm, they have the same
performance.
Ive seen horrible setups where VPS owners have requested an nginx upgrade
from their managed VPS support, but what is delivered is a software stack like
this:
nginx Apache static files and php
In this setup, nginx is doing nothing and not contributing to any increase in
speed.
In my opinion, nginx and Apache have the same performance if you use php-
fpm on Apache.
Caching
Some caching strategies and tools work better with some profiles than they do
with others, and some caching tools are not available if youre on shared
hosting.
memcached version
- memcached (or redis) application on Linux
yum install memcached
- install php extensions in WHM Software -> Module Installers -> PHP Pecl,
then add memcached and memcache for each version of php that you are using
(Youll need whm 60 for this.)
- restart Apache (from WHM)
- configure memcached :
vi /etc/sysconfig/memcached
PORT="11211"
USER="memcached"
MAXCONN="256"
CACHESIZE="512"
OPTIONS=""
systemctl restart memcached
- I use this for WHM to monitor the memcached service.
cat /etc/chkserv.d/memcached
service[memcached]=x,x,x,/bin/systemctl restart
memcached.service,memcached,memcached
restart chksrvd
/scripts/restartsrv chkservd
You should now be able to see the memcached service in WHM in services
status
- install a WordPress plugin that uses memcached; there are a few. Remember,
I warned against w3 total cache.
https://wordpress.org/plugins/memcached/
https://wordpress.org/plugins/wp-ffpc/
- install php extensions in WHM Software -> Module Installers -> PHP Pecl,
and then add redis for each version of php that you are using. (Youll need
whm 60 for this.) Im pretty sure that redis php is only available for php7.0
and php7.1.
- restart Apache (from WHM)
- configure redis :
vi /etc/redis.conf
restart chksrvd
/scripts/restartsrv chkservd
You should now be able to see the redis service in WHM in services status.
- Install a WordPress plugin that uses redis; there are a few. Remember, I
warned against w3 total cache.
https://wordpress.org/plugins/pj-page-cache-red/
This one is an object cache, which is a slightly different animal. Well talk
more about them later: https://wordpress.org/plugins/redis-cache/
Front-end Caching
There are a few caching solutions that cache before the request gets to the web
server: WordPress and php.
Varnish is common, and so is nginx inbuilt page caching. They essentially
deliver the same performance. Dont listen to people saying nginx is faster
(read more on that in my myths section).
Nginx caches to either disk or memcached. Varnish caches by default to
memory.
Both of these solutions are extremely fast, but they are both extremely hard to
set up and manage.
The upside is, you dont have to install it into each and every WordPress
website.
Below, I talk about Varnish, but the same thing applies to nginx page caching.
Here is the problem with this solution:
1. Its awesome at caching static objects, like images/js/css, but Apache is
already so fast at delivering static objects that you dont get a lot of gains. And
werent you supposed to be using a CDN for caching static assets anyway?
2. By default, Varnish will not want to cache WordPress dynamic pages, as
they are marked as dynamic. Therefore, youll need some Varnish foo to
override those settings. Then, there are pages that you dont want to cache in
Varnish (e.g., the admin console, woocommerce shopping carts, etc.), so youll
need more Varnish knowledge to exclude those from being excluded. At that
point, when a page gets updated on your site, you need to clear the cache from
Varnishyes, perhaps a WordPress plugin and more Varnish foo. As you can
see, there is a lot of Varnish knowledge required.
3. No HTTPS support is available on the free version, although it might be
coming at some stage.
4. No HTTP/2 support is available. (It needs https, and both are experimental
in the recent release.) Even if they do release it, certificates are difficult to
manage.
CDN
All your WordPress pages and comments get stored in a database called
MySQL.
You should be using mySQL version 5.6 or newer. Mariadb is basically the
same software and will eventually replace mySQL. The only difference is
ownership battles mySQL was purchased by Oracle, and in order to get
away from Oracle, mariadb was started. Mariadb version 10 or 10.1 are both
current.
The speed at which your database can respond will greatly impact how fast
your WordPress pages can respond and it can easily become a bottleneck.
Innodb
By default, mySQL and cpanel/whm use myisam for storage. Myisam is slow
and lazy, so you should upgrade to innodb.
Not only does innodb run more quickly but it doesnt crash as often, and it
recovers nicely if you do crash the server. (You dont have to repair databases
and tables after a crash; the repair is automatic.)
This will list the tables you need to review (type mySQL at the command
prompt and paste this, or put it into php_myadmin):
SELECT table_schema,table_name,engine \
FROM information_schema.tables \
WHERE 1=1 \
AND engine = 'MyISAM' \
AND table_schema NOT IN ('information_schema', 'mysql',
'performance_schema');
#!/bin/bash
MYSQLCMD=mysql
Here is my script that will upgrade all the database tables on your server. One
of the lines does a backup of each table it converts in case something gets
broken, but obviously, you should also do a proper backup.
done
done
5. Ive found that a WordFence daily scan can really work mySQL hard
sometimes. At other times, there is some site bashing mySQL, so youll have to
find out which user account and what site. Here is what I use (again, command
line or phpmyadmin). If it is WordFence, disable the daily scan.
mysql> show processlist;
2. If you are a VPS owner, its up to you to stop the DDoS attacks. There are a
few tools that can help you do this.
Fail2ban is popular. It reads the log files, looks for repeated attacks, and bans
those users.
I really like ModSecurity and lfd working together to ban users. WHM can
install and set up. I think lfd is part of CSF, which I highly recommend for
security as it does a great job of blocking the DDoS attacks. Both tools
integrate easily into WHM. It also protects all the sites running from DDoS
attacks. Its a paid option but not overly expensive.
https://configserver.com/cp/csf.html
3. Limiting the number of Web Server threads is also important.
If a DDoS attack is occurring, it can take up 100 or more web server threads.
Each thread needs a php task, which is often 100mb. If you allow too many
web server threads, your RAM becomes exhausted and your web server stops
responding.
Basically, you need to restrict the number of threads per account and have the
excess requests queued. Php-fpm is a great solution for this, which I talked
about earlier.
I wrote in more detail here in this blog post:
https://www.wpdone.com.au/error-502-on-whmcpanel-for-wordpress-
explained-and-solutions/
If you are using php-fpm, have a low number of max children; 0.5x 1.0x of
your real CPU cores is reasonable. If you have one large site, perhaps up to
two times the number of your cores is okay.
Setting a low max children seems like you are slowing or descaling your
server. However, youre really notyoure just controlling the upper limit of
resource usage per account. When the web hits exceed the number of max
children, they get queued. The queue is in both the web server (Apache or
nginx) and the Linux operating system. By default, the queue length is 128 on
Linux, which is fine, unless you have a large number of sites on your server or
youre trying to do load testing. Heres how to raise that limit:
sysctl net.core.somaxconn=1024 (or put it /etc/sysctl.conf for next reboot)
This says to only allow 15 connections from a single IP for both http and https.
Each web user should only be opening four connections, and the requests get
multiplexed (each request from the same webpage gets queued and distributed
over the four connections by the web browser). The only difficulty youll have
with such a low setting is load testing or a large corporation where everyone
surfs the site at once.
5. Use haproxy.
We have some very advanced settings in haproxytoo many to include in this
ebook, but here is a snippet. If the first URL a user requests looks like rubbish,
slow them down for a few seconds. This is generally enough to sour your
server for the attacker. This also relies on keepalive being active on the
webserver.
I wrote more details and some code in my blog post:
https://www.wpdone.com.au/even-more-ddos-defence-against-xmlrpc-and-
login-attempts-for-your-vps/
TTL on Assets
This usually arises from a page speed test saying that some of the assets on
your site need longer expiration times.
The expiration times dont affect the first pages loading speed, when youre
initially trying to impress your new visitor. It does affect subsequent pages or
subsequent visits. The default on most web servers is approximately 10
minutes, which is fine. Just ignore the stupid warning on your page speed test.
If youre still not convinced, I wrote a more comprehensive argument here, and
I included the code to fix the warning:
https://www.wpdone.com.au/why-you-should-not-be-adjusting-expires-and-
cache-control-headers-for-pagespeed-insights-on-your-wordpress-website/
Maintenance
There are a few maintenance tasks you should schedule to maintain top
performance.
First, there are a few tasks mentioned above for mySQL. I even schedule the
convert to innodb daily (see an earlier chapter for that) to catch any new
WordPress installs.
Deleting Transients
These are a bit like zombiesbits of data left over, lurking dangerously, and
eventually slowing things down. There is a plugin you can run, which is a lot
of clicking. Or you can clear them from wp-cli. Here is part of my script;
youll need to loop all accounts:
#!/bin/bash
cd /var/cpanel/users
for account in *
do
cd /home/${account}/www
sudo -u ${account} /opt/cpanel/ea-php56/root/usr/bin/php
/usr/local/bin/wp --skip-themes --skip-plugins transient delete-expired
done
Orphaned Data
After a few years of upgrades, moving your install, plugins being added and
removed, custom fields added and removed, etc., junk can get left around (like
in my garage at homeit needs a good tidy).
Something like this plugin can help clean things up:
https://wordpress.org/plugins/wp-cleanfix/
Chapter 5. WooCommerce Optimization
WooCommerce is a very flexible and common shopping solution. However, it
comes with some performance drawbacks.
Wp_options Optimizations
When you add a lot of products and skews/variations, WooCommerce puts a
lot of data in the wp_options table.
Putting an index on the wp_options table really helps large WooCommerce
installs. For more details, see my blog post:
https://www.wpdone.com.au/optimizing-wordpress-and-woocommerce-for-
thousands-of-products/
Page Caching
The other struggle with WooCommerce is page caching. You cant cache all the
pages, as there is shopping cart info on the pages.
One solution to that is using ajax to display the shopping cart. Ajax helps in
moving the shopping cart to a separate request after the initial WordPress page
has loaded. This allows normal page caching to be effective. Here are some
further details:
https://docs.woocommerce.com/document/show-cart-contents-total/
Chapter 6. Scaling WordPress
Sometimes there are only so many optimizations that you can do. At that point,
you need to get your wallet out and buy more resources. Moving to better
hosting might also help.
Eventually, you will hit a wall. You need either to go deeper into reviewing
your software stack (for example, Varnish) or start using more than one server.
Scaling mySQL
If mySQL is becoming a bottleneck, move it to another server that is more
suited to mySQL. Make sure it has SSD storage, a high number of cores, and a
large amount of RAM.
Occasionally, you might also need to scale beyond a single mySQL server, but
thats starting to get beyond the scope of this book.
Scaling Apache
The scale of Apache is rarely an issue these days. However, you might need to
up some kernel parameters for the number of connections and open files.
Haproxy in front of Apache can really help; its awesome at managing all the
connections.
Scaling Php
Php will eventually be your sinkhole of resources.
Once you have vertically scaled as much as you can (youve ordered the
largest server your hosting guys can sell you), you have a few options:
- Change to hosting that already has a cluster
- Start building your own cluster
Again, haproxy is a great tool on which to build your WordPress cluster.
Slowing Down Robots
You need Bing (Microsoft) and Google indexing your sites. However, they can
cause performance issues, especially Bingit can really hammer a site.
If left unchecked, a few sites being scanned by Bing can seriously impact
performance and resources.
Mj12bot is also a concernsome sort of social indexing spider. Mj12bot says
they will slow down from the fix below.
Baidu spider also impacts VPS servers heavily. It is a Chinese search engine
that doesnt respond to the polite request to slow down (shown below). I
generally just block the Baidu spider.
Something like this in robots.txt really helps, which just asks the crawling bot
to crawl one page per five seconds:
User-agent: *
Crawl-delay: 5
Chapter 7. The Ultimate WordPress Hosting
Stack
The Goal
What were really after is something that is super fast, all the time. We also
want something that doesnt increase support. Finally, we want to make
maintenance easier and not harder. If you have more than a few WordPress
instances, changing settings on plugins is a lot more work.
It would be icing on the cake if all the software were open source (as in, we
didnt have to pay a ton).
As you step through our stack, you can see all the principles abovefast, low
support, low maintenance, able to scale to loads of WordPress instances, and
open source. Weve built a stack that is fast and autonomouskeeping up
performance, stopping DDoS attacks, and increasing uptime availability.
A Word on Mod_PageSpeed
I cant say enough good things about mod_pagespeed. Its by Google, and its
whole goal in life is to accelerate above the fold display of a web page.
It does all the simple things, such as image shrinking, but then goes on to shrink
for each screen size and browser type differently.
Then it sticks all the modified images and metadata into memory (in
memcached). It puts a year expiration on all assets, which makes them nicely
cachable by browsers and cloudflare. It can get away with a year expiration as
it adds a checksum of the data to the URLso if the image changes, the URL
will change. No need to clear any cache.
Then it goes off and minifies all the javascripts and css files.
Mod_PageSpeed then decides to inline some images and css where it has
calculated those assets are usually above the fold. How does it know this? It
watches how each web visitor displays the web page and then sends back data
about what images/css were above the fold. Over time, it will know what is
likely to be above the fold.
Once you have mod_pagespeed up and running, its virtually maintenance- and
support-free. You arent adding plugins to WordPress, updating them, or
changing settings. If you have more than a few WordPress instances, changing
settings on plugins is a lot of work.
Haproxy
Haproxy is our second line of defense and does the lions share of our DDoS
defense. We have quite a few DDoS defense rules in haproxy, so we can stop
things like repeated login attacks well before the hits get to
Apache/PHP/WordPress.
Haproxy also does all of our HTTPS termination and certificates.
It is able to spread the hits over not just servers in one data center but across
our entire multi-datacenter cluster.
Haproxy is also the nerve center of our smart failover. Its monitoring the
health and load of all the web servers below it while also managing the load
and hits.
I cant say enough good things about haproxy. It keeps more sites running, hides
small failures, increases performance, and helps security. I love it.
An interesting feature of haproxy is the ability to queue requests. Instead of
allowing a thundering herd of elephants to trample your web server, it can
better manage this onslaught of traffic. Allowing a reasonable number of
concurrent requests to the web server actually makes it faster. Its partially a
mathematics trick. Ever thought about microwaving two meals at once? Say
they each need one minute of microwaving. If you put both in at once (enter the
thundering herd), the microwave will need two minutes. So the average
response time is two minutes. If you queue them, the first takes one minute, and
the second takes two minutes (one minute in the queue, one minute to heat),
with an average of 1.5 minutes/request. You were able to drop the average
response time from two minutes to 1.5 minutes by queuing. Sure, if you have
two microwaves (like two CPU cores), then you can handle two at onceso
you should adjust the queue depth in relation to your CPU cores.
See two references below for more details on how queuing makes web
response faster:
http://blog.haproxy.com/2011/06/28/play_with_maxconn_avoid_server_slowness_or_crash
http://www.slideshare.net/haproxytech/haproxy-web-performance-55536394
We also are able to do things like make separate queues for different URLs. We
allow more requests and a different queue for requests that we know are going
to be fast, like modpagespeed URLs, which will come from memcache. Images
and static content are also fast (and cached my modpagespeed), but slow
WordPress URLs are queued separately and with a smaller concurrent limit.
By marginally slowing some requests that look like hackers, the queue works
much better for real users. Then, if you imagine how smart it is, when a queue
is starting to get longer, haproxy can start to spread the traffic to different
servers or datacenters. Now that is just brilliant!
Haproxy really deserves its own chapter or an entire book.
The Challenges
He was struggling with unreliable software and late nights and lost weekends
fixing things, running low on disk space, and associated outages. He was also
struggling with poor uptime and poor performance. He was already paying a
premium for Australian managed hosting, and while his hosts probably meant
well, they simply werent focused on WordPress . Simple problems took a
long time to fix or were never resolved.
One of the spirals he was in was low disk space, which led to mySQL going
down. Then he had to find old backups to delete, and restart mySQL, and then
repair the mySQL databases. Have you ever had to find out what was eating
your 100gb of VPS disk space? There arent many tools that can show you disk
usage in a web environment. WHM/cpanel will give you a list of who is using
what disk space, but thats at an account/website level. After that, you end up
picking through folders in FTP.
Its a sad state of affairs when youre a business owner who charges high
hourly rates for consulting yet you know how to restart mySQL and repair
mySQL databases to prevent them from being crashed. Youre spending your
time doing basic admin work that you dont enjoy and shouldnt need doing: -
your time is better spent on your next project and customer.Surely your time
could be better spent doing anything else than managing a VPS.
The Upgrade to VPS
Dallas knows that shared hosting is mired with poor performance, poor
support, no backups, and a bunch of challenges. Therefore, some time ago he
upgraded to a VPS, which is where his bigger problems started.
He had to learn about how to run a VPS. He just thought, Its cPanelhow
hard can it be? It will be managedso I dont need to know much.
There was a lot more learning about Linux, whm/cpanel, ftp, security, Linux
software, backups, mySQL, fixing white screens of death on WordPress, and
fixing more error 500s than he ever wanted to know about.
The Messes
Weve had to slowly untangle Dallas messy DNS setups.
When you first get a VPS, youll find that you have one DNS server, but you
really need two or three separate DNS servers. Therefore, he kept his older
VPS to run a DNS.
However, there were also name servers pointing everywhere, and loads of his
Aussie clients were using U.S. name servers, only adding to the poor
performance.
Plus, when something went awry with DNS, it was a pain to pick through and
fix.
He struggled to keep his WordPress site updated, and he had occasional
hacked WordPress sites that needed to be recovered.
Poor Performance
None of his sites had even reasonable performance.
He had php 5.6 running under suPHP, a very common strategy (which, as you
recall, negates the advantages of php 5.6).
He had four cores and 4gb of RAM. Everything was squeezed in and only just
worked on a good day. On a bad day, error 500s popped up everywhere.
The Plugin Dance
Dallas had plugins for caching, plugins for security, and plugins for image
optimizing.
Then, if you wanted to change settings, there were 80 or more /wp-admin/
logins to change settings. Sure, you can use ManageWP or similar programs to
install/update plugins, but they dont fix settings.