After building the configuration files to handle very high traffic volumes for a WordPress blog on a $15 a month Linux virtual server, it seemed obvious that I should try to get the same setup working on my Beaglebone ARM server, which is currently in a colocation site.

The tl;dr of the story is: over 180 hits/sec at it’s peak, using nginx as both a cache and backend web server. The blitz.io result was:
This rush generated 3,069 successful hits in 30.00 seconds and we transferred
25.46 MB of data in and out of your app. The average hit rate of 94/second
translates to about 8,194,776 hits/day.

I started by following the same steps as above, but when it came to Varnish, the system kept falling over with an obscure error message, while the poor little SD card was thrashed to death. Rather than give up, I decided to follow the suggestion of some of the comments on my original blog post, and try caching with nginx.

First, I compiled the latest stable version of nginx (1.2.1), as the official Ubuntu version is slightly out of date. I complied it using the following switches:

./configure --prefix=/etc/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --lock-path=/var/lock/nginx.lock --pid-path=/var/run/nginx.pid --with-http_gzip_static_module --with-http_ssl_module --with-ipv6 --without-http_browser_module --without-http_geo_module --without-http_limit_req_module --without-http_limit_zone_module --without-http_map_module --without-http_memcached_module --without-http_referer_module --without-http_scgi_module --without-http_split_clients_module --without-http_ssi_module --without-http_userid_module --without-http_uwsgi_module


It turns out I also need to install libpcre3-dev to get nginx to compile, so ran:

sudo apt-get install libpcre3-dev

before trying the compile command again.

I installed the W3 Total Cache wordpress plugin as before, along with additional plugin called “WordPress Nginx proxy cache integrator” and then setup nginx with the following reasonably simple files.

The relevant parts of the nginx.conf file comes down to the following lines:

port_in_redirect off;
proxy_cache_path /dev/shm/nginx levels=1:2 keys_zone=czone:16m max_size=32m inactive=10m;
proxy_temp_path /dev/shm/nginx/tmp;
proxy_cache_key "$scheme$host$request_uri";
proxy_cache_valid 200 302 10m;

The default.conf file is slightly more complicated, consisting of the configuration for 2 hosts – one is the nginx caching front-end, and one the PHP executing back-end. I believe you can configure one host to do both, but it seems simpler this way to me.

The “frontend” server is very simple, it just says it’s a proxy cache:

server {
server_name frontend;
location / {
proxy_pass http://backend:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_cache czone;
}
}

And the “backend” server configuration is pretty much exactly the previous WordPress PHP configuration we had in place, here’s the main part:


location ~ \.php$ {
fastcgi_buffers 8 256k;
fastcgi_buffer_size 128k;
fastcgi_intercept_errors on;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass unix:/dev/shm/php-fpm-www.sock;

}

After that, I needed to add a hostname for “backend” and “frontend” to my /etc/hosts file, so it now looks like this:

127.0.0.1 localhost backend frontend

and to disable the WordPress canonical redirection, I editted /var/www/wp-content/themes/twentyeleven/functions.php and added this line near the top:

remove_action('template_redirect', 'redirect_canonical');

I’m sure there’s a nicer way to do this, but I don’t remember it off the top of my head..

So once that was all done, I was able to restart php5-fpm and nginx, and run the good old blitz.io test, and see the results.

The key figures were:

ANALYSIS
This rush generated 3,069 successful hits in 30.00 seconds and we transferred
25.46 MB of data in and out of your app. The average hit rate of 94/second
translates to about 8,194,776 hits/day

So I’m pretty happy with that, I guess I could squeeze more out of it, but the box was busy by the end, and if you’re not happy with 8 million hits/day from a board that costs £50, you’ll never be happy!

Back in November I asked where the cheap ARM servers were, and then I started experimenting with my own £60 ARM server, but now Dell have come out with their own ARM server for web providers – nicknamed “Copper”.

It’s not going to be that cheap, being based on the Dell C-Class Cloud computers with:

So that all sounds pretty nice – 4 cores per node, 4 nodes per sled, 12 sleds in a 3U Chassis, so that’s 192 cores in a 3U slot.

With this kind of specification, the nodes seem perfect for things like load balanced web servers, Cloud Foundry style “Application Execution Nodes”, and other light to moderate disk I/O tasks.

The density of cores is definitely there, as is the memory – 8GB per node is around the right balance, so the real questions remaining seem to be currently unanswered:

If anyone from Dell wants to give me those answers, I’d be fascinated to know…

Update – Thanks to Mark responding in the comments below, that’s 15 Watts per node, which gives you 750W per 3U sled, a very impressive figure.

It’s difficult to overstate the importance of email in some people’s lives – almost everything that can be done by email does is done by email for at least one person out there. Billions of emails are sent every day.

And yet, so many people, including myself, rely on email services where we are not the customer, but the product being sold to advertisers, the real customers of Internet giants like Google, Microsoft and Yahoo (still the largest email provider, despite their other well known issues).

This leads, unsurprisingly, to conflicts. Someone has an email problem, and the supplier, who has millions of users and a relatively small number of support employees, is unable to respond as the user would want.

So… there’s an obvious question – why don’t people value their email accounts enough to pay for it, and how much would they pay? well, if you look into how much a basic paid for mail service costs, it’s obviously almost nothing – £1 a month for 1GB of storage with webmail, IMAP, spam filtering, all that fun. And as they say “UK based 24/7 support – get help when you need it”. So, what more could you want (except maybe more storage..)?

Maybe it’s the interface that holds people back, these alternative webmail interfaces often aren’t great. But there’s Zimbra, a very advanced webmail product which costs a bit more (£39 per year from SMS, I’m sure others will be cheaper).

Now given how important email is, £40 a year doesn’t seem that high, though I do need more storage – that’s £20 a year per GB, and Gmail shows me using under 2GB of storage after several years of use.

So, around £60 per year, for an email service with customer service, is it worth it? I don’t know, but I do know if I keep using Gmail, one day I almost certainly will find out how important my email is to me – it’ll be the day I’m locked out and can’t get in touch with Google to resolve my problem.