Supersonic Feet

I’ve spent a good portion of my weekend banging my head against the desk after getting the hare-brained idea, following my success in setting up Apache 1.3.x with mod_proxy and Mongrel, to get Apache 2.2 running on my server so that I could then use mod_proxy_balancer and therefore mongrel_cluster.

As anyone who’s ever asked the question “How easy is it to install Apache 2 on a cPanel server?” knows, the answer has always been “don’t bother”. In the past couple of weeks however, CPanel finally added support for Apache 2 in their latest EDGE builds. After a quick bit of reading of the cPanel forums, it seemed people had it working well enough and so I embarked, without much forethought, on the upgrade.

A day or so and many, many hours of frustration later and I have it working, so I thought I’d round up my experience as before to try and save some poor souls the same experience.

First things first

My server is running on CentOS 4, so that’s what this how-to will be based on. Here’s a brief run-through of what needs to be done along the way:

1. Upgrade cPanel to EDGE build
2. Re-compile Apache to 2.2 with various extras
3. Add various proxy modules to Apache which easyapache doesn’t do for you yet
4. Install Mongrel and mongrel_cluster
5. Get our Mongrel cluster running
6. Keep our Mongrel cluster running
7. Configure Apache to point at the cluster of Mongrel servers for dynamic content
8. Either rejoice or cry

IMPORTANT NOTE

The whole of this how-to is based on upgrading to an EDGE build of cPanel. As the name suggests, this is a ‘bleeding-edge’ version that isn’t fully tested and guaranteed stable. As such, proceed at your own risk. I was able to do this as I still only have one website live on my server so it was no great deal if things went wrong.

It’s running fine for me, but I can’t take responsibility if any of this destroys your server. In theory, if it does go wrong you should be able to change your cPanel update Config back to a more stable version and get back to where you were, but I haven’t tried it and again can’t give any guarantees.

1. Upgrade cPanel to EDGE

Log into Web Host Manager (WHM) and go to ‘Update Config’ under Server Configuration. Tick the option labeled ‘Manual Updates Only (bleeding EDGE tree)’.

Scroll down the left-hand menu to the cPanel section and click on ‘Upgrade to Latest Version’. When the page loads, click the button to do the upgrade (it’ll take quite a while).

2. Install Apache 2.2

NOTE: If you already had mod_proxy going on your Apache 1.3 (as I did), you’ll need to comment out/delete your ProxyPass and ProxyPassReverse lines in /etc/httpd/conf/httpd.conf as we won’t have mod_proxy instantly available and Apache won’t update to 2.2 if it finds the commands it doesn’t understand.

Once cPanel has been updated, you can then get Apache 2.2 installed. I used easyapache via ssh as the WHM web-based version didn’t update straight away, though in hind-sight it might work if you just log out of and then back into WHM. Either way, the options are the same.

To try the WHM version first, click on ‘Apache Update’ under Software. If it offers you different version of Apache, you can continue this way. If it doesn’t, log into your server via SSH and make sure you have root/su access.

The first two steps seem to be in opposite order depending on whether you’re doing it via the browser or ssh, so just read below and do them in the order you’re offered:

• You need to choose Apache 2.2.x (latest was 2.2.3 when I did it) as your Apache version.
• Of the 9 ‘skill levels’ it offers, choose 7 to get the expert options and also reload your existing Config at the same time (will save you time).

Next up is PHP version. I chose 5.1.6 but this one’s up to you – choose the one you need. I notice it now has the helpful option of running both PHP 4.x and 5.x alongside each other though I don’t know which one it’ll force to use a .php{x} extension.

Now for the Apache and PHP Config. For PHP, again choose whatever you want. Of the Apache options, this how-to requires Deflate and Headers, so tick/check the ‘Enable Deflate’ and ‘Enable Headers’ options. Choose any other options you need and ‘save and build’. As with the cPanel update, it’ll probably take a while.

3. We need more modules!

Unfortunately, cPanel’s easyapache isn’t so easy that it offers everything we need, even some of the most commonly-required modules for people upgrading to Apache 2. Never mind, it’s relatively easy to solve (or it was once I spent a few hours searching the ‘net due to my lack of even semi-hardcore server admin knowledge).

It turns out we can add modules in without needing to recompile Apache with a little tool called apxs. We’ll be adding:

• mod_proxy
• mod_proxy_balancer
• mod_proxy_html
• mod_proxy_http

With Apache 1.3.x and cPanel, it used to leave the source files handily around for you to get at, but I couldn’t find them with the 2.2 build, so we need to download the matching source tarball for our version of Apache. You should be able to visit the Apache downloads page and get the source file to match (Server Status > Apache Status will tell you the version you’re using). In ssh, download the tar.gz into /usr/local/src and extract it:

# cd /usr/local/src
# wget http://apache.mirror.positive-internet.com/httpd/httpd-2.2.3.tar.gz# tar –xzvf httpd-2.2.3.tar.gz

You can now get to the folder containing the module files for installing:

# cd httpd-2.2.3/modules/proxy

Before we install them, we also need to make sure we have libxml2 installed. You should be able to do this via the rpm installer in WHM (Software > Install a RPM). Install libxml2.i386 and libxml2-devel.i386.

The mod_proxy_html module itself is unfortunately not a standard inclusion with Apache, so you’ll need to download it from http://apache.webthing.com/mod_proxy_html/ (see under Availability at the very bottom for the .c source code). In the modules/proxy folder still, run wget:

# wget http://apache.webthing.com/mod_proxy_html/mod_proxy_html.c

Now to add the modules (make sure you’re in the /usr/local/src/httpd-2.2.3/modules/proxy folder):

# /etc/httpd/bin/apxs –i -a –c mod_proxy.c
# /etc/httpd/bin/apxs –i -a –c mod_proxy_balancer.c
# /etc/httpd/bin/apxs –i -a –c mod_proxy_http.c

These should all go through fine. They install the modules and add the LoadModule line to httpd.conf for us. Mod_proxy_html needs a slightly different command as we have to tell it where to find libxml2. It should be in /usr/include/libxml2 but you can check by typing ‘whereis libxml2’. Once you’re sure, type:

# /etc/https/bin/apxs –c –I/usr/include/libxml2 –i mod_proxy_html.c

Finally, let’s check that Apache knows to load these four new modules. Open /etc/httpd/conf/httpd.conf in a text editor (I use pico for n00b simplicity – ‘pico /etc/httpd/conf/httpd.conf’) and search for LoadModule (ctrl+w in pico) to get to the bit we want. You should see a bunch of lines that start with ‘LoadModule’. The last few should include these:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_html_module modules/mod_proxy_html.so
LoadModule proxy_http_module modules/mod_proxy_http.so

If any are missing (proxy_html might be), add them in and save out (ctrl+x, y, enter in pico).

4. Mongrel

After the not-so-fun Apache setup (well done if you’re this far without any problems!), Mongrel is a bit easier. I already covered installing Mongrel installing for Apache 1.3.x but this adds a couple of extra gems, so if you followed my first how-to, run this gem install again anyway – it’ll just update/ignore stuff you’ve already got.

The whole benefit of Apache 2.2 (at least as far as this article is concerned) is the ability to load balance between multiple Mongrel servers. Mongrel itself has a nice extra called mongrel_cluster that’ll look after starting/stopping your cluster of Mongrels for you. Let’s install all the gems we need in one fell swoop:

# gem install daemons gem_plugin mongrel mongrel_cluster –include-dependencies

You’ll be asked which version of Mongrel to install. I tried the latest (0.3.13.3) first but it caused problems so go for 0.3.13.2 unless a new fixed version is out by the time you’re trying this.

The Mongrel page on setting up a mongrel_cluster suggests creating a mongrel user for running the mongrel servers, but this requires then chowning your entire Rails app to the new mongrel user which in turn causes trouble for you uploading files at a later date. cPanel assigns each site a username anyway and we can tell mongrel which user to run as, so I stuck with using the cPanel-assigned user (see below) for convenience. The choice is yours – see the Mongrel docs for help creating the mongrel user.

5. Setting up mongrel_cluster

Configuring your Mongrel cluster is easy – there’s a cluster configuration command that you add your normal options to, as well as the number of servers to run in the cluster and it’ll create a config file for you. Here’s an example:

# cd /home/myuser/myapp
# sudo mongrel_rails cluster::configure -e development \
-p 8000 -N 3 -c /home/myuser/myapp -a 127.0.0.1 \
-- user myuser --group myuser

The above creates a file in your app’s config folder called mongrel_cluster.yml to store the info for use maintaining the cluster. My example above tells the servers to start in development mode (swap for production once your app’s done).

The -N option sets the number of servers to run in the cluster, on ports start with the value of the -p option. In this case, three Mongrels will be run on ports 8000, 8001 and 8002.

-c sets the application root while -a 127.0.0.1 tells the servers only to listen to local requests (the Apache server) rather than outside requests direct to the ports. You might want to leave this option out initially so you can test each server is running correctly before we get Apache set up.

Finally, we set the user and group that the servers should run as. As I noted above, I used my website username for simplicity.

Now we can get the cluster running:

# sudo mongrel_rails cluster::start

You should get a message saying it’s starting 3 servers and you can then (if you didn’t add the -a option) load each server in your browser with http://www.mywebsite.com:8000 etc.

To stop the cluster:

# sudo mongrel_rails cluster::stop

6. Keeping our cluster running

As it stands, as soon as our server reboots, we lose our Mongrel cluster, so it needs to be added to load on bootup. mongrel_cluster has an init script ready-made, we just need to set it up:

# cd /etc/init.d
# sudo ln -s /usr/local/lib/ruby/gems/1.8/gems/mongrel_cluster-0.2.0/resources/mongrel_cluster mongrel_cluster
# sudo chmod +x mongrel_cluster
# sudo /sbin/chkconfig --level 345 mongrel_cluster on

That last line works for RedHat-type systems. There are examples for others in the reference links at the end of this article on the Mongrel site and Coda Hale’s blog.

Lastly, we need to link the mongrel_cluster.yml config file from our app:

# sudo mkdir -p /etc/mongrel_cluster
# sudo ln -s /home/myuser/myapp/Config/mongrel_cluster.yml /etc/mongrel_cluster/myapp.yml

You can add config files for all your apps in this way - just make sure each is given a different name and they’ll all be run on bootup.

So now Mongrel’s all set up and running as a nifty cluster, we can tell Apache to take note.

7. Pointing Apache at our cluster

With all we’ve done so far, Apache still doesn’t know about either our Rails app or the Mongrel cluster it’s running on. I set my up on a sub-domain so I’ll use such a setup as an example, but using your site root is pretty much the same.

As I’m using a sub-domain, it needed creating first. cPanel creates sub-domains pointing at similarly named sub-folders of public_html, so first I made a symbolic link to my app’s public folder where the sub-domain folder would normally be created (we still want Apache to serve the static stuff). Let’s say we’re gonna have the app at http://myapp.mysite.com

# cd /home/mysite/public_html
# ln -s ../myapp/public/ myapp

The sub-domain itself can be created via the user cPanel - just log in, go to the sub-domains section and create your myapp.mysite.com sub-domain. It’ll use the sym-link rather than create a folder which is handy. If you’re using your site root (www.mysite.com) rename public_html to public_html_bak and create the sym-link to public_html instead:

# cd /home/mysite
# mv public_html public_html_bak
# ln -s myapp/public/ public_html

Time to edit httpd.conf again…

# pico /etc/httpd/conf/httpd.conf

Search for your domain/sub-domain and you should get to a section enclosed in VirtualHost tags. Below the line that starts ScriptAlias /cgi-bin/ add:

RewriteEngine On

# Rewrite index to check for static
RewriteRule ^/$ /index.html [QSA]

# Rewrite all non-static requests to cluster
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI} [P,QSA,L]

# DeflateAddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/xml application/xhtml+xml text/javascript
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4.0[678] no-gzip
BrowserMatch bMSIE !no-gzip !gzip-only-text/html

Almost done! Scroll down a little further until you see the closing </VirtualHost> tag and below it, add:

<Proxy balancer://mongrel_cluster>
BalancerMember http://127.0.0.1:8000BalancerMember http://127.0.0.1:8001BalancerMember http://127.0.0.1:8002</Proxy>

OK, so what did all that do? The bits added inside the VirtualHost set some rewrite conditions to check whether the requested file is an existing static file or not. If not, it passes the request on to a proxy balancer called mongrel_cluster. This is what we define later in the Proxy tags.

The balancer can be called anything (for when you need to do this with more than one app) and lists its ‘member’ servers. Here we’ve listed our three Mongrel servers. Rails requests will now be balanced evenly across all three.

Time to save and exit (in pico: ctrl+x, y, enter twice). Before we take the plunge and restart (potentially breaking) Apache, we can check the config first:

# /etc/httpd/bin/apachectl configtest

If all is well, it’ll say ‘Syntax OK’. If not, I’m afraid you’ve got some spell-checking/debugging to do though it should give some hints on where the problem is.

Assuming you got the OK, it’s time to restart Apache and make sure it all works.

# service httpd restart

Now load your app in the browser, e.g. http://myapp.mysite.com...

8. All done?

Everything works and you issue a guttural “’Ave it!”. Or, everything doesn’t work and you punch that nice, fragile TFT in front of you, causing more anger at the incurred cost.

Let’s hope for the former, eh?

With thanks to…

This article is based hugely off the good work of others, so time to give credit where due. The following articles/sites were instrumental in me eventually getting sorted and then writing this how-to for cPanel users:

• Time For A Grown-Up Server: Rails, Mongrel, Apache, Capistrano and You, by Coda Hale. I missed out the Capistrano bit here as I don’t use it yet, but if you do then clicky linky.
• Mongrel Documentation: Apache and mongrel_cluster
• Apache Week. Running a Reverse Proxy with Apache. Helped me sort out mod_proxy_html.
• http://www.sei-info.co.jp/IPAHP/HowToInstall.html - includes help compiling libxml2 from source if the rpms don’t work for you.
• mod_proxy_html
• libxml

There were a couple of others that I unfortunately haven’t kept track of, so apologies to them.

Round-up

Hopefully the above helps someone. I know it drove me insane for a while there and it’s often hard to find everything you need in one place when it comes to setting stuff up on *nix systems. In theory, the above should be just that for cPanel users (on CentOS at least, hopefully other RedHat-type systems).