20 May 09 Server Upgrade Status: Getting There
Welcome, this is my first post from the new server!
I’m really, really glad I went with renting a 2nd server and migrating to it instead of my original plan of just re-imaging. I would have been down for days while I figured out Ubuntu without Plesk.
I got my new server and pressed the magic button to have it re-imaged to Ubuntu 8.04. It all seemed to work just fine. I spent last Saturday going to town on it, getting the version of Subversion I wanted to use built, getting mod_wsgi built, and other simple “new server” type tasks. But then I noticed that the following on Sunday:
/dev/md6 4.7G 228M 4.5G 5% /var /dev/md7 221G 83M 221G 1% /home
The sizes of /var and /home were swapped compared to the old server. I had a huge /var and a small /home before. I became very worried because on the old server, all the databases, website data, and logs all go under /var. Was I going to run out of room very quickly?
I stewed on this for a day or so, and I even toyed with the idea of reimaging to CentOS 5 and getting Plesk back. After all, my main goal was to get Python 2.5, and I subsequently learned you can run different versions of Python side by side. So in theory I could install Python 2.5 on CentOS if I built from source. Then I could tell mod_wsgi to use Python 2.5, meanwhile the rest of the system would still use Python 2.4, and things like the yum package manager would keep working.
But ultimately I decided I liked Ubuntu better. I liked the way they setup Apache for easy multiple virtual hosts. I liked the aptitude program as a simple GUI for apt-get. And Ubuntu is my development environment on my laptop, so having them both match would be a good thing.
I did ask my host if they could either customize the image to make /var bigger or help me do it. They said no to both. This wasn’t totally surprising or unexpected.
I decided it didn’t matter how big /var was. I’ll just put everything on /home, under a directory of my own making. I configured MySQL to use the new directory on /home, and all my virtual host websites will use it as well. Now only system logs will use /var, and it should be big enough for them.
With that settled I set of to try to make it work. I began moving deathofagremmie.com to the new server. Moving the database and WordPress files was easy enough. I then hacked my way through the DNS configuration, using the old server as a guide. I really wasn’t sure I had done it right, but what the heck, I thought I’d give it a try. I then had my host transfer the domain to the new server. There wasn’t much to do except wait for the DNS to propogate. I did this late at night, so I went to bed. The next day at work I noticed that it had transferred, and the website even came up! A few things didn’t work, but nothing I couldn’t fix at home later. But when I came home from work, DNS was still pointing to the old server. I had to wait until almost 24 hours had elapsed before it finally updated for me at home.
Once DNS was correct, I could more easily fix the problems from the migration. It turns out they were minor. I had copy/pasted some Apache configuration from the default files, and I had the “AllowOverride” option set to “none” on my virtual host. This meant Apache would not look for .htaccess files, and thus the mod_rewrite rules that WordPress uses weren’t working, and the authorization I had setup on the admin login was busted. No problem, I’d actually rather not have Apache look for .htaccess files for efficiency reasons. I just moved those directives out of .htacess and put them into my configuration file for the virtual host. Why didn’t I think of that before? But that wasn’t quite the whole story. Mod_rewrite still didn’t work, but that was only because it wasn’t enabled by default in Ubuntu. The Ubuntu team did a really good job of paring down the Apache configuration, and making it very easy to enable and disable modules. A simple “a2enmod” command and a graceful restart fixed it. The Ubuntu Server Guide was a great help to me while I was doing all this.
And yesterday I received my copy of Pro DNS and Bind by Ron Aitchison. By quickly reading the first few chapters the mystery around those DNS files was suddenly solved. I went back and redid those files with my new knowledge and I feel confident about them now. I highly recommend this book.
And, using my new copy of The Book of Postfix, I was able to get my mail aliases for deathofagremmie.com to work. Postfix was much easier to setup than I thought it would be. The developers did a great job of “just making it work” out of the box with minimal configuration. I don’t have any plans to have any real mailboxes on the server, I just want aliases to forward to my Google mail account. This was easy to setup, and is described in Chapter 14 “A Mail Server For Multiple Domains.” I used the virtual alias technique to accomplish this. Edit a few files, run some commands on them, restart and it all worked.
The last wrinkle of the night was figuring out how to change the hostname under Ubuntu. This was solved by doing some quick Googling. You basically have to edit /etc/hostname, /etc/hosts, then stop and start the hostname service. With that fixed, email from my new server looks correct.
So, what’s left? I’m liking this new configuration, so I think I’ll go with it. Now I have to move the two remaining sites: one is a Django powered site, and one is the current PHP based SurfGuitar101.com. After I get them moved and tested I can shut down the old server, and get back to working on the new Django powered version of SurfGuitar101.com.
06 Apr 09 Infrastructure: Trac & Subversion
I’ve been wanting to get some kind of issue tracker up and running for some time now. Trac seems like a great choice. We’ve used it where I work, and the Django project uses it. I even managed to install it on Windows at work. Still, I was kind of dreading trying to get it working on the dedicated server I rent. I finally gathered the strength and tackled this problem this weekend, and it went far easier than I imagined.
Subversion
First of all, I decided I might as well upgrade my Subversion (SVN) server while I am at it. I see that Subversion 1.6 is out now. However, reading the fine print, I noticed that they seemed to have changed their Python bindings in 1.6, and I wasn’t sure if Trac is compatible with this. So without doing any further research I decided to just run the last stable version before that, 1.5.6.
My dedicated server is running Fedora Core 6, which isn’t maintained anymore, so there is no way to my knowledge of getting a binary package for these recent builds of SVN. I need to build from source. I had done this once before, and I even took detailed notes (which I had forgot about). Building from source is fairly easy, but there is one gotcha on the AMD64 server I run, you need to invoke the configure script with an –enable-shared switch. Luckily I wrote this down from the first time I did this. Getting the required dependencies for the source build isn’t too hard. The Subversion folks helpfully package some of the less readily available dependencies, so it is just a matter of grabbing them and untarring them on top of the unpacked source tarball.
Since I wanted to integrate Trac with my Subversion repository, I needed to ensure I built the Python Subversion bindings. I used Yum, the package manager that comes with Fedora, to make sure I had SWIG installed before I ran configure to build SVN. Then it is a simple matter of building the Python SWIG bindings after Subversion proper is built. This is explained very well in the Subversion documentation.
This seemed to go well, although I had a minor heart attack when Apache crashed the first time I tried to restart it with the new SVN in place. Another restart and it was fine. Hmmm. In short order I had upgraded my existing repository and things seemed to be working fine.
A New Subdomain
I then created a new subdomain to host my issue tracker. I rely on the Plesk control panel to do this lifting for me. It came installed with the server, and I rely on it heavily to configure Apache, the mail server, etc. I’m not a hard core server admin, so this is a big help. Although I can see the day when the training wheels can come off as I become more familiar with Linux and these tools. I can sort of see what Plesk is doing by examining the config files it creates and it doesn’t appear to be rocket science. Still, it is a big time saver for me.
Trac
To get Trac installed requires getting all the dependencies in place first. In most cases, I was was able to use Yum to get most of the dependencies in binary form from the Fedora repository. Despite the fact that Fedora Core 6 is pretty old, the version numbers of the dependencies in the repository were still compatible with the newest version of Trac. The one notable exception was the template engine Trac uses, Genshi. In this case a simple “easy_install Genshi” did the trick. Nice.
I might have been able to easy_install Trac, but the docs say that this only works for Python 2.5 and 2.6. I’m still running 2.4 on the dedicated server. Upgrading my OS is definitely on the long term to-do list, but I must take baby-steps for now. But it was a simple matter of grabbing the Trac tarball, untarring it, and doing the usual “python setup.py install”. It went flawlessly.
Now luckily I had setup Trac at work before, so I already knew what to do. I ran the command-line Trac admin tool to create a project and tied it to my new Subversion repository. Trac comes with a development server, and I ran that after configuring the project. I could then point my browser at my server and see my new Trac project for the first time! Things are cooking at this point.
Mod_WSGI
Of course I can’t use the development server for real work. So the next step was to get Apache to serve my Trac project. I once again chose mod_wsgi as the deployment method, after just recently converting The Madeira site from mod_python to mod_wsgi. The mod_wsgi documentation is excellent, and a wiki page covers integrating Trac and mod_wsgi in great detail. After studying the docs for a short while I had the magic Apache configuration down. I restarted Apache, and once again I was amazed that things were working on the first try. I had been pretty lucky so far. (In fact the most trouble I had that day was trying to change the logo on the Trac site!)
At Last…
I was now ready to configure my Trac project and get my new Subversion repository loaded. I had an existing Subversion repository that I was doing all my work in. However I had checked in some settings files that contained database password information. Shortly after realizing this I just locked the whole repository down. Since then, I have learned the Django settings.py and local_settings.py trick, and have placed the sensitive information in the local_settings.py file (which is not controlled in SVN). Now I can have a public read-only repository again.
So here it is, ready for beta testing: http://code.surfguitar101.com. Now there isn’t anything stopping me; I have to do the real work of deploying a beta version of SG101 2.0 for testing and feedback.
10 Jan 09 Experimenting with mod_wsgi
I’m getting a bit closer to wanting to deploy some of the work I have done on the new site in order to get feedback and find bugs. I’ve already got The Madeira’s site running under mod_python, and it seems like a bit of a pain to get another site running under mod_python. Or, I should say, I don’t fully understand how to do it. 
First a bit of background on the server I am renting. It is running Fedora Core 6 and has Plesk installed. Plesk is a web-based application that manages the server for you. I click around in it to add domains, sub-domains, configure the mail service, etc. Plesk auto-generates the apache configuration files based on your settings.
I’ve been reading good things about an alternate way to deply Python web applications: mod_wsgi. So a couple of days ago I decided to try it out. Here is a quick sketch of what happened.
It was easy enough to grab the source and to do the usual “make” and “make install” to build it. However, I did notice right away that (via Graham Dumpleton’s excellent documentation) that the mod_wsgi.so module was linking against the static version of my Python library instead of the dynamic one. Of course if I am going to have multiple processes running a Python interpreter, I am going to want them to share the code to avoid bloat. I did some poking around on my server and found the correct Python shared library, and I put a symlink to it inside my python/config directory. Completely rebuilding mod_wsgi at that point seemed to resolve that issue.
I then configured a subdirectory on the virtual host running this site to be served by the sample wsgi application. This worked straight away when mod_wsgi was configured to run in embedded mode. However the real goal is to run in daemon mode, in which separate processes are launched to run the application, and apache defers to them. You can also run these processes as a different user than apache for added security.
When I configured mod_wsgi for daemon mode I ran into problems. Fortunately I received excellent help from Graham Dumpleton on the mod_wsgi user’s list. First of all I was still running mod_python, and this was probably still using the static Python library. I temporarily disabled it and Graham had me change the apache log level and examine the logs. For whatever reason, the WSGI daemon processes were not inheriting the apache user and group, instead they were using -1/-1 for uid/gid. When I added configuration settings to explicitly set the user and group for these WSGI processes to run under, everything was fine (if mod_python was out of the picture that is).
After a bit of closer examination, it turns out that in my main http.conf file I have this:
Include conf.d/*.conf User apache Group apache
So for whatever reason, user and group ID’s were not set when my conf.d/mod_wsgi.conf file was being processed.
So now I am poised to cut The Madeira site over to mod_wsgi and retire mod_python. I can then more easily create a new process to run a test site for SG101 2.0.
While I have no real runtime experience with mod_wsgi yet, I am still very impressed with the project and Graham Dumpleton’s extensive documentation and knowledge of apache.
