13 Jun 09 Installing memcached for use with Python and Django

I installed memcached on my production server a while back. It’s supposed to be thee way to get fast and efficient caching for your Django powered website. I remember the process as being somewhat less than satisfying. Tonight I decided to get it running on my development box, which is running Ubuntu 8.04. So I took a lot of notes and present them here for my own future reference. I hope this may help someone. And as you can see, I have a few questions myself that perhaps someone can help me with. Read on…

First of all, to get memcached itself you have two choices. There is an Ubuntu package for it, so you could use apt-get to install it. But if you want to run the latest and greatest version, you need to build from source. That’s what I wanted. To do this, you visit http://danga.com/memcached. To get the latest version, here is what I did:

$ wget http://memcached.googlecode.com/files/memcached-1.2.8.tar.gz
$ tar xvfz memcached-1.2.8.tar.gz
$ cd memcached-1.2.8

If you read the README file in here you’ll see that memcached requires the libevent-dev package. No problem, we’ll use apt-get to install that:

$ sudo apt-get install libevent-dev

Now we can build memcached from source:

$ ./configure
$ make
$ sudo make install
$ cd ..

Now we are off to get cmemcache, which is the Python binding to libmemcache, a C API used to interact with memcached. This means we are going to have to first obtain and build libmemcache, and then obtain and build cmemcache, which will finally allow us access via Python. So head on over to http://gijsbert.org/cmemcache/.

Here is where it gets kind of weird. As I mentioned, cmemcache depends on libmemcache, which is apparently not maintained any more. The last change to libmemcache was in February of 2006. I sure hope it is still compatible with the latest version of memcached that I just installed. Well, let’s go ahead and try. Head on over to http://people.freebsd.org/~seanc/libmemcache/ to get libmemcache. Here is how I obtained the last version of libmemcache.

$ wget http://people.freebsd.org/~seanc/libmemcache/libmemcache-1.4.0.rc2.tar.bz2
$ tar xvfj libmemcache-1.4.0.rc2.tar.bz2

But wait, the cmemcache guy has a patch for libmemcache. Hmmm. Okay, well let’s get that and apply it. I didn’t have the patch command installed, so I show the apt-get command to obtain and install that too. Here then is how I got the patch, applied it, then configured and built the libmemcache library:

$ wget http://gijsbert.org/downloads/cmemcache/libmemcache-1.4.0.rc2.patch
$ sudo apt-get install patch
$ patch -p1 < libmemcache-1.4.0.rc2.patch
$ cd libmemcache-1.4.0.rc2
$ ./configure
$ make
$ sudo make install
$ cd ..

Phew. Okay, now we can go back and get cmemcache installed. Again, this is the Python binding to libmemcache, which will allow us efficient access to that C library:

$ wget http://gijsbert.org/downloads/cmemcache/cmemcache-0.95.tar.bz2
$ tar xvfj cmemcache-0.95.tar.bz2
$ cd cmemcache-0.95
$ sudo python setup.py install

Alright, we finally got everything installed. Now for some housekeeping. I want memcached to run whenever I boot my development box. So I’m going to create an init.d script to start memcached whenever I turn on my development server. I decided to use the very useful start-stop-daemon program to run memcached. This script will start memcached with default settings and will run it as the user nobody. You should customize it if you require a different port number or cache size. Save the following into a file called memcached:

#! /bin/sh
# init.d script for memcached; BGN 23 May 2009

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/memcached
NAME=memcached
DESC=memcached

test -x $DAEMON || exit 0

set -e

case "$1" in
  start)
	echo -n "Starting $DESC: "
	start-stop-daemon --quiet --exec $DAEMON --start -- -d -l 127.0.0.1 -p 11211 -u nobody
	echo "$NAME."
	;;
  stop)
	echo -n "Stopping $DESC: "
	start-stop-daemon --quiet --exec $DAEMON --stop
	echo "$NAME."
	;;

  restart|force-reload)
	#
	#	If the "reload" option is implemented, move the "force-reload"
	#	option to the "reload" entry above. If not, "force-reload" is
	#	just the same as "restart".
	#
	echo -n "Restarting $DESC: "
	start-stop-daemon --quiet --exec $DAEMON --stop
	sleep 1
	start-stop-daemon --quiet --exec $DAEMON --start -- -d -l 127.0.0.1 -p 11211 -u nobody
	echo "$NAME."
	;;
  *)
	N=/etc/init.d/$NAME
	echo "Usage: $N {start|stop|restart|force-reload}" >&2
	exit 1
	;;
esac
exit 0

Now we are going to copy this script into /etc/init.d and use the update-rc.d script to generate the crazy System-V style links needed to run our script at startup. I don’t really care about order here, so I just use the default settings.

$ sudo cp memcached /etc/init.d
$ sudo chmod 755 /etc/init.d/memcached
$ sudo update-rc.d memcached defaults

I’ll now start memcached manually:

$ sudo /etc/init.d/memcached start

So that was a lot of work, but it is finally running. However, one nagging issue remains in my mind. When I try to run the tests that are included with cmemcache, one of them fails. This is kind of disturbing. I’ve noticed at least one other blogger has run into this.

$ cd cmemcache-0.95
$ python test.py

So there you have it. There was a lot of hoops to jump through and some lingering questions in my mind about whether this is going to work right. It seems a little odd to me that this is the state of memcached for Python users, especially since it is so widely touted by the Django community. I have been using memcached with one of my Django powered sites, and it does seem to work. I just wish all the tests passed and that I wasn’t relying on apparently unmaintained software.

Edit: The route I took was to get to memcached access through cmemcache. There is another method, which doesn’t require so much compiling, and that is using the python-memcached API. So if you have more difficulty than I did with cmemcache, you could try that route. It seems to be better supported, but probably isn’t quite as fast as using cmemcache. It may work just fine for your application however.

Tags: cmemcache, daemon, django, libmemcache, memcached