The Bug

We use Subversion at work for revision control. My colleague and I were trying to merge a branch back to trunk when we ran into some strange behavior. We make use of Subversion properties, which allow you to attach arbitrary metadata to files and directories. Our project has to deliver our source code and documentation to the customer in a required directory format (can you guess who our customer is?). However not all files need to be sent to the customer. To solve this problem we use a simple "yes/no" delivery property on each file to control whether it is delivered or not. Before making a delivery, a script is run that prunes out the files that have the delivery flag set to "no".

When our merge was completed, many files were marked with having merge conflicts on the delivery property. Looking through the logs it was discovered that after we had made our branch, someone had changed the delivery property on some files to "yes" on the trunk. Someone else had also changed the delivery property independently to "yes" on the branch. When we attempted to merge the branch back to trunk, we were getting merge conflicts, even though we were trying to change the delivery property value to "yes" on both the trunk and branch. Why was this a conflict? This didn't seem quite right.

I signed up for the Subversion user's mailing list and made a short post summarizing our issue. I later learned that it is proper etiquette to attach a bash script that can demonstrate the problem. Despite this, a Subversion developer took interest in my post and created a test script in an attempt to reproduce our issue. At first it looked like he could not reproduce the problem. However another helpful user pointed out a bug in his script. Once this was fixed, the developer declared our problem a genuine bug and created a ticket for it in the issue tracker.

The Patch

Impressed by all of this, I thanked him for his efforts and tentatively asked if I could help. The developer told me which file and function he thought the problem might be in. I downloaded the Subversion source and began looking at the code. I was fairly impressed with the code quality, so I decided I would try to create a patch for the bug over the weekend. We really wanted that bug fixed, and I was genuinely curious to see if I would be able to figure something out. It would be an interesting challenge and a test of my novice open source skills.

When the weekend came I began a more thorough examination of the Subversion website. The Subversion team has done a great job in providing documentation on their development process. This includes a contributing guide and patch submittal process. I also discovered they had recently added a makefile that downloaded the Subversion source code and the source for all of Subversion's dependencies. The makefile then builds everything with debug turned on. Wow! It took me a few tries to get this working, but the problems were because I did not have all the development tools installed on my Ubuntu box. Once this was sorted, everything went smoothly, and in a matter of minutes I had a Subversion executable I could run under the gdb debugger. Nice!

I studied the code for about an hour, peeking and poking at a few things in the debugger. I used the script the developer wrote to recreate the problem. I wasn't quite sure what I was doing, as I was brand new to this code base. But the code was clearly written and commented well. My goal was to get a patch that was in the 80-100% complete range. I wanted to do enough work that a core developer would be able to see what I was doing and either commit it outright or easily fill in the parts that I missed. After a while I thought I had a solution and generated a patch. I sent it to the Subversion developer's mailing list as per the contributing guide.

The Wait

Next I began probably the worst part for a contributor. I had to wait and see if I got any feedback. On some open source projects a patch may languish for months. It all depends on the number of developers and how busy they are. My chances didn't look good as the developers were in the initial stages of getting a beta version of 1.7 out the door. It was also not clear to me who "owned" the issue tracker. On some projects, the issue tracker is wide open to the community. Was I supposed to update the ticket? I wasn't quite sure, and the contributing guide was silent on this issue. I eventually concluded I was not; it looked like only committers were using the tracker. Patches were being discussed on the mailing list instead of in the tracker. This is a bit different than some projects I am familiar with.

I didn't have to wait long. After a few days, the original developer who confirmed my bug took interest again. He looked at my patch, and thought I had missed something. He suggested a change and asked for my opinion. I looked at the code again; it seemed like a good change and I told him I agreed. I also warned him I was brand new to the code, and to take my opinion with a grain a salt. After running my change against the tests, he then committed my patch! One small victory for open source!

Final Thoughts

So what went right here? I have to hand it to the Subversion team. They have been in business a long time, and they have excellent documentation for people who want to contribute. The makefile they created that sets up a complete development environment most definitely tipped the scale for me and enabled me to create my patch. Without that I'm not sure I would have had the time or patience to get all that unfamiliar source code built. The Subversion team has really worked hard at lowering the barrier for new contributors.

My advice to people who want to contribute to open source but aren't quite sure how to go about doing it:

• Spend some time reading the documentation. This includes any FAQ's and contributor guides (if any).
• Monitor the user and developer mailing lists to get a feel for how the community operates. Each project has different customs and traditions.
• You may also wish to hang out on the project's IRC channel for the same reason.
• When writing on the mailing lists, be extremely concise and polite. You don't want to waste anyone's time, and you don't want to be seen as someone who thinks they are entitled to a fix. Just remember you are the new guy. You can't just barge in and make demands.
• Ask how you can help. Nothing makes a developer happier when someone asks how they can help. Remember, most of the people in the community are volunteers.
• Open source can sometimes be "noisy". There will be people who won't quite understand your issue and may hurriedly suggest an incorrect solution or give incomplete advice. Study their responses and be polite. You may also wish to resist the temptation to reply right away. This is especially hard when you are new and you don't know who the "real" developers are. However you should assume everyone is trying to help.
• Finally, be patient. Again, most folks are volunteers. They have real jobs, families and lives. The project may also be preoccupied with other tasks, like getting a beta out the door. Now may not be a good time for a brand new feature, or your bug may not be considered a show stopper to the majority of the community.

A big thank-you to Stefan Sperling from the Subversion team who shepherded my bug report and patch through their process.

I hope this story encourages you to contribute to open source software!

Who's What?

My website, like many others, has a "who's online" feature. It displays the names of authenticated users that have been seen over the course of the last ten minutes or so. It may seem a minor feature at first, but I find it really does a lot to "humanize" the site and make it seem more like a community gathering place.

My first implementation of this feature used the MySQL database to update a per-user timestamp whenever a request from an authenticated user arrived. Actually, this seemed excessive to me, so I used a strategy involving an "online" cookie that has a five minute expiration time. Whenever I see an authenticated user without the online cookie I update their timestamp and then hand them back a cookie that will expire in five minutes. In this way I don't have to hit the database on every single request.

This approach worked fine but it has some aspects that didn't sit right with me:

• It seems like overkill to use the database to store temporary, trivial information like this. It doesn't feel like a good use of a full-featured relational database management system (RDBMS).
• I am writing to the database during a GET request. Ideally, all GET requests should be idempotent. Of course if this is strictly followed, it would be impossible to create a "who's online" feature in the first place. You'd have to require the user to POST data periodically. However, writing to a RDBMS during a GET request is something I feel guilty about and try to avoid when I can.

Redis

Enter Redis. I discovered Redis recently, and it is pure, white-hot awesomeness. What is Redis? It's one of those projects that gets slapped with the "NoSQL" label. And while I'm still trying to figure that buzzword out, Redis makes sense to me when described as a lightweight data structure server. Memcached can store key-value pairs very fast, where the value is always a string. Redis goes one step further and stores not only strings, but data structures like lists, sets, and hashes. For a great overview of what Redis is and what you can do with it, check out Simon Willison's Redis tutorial.

Another reason why I like Redis is that it is easy to install and deploy. It is straight C code without any dependencies. Thus you can build it from source just about anywhere. Your Linux distro may have a package for it, but it is just as easy to grab the latest tarball and build it yourself.

I've really come to appreciate Redis for being such a small and lightweight tool. At the same time, it is very powerful and effective for filling those tasks that a traditional RDBMS is not good at.

For working with Redis in Python, you'll need to grab Andy McCurdy's redis-py client library. It can be installed with a simple

$ sudo pip install redis

Who's Online with Redis

Now that we are going to use Redis, how do we implement a "who's online" feature? The first step is to get familiar with the Redis API.

One approach to the "who's online" problem is to add a user name to a set whenever we see a request from that user. That's fine but how do we know when they have stopped browsing the site? We have to periodically clean out the set in order to time people out. A cron job, for example, could delete the set every five minutes.

A small problem with deleting the set is that people will abruptly disappear from the site every five minutes. In order to give more gradual behavior we could utilize two sets, a "current" set and an "old" set. As users are seen, we add their names to the current set. Every five minutes or so (season to taste), we simply overwrite the old set with the contents of the current set, then clear out the current set. At any given time, the set of who's online is the union of these two sets.

This approach doesn't give exact results of course, but it is perfectly fine for my site.

Looking over the Redis API, we see that we'll be making use of the following commands:

• SADD for adding members to the current set.
• RENAME for copying the current set to the old, as well as destroying the current set all in one step.
• SUNION for performing a union on the current and old sets to produce the set of who's online.

And that's it! With these three primitives we have everything we need. This is because of the following useful Redis behaviors:

• Performing a SADD against a set that doesn't exist creates the set and is not an error.
• Performing a SUNION with sets that don't exist is fine; they are simply treated as empty sets.

The one caveat involves the RENAME command. If the key you wish to rename does not exist, the Python Redis client treats this as an error and an exception is thrown.

Experimenting with algorithms and ideas is quite easy with Redis. You can either use the Python Redis client in a Python interactive interpreter shell, or you can use the command-line client that comes with Redis. Either way you can quickly try out commands and refine your approach.

Implementation

My website is powered by Django, but I am not going to show any Django specific code here. Instead I'll show just the pure Python parts, and hopefully you can adapt it to whatever framework, if any, you are using.

I created a Python module to hold this functionality: whos_online.py. Throughout this module I use a lot of exception handling, mainly because if the Redis server has crashed (or if I forgot to start it, say in development) I don't want my website to be unusable. If Redis is unavailable, I simply log an error and drive on. Note that in my limited experience Redis is very stable and has not crashed on me once, but it is good to be defensive.

The first important function used throughout this module is a function to obtain a connection to the Redis server:

import logging
import redis

logger = logging.getLogger(__name__)

def _get_connection():
    """
    Create and return a Redis connection. Returns None on failure.
    """
    try:
        conn = redis.Redis(host=HOST, port=PORT, db=DB)
        return conn
    except redis.RedisError, e:
        logger.error(e)

    return None

The HOST, PORT, and DB constants can come from a configuration file or they could be module-level constants. In my case they are set in my Django settings.py file. Once we have this connection object, we are free to use the Redis API exposed via the Python Redis client.

To update the current set whenever we see a user, I call this function:

# Redis key names:
USER_CURRENT_KEY = "wo_user_current"
USER_OLD_KEY = "wo_user_old"

def report_user(username):
 """
 Call this function when a user has been seen. The username will be added to
 the current set.
 """
 conn = _get_connection()
 if conn:
     try:
         conn.sadd(USER_CURRENT_KEY, username)
     except redis.RedisError, e:
         logger.error(e)

If you are using Django, a good spot to call this function is from a piece of custom middleware. I kept my "5 minute cookie" algorithm to avoid doing this on every request although it is probably unnecessary on my low traffic site.

Periodically you need to "age out" the sets by destroying the old set, moving the current set to the old set, and then emptying the current set.

def tick():
    """
    Call this function to "age out" the old set by renaming the current set
    to the old.
    """
    conn = _get_connection()
    if conn:
       # An exception may be raised if the current key doesn't exist; if that
       # happens we have to delete the old set because no one is online.
       try:
           conn.rename(USER_CURRENT_KEY, USER_OLD_KEY)
       except redis.ResponseError:
           try:
               del conn[old]
           except redis.RedisError, e:
               logger.error(e)
       except redis.RedisError, e:
           logger.error(e)

As mentioned previously, if no one is on your site, eventually your current set will cease to exist as it is renamed and not populated further. If you attempt to rename a non-existent key, the Python Redis client raises a ResponseError exception. If this occurs we just manually delete the old set. In a bit of Pythonic cleverness, the Python Redis client supports the del syntax to support this operation.

The tick() function can be called periodically by a cron job, for example. If you are using Django, you could create a custom management command that calls tick() and schedule cron to execute it. Alternatively, you could use something like Celery to schedule a job to do the same. (As an aside, Redis can be used as a back-end for Celery, something that I hope to explore in the near future).

Finally, you need a way to obtain the current "who's online" set, which again is a union of the current and old sets.

def get_users_online():
    """
    Returns a set of user names which is the union of the current and old
    sets.
    """
    conn = _get_connection()
    if conn:
        try:
            # Note that keys that do not exist are considered empty sets
            return conn.sunion([USER_CURRENT_KEY, USER_OLD_KEY])
        except redis.RedisError, e:
            logger.error(e)

    return set()

In my Django application, I calling this function from a custom inclusion template tag .

Conclusion

I hope this blog post gives you some idea of the usefulness of Redis. I expanded on this example to also keep track of non-authenticated "guest" users. I simply added another pair of sets to track IP addresses.

If you are like me, you are probably already thinking about shifting some functions that you awkwardly jammed onto a traditional database to Redis and other "NoSQL" technologies.