In a previous post, I blogged about building a "Who's Online" feature using Redis and Python with redis-py. I've been integrating Celery into my website, and I stumbled across this old code. Since I made that post, I discovered yet another cool feature in Redis: sorted sets. So here is an even better way of implementing this feature using Redis sorted sets.

A sorted set in Redis is like a regular set, but each member has a numeric score. When you add a member to a sorted set, you also specify the score for that member. You can then retrieve set members if their score falls into a certain range. You can also easily remove members outside a given score range.

For a "Who's Online" feature, we need a sorted set to represent the set of all users online. Whenever we see a user, we insert that user into the set along with the current time as their score. This is accomplished with the Redis zadd command. If the user is already in the set, zadd simply updates their score with the current time.

To obtain the curret list of who's online, we use the zrangebyscore command to retrieve the list of users who's score (time) lies between, say, 15 minutes ago, until now.

Periodically, we need to remove stale members from the set. This can be accomplished by using the zremrangebyscore command. This command will remove all members that have a score between minimum and maximum values. In this case, we can use the beginning of time for the minimum, and 15 minutes ago for the maximum.

That's really it in a nutshell. This is much simpler than my previous solution which used two sets.

So let's look at some code. The first problem we need to solve is how to convert a Python datetime object into a score. This can be accomplished by converting the datetime into a POSIX timestamp integer, which is the number of seconds from the UNIX epoch of January 1, 1970.

import datetime
import time

def to_timestamp(dt):
    """
    Turn the supplied datetime object into a UNIX timestamp integer.

    """
    return int(time.mktime(dt.timetuple()))

With that handy function, here are some examples of the operations described above.

import redis

# Redis set keys:
USER_SET_KEY = "whos_online:users"

# the period over which we collect who's online stats:
MAX_AGE = datetime.timedelta(minutes=15)

# obtain a connection to redis:
conn = redis.StrictRedis()

# add/update a user to the who's online set:

username = "sally"
ts = to_timestamp(datetime.datetime.now())
conn.zadd(USER_SET_KEY, ts, username)

# retrieve the list of users who have been active in the last MAX_AGE minutes

now = datetime.datetime.now()
min = to_timestamp(now - MAX_AGE)
max = to_timestamp(now)

whos_online = conn.zrangebyscore(USER_SET_KEY, min, max)

# e.g. whos_online = ['sally', 'harry', 'joe']

# periodically remove stale members

cutoff = to_timestamp(datetime.datetime.now() - MAX_AGE)
conn.zremrangebyscore(USER_SET_KEY, 0, cutoff)

Comments

comments powered by Disqus