But then yesterday I was checking various links on my site, and one link in particular generated a traceback from my middleware: “AttributeError: ‘WSGIRequest’ object has no attribute ‘user’.” What? Why was this happening on this one particular view and only this view?

One of the first things I do in my process_response function is check to see if the user is authenticated:

if request.user.is_authenticated():
    ...

After putting a breakpoint in this code with pdb, I observed that indeed the request object my middleware received had no “user” attribute. This had me stupified until I suddenly realized that the link I had clicked on was of the form /xxx/yyy instead of /xxx/yyy/ due to a typo in my template. I forgotten the trailing slash. Aha, this was a clue, but I still couldn’t piece it together, and I had to get some sleep. I don’t know about you, but I hate going to sleep when you have a nagging unsolved problem.

Tonight I tackled the problem again. I remembered I have the APPEND_SLASH setting in my settings.py file set to True. This normalizes all my URLs such that they should all end in a trailing slash. This is implemented itself via a piece of middleware, the so called “Common” middleware which is baked into Django itself. I knew that there was some interaction going on between the CommonMiddleware and my own middleware. After reviewing the middleware docs to refresh my memory on how all this works, I turned my attention to my MIDDLEWARE_CLASSES setting:

MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'debug_toolbar.middleware.DebugToolbarMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'gpp.core.middleware.InactiveUserMiddleware',
    'gpp.core.middleware.WhosOnline',
    'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
)

Because of the order above, the first piece of middleware to process a request is the CommonMiddleware. When it observed the lack of a trailing slash, it returned a HttpResponseRedirect object to redirect the user to the correct URL with a trailing slash. This immediately killed the the process_request chain of middleware processing. Thus, the AuthenticationMiddleware never got to run, and this is the middleware that attaches the user object to the request. Furthermore, since the request processing ended, we now start the process_response processing. This works through the above middleware list backwards. So when my WhosOnline middleware gets called, it is suddenly presented with a request object that has no user attribute. And bam, I hit my bug.

To fix this problem, one could reorder the list of middleware. But finding a middleware ordering that will satisfy all of your installed middleware is a kind of a maddening process that requires flipping through many doc pages. It just appeared to be too brittle of a solution. I’ll probably add some middleware in the future and completely upset the apple cart again. In the end, I decided to just guard against the case of not having a user attribute on the request object and just bail out early. In this use case, it isn’t important to see every single response. I can simply make it up on the next one. So I modified my code to look like this:

def process_response(self, request, response):
    """
    Keep track of who is online.
    """
    # Note that some requests may not have a user attribute
    # as these may have been redirected in the middleware chain before
    # the auth middleware got a chance to run. If this is the case, just
    # bail out. We also ignore AJAX requests.
    if not hasattr(request, 'user') or request.is_ajax():
        return response
    if request.user.is_authenticated():
        ...

This was a difficult and unexpected bug. However it produced one of those rare, genuine “light bulb” moments as it forced me to take a hard look at Django’s middleware processing in detail and finally learn it front to back.

In any event, after reading the documentation, it wasn’t hard to implement caching for my purposes. The following code simply caches the last 16 requests. Once the cache is full, it is flushed completely and ready to be filled again. Simple, crude, but hopefully somewhat effective.  I’m not a Javascript expert, but this will work for me. I hope someone else finds it useful and can use it as a starting point for their use-case.

$(function() {
    var cache = {};
    var cacheSize = 0;
    $("#id_of_your_text_box").autocomplete({
        delay: 400,
        minLength: 2,
        source: function(request, response) {
            if (cache[request.term]) {
               response(cache[request.term]);
               return;
            }
            $.ajax({
                url: "/your/url/goes/here/",
                type: "GET",
                data: {
                    q: request.term,
                    limit: 10
                },
                dataType: "json",
                success: function(data, textStatus) {
                    if (cacheSize >= 16) {
                       cache = {};
                       cacheSize = 0;
                    }
                    cache[request.term] = data;
                    ++cacheSize;
                    response(data);
                },
                error: function(xhr, textStatus, ex) {
                    alert('Oops, an error occurred. ' + xhr.statusText + ' - ' +
                      xhr.responseText);
                }
            });
        }
    });
});

The key to making this to work is to provide your own source function to the autocomplete widget. The widget will call this function when it needs data. Inside the function we simply check to see if request.term is already in our cache. If it is, then we simply return the cached result via the response callback function. If not, we have to make an ajax call to the server to retrieve the data.  When the ajax call completes, we store the result in our cache and then return it to the widget.

Again, this code simply stores the last 16 requests. When we run out of room we simply empty the cache and start over. A more sophisticated algorithm could use a different strategy like removing only the first item put in the cache or even the “least recently used” cache item. Break out your college computer science textbook on caching strategies and go nuts here.

The other feature this code doesn’t do is provide matching capabilities. For example, if the user has typed “foo” in the input box, this code will not return results for “bar foo”. If you need this, I suggest looking at the caching example on the UI site, or even at Jörn Zaefferer’s source code.

Since I was using my webcam from Windows XP, I decided to write my script under the new 1.7.1 version of Cygwin that came out recently. You’ll need to get both the Paramiko package and the python-crypto package.

Happy New Year everyone!

So I’m not sure what shell=True does on Windows, and it certainly isn’t documented. I do not recommend using it unless you know what is happening  in the implementation. Interestingly, shell=False is the default. I should have left it that way.

Python leverages the GNU Readline Library in its shell code. The Readline library is used in a number of open source projects, notably the BASH shell. The Readline library looks for a configuration file at startup in a number of places. On Unix-like operating systems, the configuration file can be found at ~/.inputrc and falls back to /etc/inputrc.

One line is all you need in your ~/.inputrc file:

set editing-mode vi

Unfortunately I don’t know how to make this work under Windows.

Eventually as I got more and more excited about Python I decided to wholly embrace the customs of the community. But how could I easily tell my favorite editor Vim to use 4-space indents for just Python code? I still do a lot of C++ at work, and I still like 3-space indents in other file types.

My first attempt was to use a Vim modeline embedded right in my source files. At the bottom of my Python files I placed this line:

# vim: set sw=4 ts=4:

When Vim first opens a file, it scans the first and last few lines in the file, looking for modelines, and will use the settings just for that file.

This obviously worked, but soon I found it ugly and distracting to have that at the bottom of each file. And sometimes I would forget to put it there, and not realize it until a bunch of code had been written.

It turns out there is a far easier way, but it took some digging to discover. Vim has a very elaborate system of determing file types. If you peek into your Vim installation directory, you’ll see a ftplugin directory. This directory is full of little files that get executed once the type of a file has been determined. Some people on the Web suggest that you should modify these files directly. Thus I could modify the ftplugin/python.vim file, and place my tabstop and shiftwidth settings in there. But you would be modifying a Vim file, and your changes would get lost the next time you upgrade Vim.

There is a much better solution. After Vim executes the python.vim file in the ftplugins directory, it looks for a file called ~/.vim/after/ftplugin/python.vim (on Unix; on Windows I’ve had luck with C:\Progam Files\Vim\vimfiles\after\ftplugin\python.vim). If this file exists, it will be executed after the “official” python.vim file.

Inside your personal python.vim file, the documentation says to use the “setlocal” instead of “set” command. Thus, my file looks like:

setlocal tabstop=4
setlocal shiftwidth=4

Although to be honest, I’ve used the regular set for some time (by accident) without seeing any strange effects.

Happy Vimming!

I have used Vim on every programming task at every job I’ve had. I’m constantly learning new things about it, and the commands and keystrokes are now part of my DNA. I don’t even think how to execute commands anymore and I am often hard pressed to recall them to others learning Vim. I have to sit at a keyboard with Vim running in order to understand what my fingers are doing.

A while back, I discovered the auto-complete feature, where you type Ctrl-n in insert mode and Vim will attempt to complete the word you are typing based on all the words in all buffers and how frequently they occur. This is incredibly handy when programming to quickly pound out those function and variable names. I used “imap <S-Tab> <C-n>” in my .vimrc file to map this to shift-tab, which I felt was more convenient.

I also picked up a really handy way to cycle through all your buffers.  I used the tab and shift-tab mapping mentioned in that Vim tip. Combine that with Vim’s tab feature I can very easily find the file I am looking for.

And finally I had another big “wow, Vim can do that?” moment just last week. Since version 7, Vim has sported an “omni-complete” framework. Basically it is an architecture for people to hang code completion routines into Vim. In fact, right out of the box, Vim 7 has support for Javascript, CSS, and yes, even Python! Python completion is only available if you are using a version of Vim that has been compiled with Python support (e.g. on Ubuntu). Imagine my shock when I was editing a CSS file and I typed “display:” and then hit Ctrl-X Ctrl-O for the first time and a drop down list of all the possible values (inline, block, none, etc) appeared! No more hunting for that w3schools website. And the Python support is just as handy. Type “os.path.” followed by Ctrl-X Ctrl-O and you can see all the entities and function in the os.path module. I had no idea!

So all of this reminds me it is time to once again donate to the Vim project. Thanks Bram for all your amazing work you do on this invaluable tool.

P.S. I have no interest in the great editor holy war. Use a tool that you find intuitive and useful and learn it, back to front. I’m sure there is nothing wrong with other popular editors. But for me, Vim is it. I can’t imagine how many hours of productivity it has given me over the years.