In any event, after reading the documentation, it wasn’t hard to implement caching for my purposes. The following code simply caches the last 16 requests. Once the cache is full, it is flushed completely and ready to be filled again. Simple, crude, but hopefully somewhat effective.  I’m not a Javascript expert, but this will work for me. I hope someone else finds it useful and can use it as a starting point for their use-case.

$(function() {
    var cache = {};
    var cacheSize = 0;
    $("#id_of_your_text_box").autocomplete({
        delay: 400,
        minLength: 2,
        source: function(request, response) {
            if (cache[request.term]) {
               response(cache[request.term]);
               return;
            }
            $.ajax({
                url: "/your/url/goes/here/",
                type: "GET",
                data: {
                    q: request.term,
                    limit: 10
                },
                dataType: "json",
                success: function(data, textStatus) {
                    if (cacheSize >= 16) {
                       cache = {};
                       cacheSize = 0;
                    }
                    cache[request.term] = data;
                    ++cacheSize;
                    response(data);
                },
                error: function(xhr, textStatus, ex) {
                    alert('Oops, an error occurred. ' + xhr.statusText + ' - ' +
                      xhr.responseText);
                }
            });
        }
    });
});

The key to making this to work is to provide your own source function to the autocomplete widget. The widget will call this function when it needs data. Inside the function we simply check to see if request.term is already in our cache. If it is, then we simply return the cached result via the response callback function. If not, we have to make an ajax call to the server to retrieve the data.  When the ajax call completes, we store the result in our cache and then return it to the widget.

Again, this code simply stores the last 16 requests. When we run out of room we simply empty the cache and start over. A more sophisticated algorithm could use a different strategy like removing only the first item put in the cache or even the “least recently used” cache item. Break out your college computer science textbook on caching strategies and go nuts here.

The other feature this code doesn’t do is provide matching capabilities. For example, if the user has typed “foo” in the input box, this code will not return results for “bar foo”. If you need this, I suggest looking at the caching example on the UI site, or even at Jörn Zaefferer’s source code.

Check out the screenshot of the event submit form:

The datepicker is very useful, and will add a lot to the site, I think. My existing PHP application is using a much less elegant form. So far this form only handles non-repeating events. I will add support for those later.

I then turned my attention to the backend: how to synchronize my event models with Google calendar? It turns out there is already a Django application designed for this purpose: django-cal. It is designed in a very generic way and seems to be very well done. I decided I didn’t need its full functionality, but I did reference the source code when designing my back end. So a big thank-you to the creators of django-cal!

At this point I have enough functionality to add events and get them on the Google calendar. I have coded, but not tested, update and delete functionality. I am using the batch facility of the gdata API. In a typical work cycle, I need to approve anywhere from 1 to 7 or so events at once. This isn’t a lot of events, but I think doing it in a batch mode will speed a synchronization cycle up noticeably.

I store a “status” field in each event that keeps track of the event status: new, new and approved, edited, edited and approved, deleted, deleted and approved, and “on calendar”. When a user adds an event it becomes “new”. An admin reviews the event, and marks it “new and approved”. After synchronization, the status changes to “on calendar”. Likewise, my plan is to let users edit and request deletes for any events that they submit. My custom admin synchronization view will the perform the operations on all events with the “… and approved” status in a batch mode.

The custom admin view I wrote about in the last entry is also working well. I overrode the base django.contrib.admin template, so it looks well-integrated with the rest of the admin part of the site. I also figured out how to use the breadcrumbs for admin navigation, and noticed the base template will display a variable called “messages” with the nice green checkmark, if present. Likewise, the admin CSS has a class called “errorlist” for displaying errors. So it is quite easy to make a pretty professional looking custom admin screen by studying the base template and admin CSS files. I will post a screenshot of my custom admin view after I get some more events through the work cycle in a future blog post.

The Python gdata API isn’t as clumsy as I first thought it was. I was able to wrap it in a Calendar class to make it easier to use. If you need to update or delete an event in batch mode, it seems like you have to retrieve the event from Google first. I struggled against this initially, since I was storing away the “edit link” for each event I create. I was hoping to use the edit link to avoid another trip to Google for an update. But after searching the Google Calendar Data API group archives, it seems that is indeed recommended to retrieve any event prior to an update or delete, at least in batch mode.

This seems to be working out well so far. After I code and test the other operations I will blog with more details.

1. Leveraging jQuery. I did make use of jQuery for the client side Javascript. This really did simplify things. In the old version of the code, I had a 282 line Javascript file. The new version weighs in at 161 lines, which is 57% the size of the earlier version.
2. Use of JSON encoder library. It turns out that Django includes a version of the simplejson library already. This is used in Django’s serialization feature. I probably could have used this feature, but I didn’t need to serialize an entire model, and wanted to rename some of the fields. So I simply did a “import django.utils.simplejson as json” and went to town. My server side code was now much simpler than the previous version where I encoded by hand.
3. I did continue to use the markItUp! editor to let users edit their message to go along with their location. I learned that the size of the editor is easily adjusted through the use of CSS. So I was able to override the default size of the editor by including my own CSS style sheet below the default sheet.
4. Reducing complexity. Here is where it got interesting. I decided to add a “json” field to the MapEntry model. This field is populated whenever a model object is saved by providing my own save() method. Inside my save() I use Django’s template render_to_string() to render the HTML  message that appears in the pop-up balloon on the map. This consists  of the user’s avatar followed by a message that is marked up in Markdown. I then build a JSON representation of the MapEntry using Django’s simplejson. This representation consists of the following fields: name, location, latitude, longitude, and the HTML message. By pre-saving all of this information, it makes it much more efficient to retrieve 200+ users’ information when the map is first loaded. In the view function, I make use of the values_list() queryset method to retrieve all the json fields. I don’t need the ORM to convert each MapEntry to an actual Python object.

Another thing I did differently for the port was when a user updates or adds their position on the map, I don’t reload the entire map. I just adjust their information to reduce bandwidth with an AJAX post. I’m not sure why I didn’t do this before, perhaps because it was to tedious to do without jQuery. It was just easier to reload the whole thing.

This was all well and cool, doing all this pre-computation, but what happens when a user changes her avatar? Her entry on the map will likely have a broken graphic. No problem. Here I took advantage of Django’s signals. I attached a signal handler to listen for changes to the UserProfile model. Whenever the UserProfile is saved, my signal handler runs and re-saves the corresponding MapEntry to regenerate the JSON. Very slick.

And finally, another thing that I learned about Django was that simple_tags can have defaulted arguments. I added an argument to my avatar tag so that I could apply CSS to the generated HTML img tag if needed. In the Member Map pop-up balloon, it looked nicer if the avatar was floated to the left.

So all and all, again, I’m very happy with how easy Django makes writing an application like this. Using jQuery was also a big productivity booster.

Here are some screenshots. You can also see the new Blueprints CSS in action as well.

Member Map 1

Member Map 2

The existing application consists of the PHP-Nuke code, written in PHP, on the server side, and a fair amount of Javascript on the client side. All interaction with Google Maps is done through the Javascript code.

For the port to Django, I want to do a couple of things different.

1. Leverage jQuery in the client-side Javascript. The existing version was written before I was aware of Javascript libraries, and it is all “vanilla” Javascript. By using jQuery, the amount of code I have to write will be much smaller. And by taking advantage of jQuery, it should be more portable and browser robust.
2. On the server side, the existing application did not use any JSON encoder library. I simply constructed the JSON by hand. I can’t remember why I did this. I think it was partly because I didn’t know any better, and partly because the PHP version I had on the original hosting server may not have had it. I will explore what Python and/or Django options exist.
3. I’ll leverage the markItUp! editor that I’ve been using for comments to let the users enter Markdown comments. I’ve used this in the last couple of apps I’ve written and it will be easy to implement. The only concern with doing this is the user may be tempted to write a lot more than in the older app, and it may make for large “balloons”. We’ll see.
4. I want to make the new application less database and computationally intense than the older version. I don’t have any problems running the current version with 200 users, but I think I could do it a bit smarter this time around. This means I am going to try to save marked up and JSON encoded strings in the database rather than compute them on every page load. I am also wary of instantiating 200+ map entry model objects with the Django ORM (albeit in a loop) just to generate the JSON. Luckily Django lets you use raw SQL when needed. But in this case I think a simple use of the values() method will really be useful here.

So, those are my going in goals. I’ve already started working on this, and I’ve learned a lot so far. In the next post or two I will detail the progress and what problems I ran into.

The idea here is to add a delete link on the shouts that the user created as she is viewing the shout history. We will use the jQuery library to dynamically add click handlers to these links. The click handler functions will make an AJAX post to the server and request that a shout be deleted. The server view function will verify that the user sending the request actually created the shout in question. If so, the shout will be deleted from the database and then a response sent back to the client. This response will be then be used by our Javascript to hide the deleted shout. Thus we will have deleted a shout without refreshing the entire page.

The first thing I did was modify the shout history template to output a link underneath a shout if the current user authored that shout. This template (shoutbox/view.html) was presented in part 5. I’ll just show the new code inside the for loop that adds these links:

    {% ifequal user shout.user %}
        | <a href="#" class="shout-del" id="shout-del-{{ shout.id }}">Delete</a>
    {% endifequal %}

The important thing to note is our links don’t go anywhere, they have the CSS class “shout-del”, and we give them an id of “shout-del-xx” where xx is the id of the shout from the database.

I then modified the shoutbox_app.js document ready function to look for all links that have the class “shout-del” and I attach a click function to them. The click function asks the user via the Javascript confirm() function if they really want to delete the shout. If they confirm, I then use javascript regular expression magic to pull out the shout’s  primary key from the <a> tag’s id, and send a POST request to the server to delete that shout. The relevant snippet is shown below.

     $('.shout-del').click(function () {
         if (confirm('Really delete this shout?')) {
             id = this.id;
             if (id.match(/shout-del-(\d+)/)) {
                $.post('/shout/delete/', { id : RegExp.$1 }, function(id) {
                    id = '#shout-del-' + id
                    $(id).parents('tr').hide();
                    $('div.shoutbox-history table tr:visible:even').removeClass('odd');
                    $('div.shoutbox-history table tr:visible:odd').addClass('odd');
                    }, 'text');
             }
         }
         return false;
     });

We are posting to the URL /shout/delete, and passing along the shout ID in the id parameter. We also supply a callback function that gets called when the delete happens successfully. More on that later. Finally the click function returns false so that the default action of clicking on a link is not executed.

We now turn to the server, and examine the view function that handles the URL /shout/delete. You’ll notice similar code as presented in part 5′s edit view function. This time I have discovered Django’s classes that derive from HttpResponse, and I use them in the error cases.

def delete(request):
    """This view deletes a shout. It is called by AJAX from the shoutbox history view."""
    if request.user.is_authenticated():
        id = request.POST.get('id', None)
        if id is None:
            return HttpResponseBadRequest()
        try:
            shout = Shout.objects.get(pk=id)
        except Shout.DoesNotExist:
            return HttpResponseBadRequest()
        if request.user != shout.user:
            return HttpResponseForbidden()
        shout.delete()
        return HttpResponse(id)

    return HttpResponseForbidden()

We check to see if the user is authenticated before proceeding. We then retrieve the shout ID from the POST QueryDict. From the ID we retrieve the actual shout from the database using the Django ORM. And finally, if and only if the user making the request is the same user that created the shout, we delete it. If all of that went well, we send back a positive HTTP response that includes the ID of the shout that was deleted.

Now refer back to our Javascript, posted above.  In the callback function we receive the ID as text from the server. We then convert the numeric ID to the CSS ID and use jQuery to select the parent <tr> of our <a> tag, and hide it. Wow! That was a very compact way to do all of that work.

The remaining logic takes care of the table presentation. I used CSS to style every other row in the shout history table with a class called “odd”. Since we just hid one row, that scheme got messed up. Luckily, the compact notation of jQuery again comes to the rescue and we can quickly restyle all odd visible rows with our class “odd”, taking care to remove the style from the new even rows first.

And there we have a shoutbox written in Django and Javascript for the new version of my site. Not bad for my first attempt, and it certainly was much easier than I imagined. All in all, there is a lot less Python code than the existing PHP code, and we even get neat-o AJAX’y features that the old one didn’t have. So I think we will go with this for the first pass of the site. Once I get more comfortable with jQuery I may decide to redo the scrolling itself with it. We could also look into posting shouts to the database without a complete page refresh. There is always something to go back and refactor as you learn more about your tools and best practices. I hope someone found this quick series of posts useful.

The first thing I did was create a “shout history” view. This allows you to view the shouts in a full page context. In my views.py file I created this:

def view(request, page=1):
    """This view allows one to view the shoutbox history."""
    paginator = DiggPaginator(Shout.objects.all(), SHOUTS_PER_PAGE, body=5, tail=3, margin=3, padding=2)
    try:
        the_page = paginator.page(int(page))
    except InvalidPage:
        raise Http404

    return render_to_response('shoutbox/view.html', {
        'page': the_page,
        },
        context_instance = RequestContext(request))

Here I am using the DiggPaginator, which is a very nice paginator that gives you “Digg-style” pagination. I then pass the page object to the template which does most of the interesting work.

In order to get the edit-in-place funcitonality to work, I am leveraging the Jeditable jQuery plugin. In order to use this plugin, in my shoutbox/view.html template I include <script> tags to bring in both the jQuery javascript library and the Jeditable plugin code. I then loop over the shouts for the page and display them along with the avatar and links to the user that made the shout. You’ll also see me using the smilify fiilter I discussed in part 4.

{% extends 'base.html' %}
<pre>{% load avatar_tags %}
{% load smiley_tags %}
{% block custom_css %}
<link rel="stylesheet" type="text/css" href="{{ MEDIA_URL }}css/shoutbox_app.css" />
<link rel="stylesheet" type="text/css" href="{{ MEDIA_URL }}css/pagination.css" />
{% endblock %}
{% block custom_js %}
<script type="text/javascript" src="{{ MEDIA_URL }}js/jquery-1.2.6.min.js"></script>
<script type="text/javascript" src="{{ MEDIA_URL }}js/jquery.jeditable.mini.js"></script>
<script type="text/javascript" src="{{ MEDIA_URL }}js/shoutbox_app.js"></script>
{% endblock %}
{% block title %}Shout History{% endblock %}
{% block content %}
<h2>Shout History</h2>
{% if page.object_list %}
{% include 'core/pagination.html' %}

<div class="shoutbox-history">
<table>
{% for shout in page.object_list %}
<tr class="{% cycle 'even' 'odd' %}">
<th>
<a href="{% url bio-view_profile username=shout.user.username %}">{% avatar shout.user %}</a>
<a href="{% url bio-view_profile username=shout.user.username %}">{{ shout.user.username }}</a>
</th>
<td>
<div {% ifequal user shout.user %}class="edit" id="shout-{{ shout.id }}"{% endifequal %}>{{ shout.shout|smilify|urlize }}</div>
</div>
<br />
<span class="date">{{ shout.shout_date|date:"D M d Y H:i:s" }}</span>
</td>
</tr>
{% endfor %}
</table>
</div>

{% include 'core/pagination.html' %}
{% else %}
<p>No shouts at this time.</p>
{% endif %}
{% endblock %}

In order to get the edit-in-place working, I need to enclose the shouts the user can edit in a special div with the class “edit”. I do this inside the loop, adding the class attribute if the current user matches the shout user. I also give each of these divs an id of “shout-xx” where xx is the id of the shout being displayed. I thought about just using the shout id as the id, but thought this might clash with other page elements, so I prefixed it with “shout-”. This adds a tiny bit of work back in the view code, but not much.

Now that I have “tagged” the divs the user can edit, I configure the plugin with the following bit of jQuery magic in the shoutbox_app.js file:

$(document).ready(function() {
     $('.edit').editable('/shout/edit/', {
         loadurl : '/shout/text/',
         indicator : 'Saving...',
         tooltip   : 'Click to edit your shout...',
         submit : 'OK',
         cancel : 'Cancel'
     });
 });

This code instructs jQuery to find all elements with the class “edit” and call the editable() function on them. The first parameter to editable() is the URL to send the edited text to when the user hits enter or clicks OK. In our case, this is /shout/edit. I’ll show this view in a minute. The next parameter to the editable() function is a javascript options object with several interesting options:

• loadurl – this is the URL that Jeditable will call to retrieve the text to edit. Recall that our text is smilified, so we don’t want the user edited the XHTML markup. We want the user to edit the original shout text.
• indicator – after saving the edit, the Jeditable plugin will display this text to give the user feedback something is happening while it posts the edited text to the server.
• tooltip – Jeditable will display this text as a tooltip when the user hovers their mouse over the div.
• submit – Provides the text for a submit button.
• cancel – Provides the text for a cancel button.

Holy crap that was easy. Now all we need to do is provide those 2 view functions. One to receive and save the edited text, and one that is called to retrieve the original shout. First, here is the /shout/view view function that retrieves the original text.

shout_id_re = re.compile(r'shout-(\d+)')
def text(request):
    """This view function retrieves the text of a shout; it is used in the in-place
    editing of shouts on the shoutbox history view."""
    if request.user.is_authenticated():
        m = shout_id_re.match(request.GET.get('id', ''))
        if m is None:
            return HttpResponse(u'')
        try:
            shout = Shout.objects.get(pk=m.group(1))
        except Shout.DoesNotExist:
            return HttpResponse(u'')
        return HttpResponse(shout.shout)

    return HttpResponse(u'')

Jeditable is expecting that this function return the raw shout, not XHTML markup. It will pass the id of the div the user wants to edit as a GET parameter. First we check to see if the user is authenticated as a minimal check. Note that we don’t use the login_required decorator, because that would redirect the Jeditable request to the login page if the user wasn’t logged in. We then use a regular expression to pull the numeric part out of the div’s id (recall that it comes in the form shout-xx). With the id in hand we simply query the shout object using the Django ORM. We then return the shout text as an HttpResponse.

Finally here is the view function that Jeditable posts the edited data to.

def edit(request):
    """This view accepts a shoutbox edit from the shoutbox history view."""
    if request.user.is_authenticated():
        m = shout_id_re.match(request.POST.get('id', ''))
        if m is None:
            return HttpResponse(u'')
        try:
            shout = Shout.objects.get(pk=m.group(1))
        except Shout.DoesNotExist:
            return HttpResponse(u'')
        if request.user != shout.user:
            return HttpResponse(u'')
        new_shout = request.POST.get('value', '').strip()
        if new_shout == '':
            return HttpResponse(u'')
        shout.shout = new_shout
        shout.save()
        return render_to_response('shoutbox/render_shout.html', {
            'shout': shout,
            },
            context_instance = RequestContext(request))

    return HttpResponse(u'')

This function does do a lot of error checking, so it looks more complicated than it really is. This time Jeditable is expecting us to return XHTML markup, and it passes 2 things to us as POST parameters: the div id appears as “id” and the new text appears as “value”.

In short, this function checks to make sure the user posting the new shout text is the same user that created the shout in the first place. If this is the case, it simply updates the shout text and saves it back to the database. We then render a template to return the new XHTML markup for the div. Remember that we have to smilify the text, so our template looks like this:

{% load smiley_tags %}
{{ shout.shout|smilify|urlize }}

We load our smiley tags discussed in part 4, and filter the shout through our smilify filter and the Django urlize filter.

One thing that is very useful to add at this point is some CSS for our editable div tags. I added a :hover rule that changed the background color and mouse cursor. This provides additional visual cues to the user that the text can edited when the mouse is hovered over it.

And that is basically it! Here is a screen shot of a user editing a shout in-place on the shout history page.

Edit-in-place in action

I plan on adding one more blog post about the shoutbox. All we have left to do is an in-place delete of a shout on the shout history page. Stay tuned for part 6.